If you remove the mechanically preventable bugs from being a consideration, by definition the only one you now need to focus on are the ones NOT prevented by mechanism.
How is this not a win? We only have so many decisions we can make per day.
So, let's take the most extreme example of pursuing safety, WUFFS.
Unlike general purpose languages WUFFS has a very specific purpose (Wrangling Untrusted File Formats Safely) so it doesn't need to worry that it can't solve some of your problems, which frees it to completely refuse to do stuff that's unsafe, while going real, real fast.
WUFFS gets to completely omit bounds checks, which you probably wouldn't have dared try in C because it's so obviously unsafe, but WUFFS already proved at compile time that it can't ever have bounds misses so it needn't do these checks. WUFFS gets to also omit integer overflow checks, because again it proved at compile time that your code is correct and cannot overflow for any input. And since WUFFS knows how big the data is at all times it gets to automatically generate loop unrolling and suchlike accelerations.
WUFFS isn't a general purpose language, it doesn't have strings, it doesn't have growable arrays (what C++ calls "vectors"), it doesn't have any dynamic memory allocation, but then we weren't talking about how much you love general purpose features, we were talking about safety preventing security bugs. Which is exactly what WUFFS does.
I can't respond to that unless you are more specific about what are "the ones that lead to really bad outcomes". There are a lot of arbitrary-code-execution attacks that are enabled by buffer overflows, and IMHO that's as bad as it gets. There is absolutely no legitimate excuse for a buffer overflow in today's world. Switching to safe(r) language won't prevent all attacks, but it will make a big dent.
It defeats some extremely important classes of exploits. And I'm not sure how they're not ones that lead to really bad outcomes since they lead to fun ones such as arbitrary code execution all the time. I can create a C program with hideous vulnerabilities in about five minutes without doing anything that isn't totally standard and normal (albeit obviously vulnerable so technically buggy). I'd have to actually look up how to make my code vulnerable in languages with more safety features.
Yes, they can defeat a few classes of exploits, but generally not the ones that lead to really bad outcomes.