This is more due to the inability of promotion committees to capture the usefulness of scientific work than due to the uselessness of this for science. Sure, if you have nothing else to show for yourself, it might be bad for your career. But I feel like people who work in academia already forego many career opportunities. That being said, you still need to put food onto the table.
But posterity will remember you. Today, Bach is considered one of the most prolific musical composers of history. But back when he lived, he didn't receive that much fame. In fact, after his death, people forgot about him. And Bach lived in a time before copyright. He used existing themes by other people and rearranged them for the instruments he used. Many of those folks might have been famous back then, but now many their names are only known to historians (exceptions exist, e.g. Vivaldi).
It's worth noting that the consensus in the MathOverflow thread seems to have been that it was a complete waste of time to spend any further time on that paper and the proof. That's the opinion that they reached based on the general issues with the proof itself, and not any promotion committee decisions: specifically the fact that multiple mathematicians tried to read the paper in detail, in seminars, to understand it, and reported that they failed. That would certainly do it!
I don't have an opinion myself on whether it was a mistake to spend more time on it, but you can clearly see professional mathematicians deciding not to work on something "important" for the legitimate reason that they thought it was unsalvageable. If that's correct, then a promotion committee should penalize people for doing it anyway.
But posterity will remember you. Today, Bach is considered one of the most prolific musical composers of history. But back when he lived, he didn't receive that much fame. In fact, after his death, people forgot about him. And Bach lived in a time before copyright. He used existing themes by other people and rearranged them for the instruments he used. Many of those folks might have been famous back then, but now many their names are only known to historians (exceptions exist, e.g. Vivaldi).