If Unicode weren't broken it wouldn't need any "normalization functions".
And no, I don't have anything better to propose at the moment. Unicode is a mess, but currently without realistic alternatives.
Still I would like to live long enough to see that we finally manage to create some proper text format without all kinds of crazy gotchas and deficiencies.
Are you suggesting that a better version of Unicode does not allow combining characters? [1]
Some languages have an explosive amount of characters if you regard each possible modifier in combination with each base character, some of which aren't practically used. By allowing the composition of modifiers and base letters, all the historically used ones are available, and all the odd ones are technically expressible.
I'm not sure how you encode all the world's languages, live and dead ones, without a few gotchas and deficiencies.
What would an always-normalized Unicode look like, if not either having really, really many characters, or having a non-trivial syntax?
One solution to allow combination characters, but only in a normalised way is to only allow one of the same modifiers, and only allow modifiers in the normal order. This makes parsing Unicode quite complex. If you allow for non-normalised Unicode and write a post-processor that normalises, you get to status quo.
So either you are suggesting to disallow non-normalised Unicode with the current definition, or something different altogether. I can't imagine what that alternative looks like.
(Also, I'm a Unicode fanboy. Sorry for the intensity.)
I think normalization and compound glyphs are orthogonal concepts.
The point would be to have only one canonical binary representation. A representation that is also free of all kinds of "compromises" which are only there for legacy reasons…
Additionally the whole madness should be resolved that Unicode mixes content, representation, and layout (especially as it fails miserably at all of them).
Than there are the problems that Unicode is actually incapable of representing all kinds of scripts. (Just think for example about stuff written top-to-bottom, and not LTR / RTL). Some "unimportant" things like Math can't be represented in Unicode also…
And I won't even talk about the issue that the committee was taken over by some political movement, namely the woke fraction.
I think it's actually a kind of joke that we still didn't mange to invent some sane text format for computers. Unicode doesn't cut it.
But frankly I admit that we won't get anywhere — as even Unicode is full of madness it works "good enough" by now through even more layers of madness on top, so we won't get rid of it ever again. That ship sailed I fear. Still I can't stop banging my head when I hear about things like issues with normalization, something that shouldn't exist in the first place in a sane world.
