For everybody to have maximal rights, you have to prohibit taking away others' rights. The GPL just codifies this: we give you the right to use this however you like as long as you don't take that right away from others.
You can draw a parallel to physical goods: everybody has the right to do whatever they want with whatever they've bought. However, this does not translate to being able to sell that thing to somebody else while restricting what that person can do with it. You do not have the right to restrict what people do with things you used to own.
I think there are two possible assumptions (or axioms) here.
Suppose there's a work X licensed under the GPL. You've assumed that if I create a derivative work Y, then everyone else has an automatic right to Y. If I were to restrict Y, I'd be taking away this right, and so the GPL stops me from restricting it. In fact, it severely limits what I can do with my code, but enforces everyone else's freedom to use my code.
Let us assume the opposite: that a work does not belong to the public domain until its creator chooses to share it. Then any work X that has already been shared is fully protected (by any free software license). If I create a derivative work Y and keep it secret, I in no way limit people's use of X, so I have not violated any of their rights. Similarly, if I make a restrictive license on Y, I still haven't violated their rights. They never yet had a right to Y, only to X. In this case, the GPL is violating my rights to do as I choose with my personal work, Y.
So we have two opposing assumptions and two different conclusions: If you think a work you create immediately "belongs" to the public, then the GPL protects people's rights; if you think it "belongs" to its creator until shared, then the GPL violates people's rights.
I myself am in the second camp, so I think that the GPL does not technically qualify as a fully free software license. Perhaps the distinction could be phrased this way: I believe that a potential author of a piece of code should be able to do whatever they want with it, while the GPL is predicated upon the belief that all potential -users- of a piece of code should be able to do whatever they want with it.
But I don't think you can have both. You either have to allow restriction of the users' options (if the author so chooses), or you have to restrict the author's options.
You can draw a parallel to physical goods: everybody has the right to do whatever they want with whatever they've bought. However, this does not translate to being able to sell that thing to somebody else while restricting what that person can do with it. You do not have the right to restrict what people do with things you used to own.