The main problem is that you need to have contributors sign a copyright assignment/CLA, otherwise their code is going to be AGPL only and you cannot license it commercially.
Or you don't have any contributors, which is the base case, I guess.
And, you'll trigger the same response in potential contributors: There's a generally anti-CLA attitude in open-source/free software circles because it means if you contribute your contributions can be used to enrich someone else.
One could write patches and refuse to sign the CLA. The maintainer would be unable to incorporate those patches into the repository without losing the ability to relicense.
Maybe it would be useful to reframe the CLA as the price of centralized maintenance. It's free software so it's perfectly possible to refuse to sign the CLA, modify the software regardless and even publish the changes. It just means the software must be forked and maintained separately.
Or you don't have any contributors, which is the base case, I guess.