Or perhaps not taken from the taxpayers in the first place?
[Edit: I am actually fairly left of center politically, but this does not mean that I am completely uncritical of what a government does with taxpayers money - and the last Labour government went insane with so many stupid projects and initiatives that we will have to live with for decades.]
It's a matter of perspective, there are shrill voices saying that if we scrap Trident all our economic woes will be over, but really there are much, much lower hanging fruit than that. A windfall tax on the big IT outsourcing firms would be a good start.
All that power and responsibility, and not being compensated for it could be dangerous. Wouldn't it deter anyone who wasn't wealthy from office, and/or make them vulnerable to corruption?