No. The gravity of the Earth doesn't do any work on the probe. It's a coupling mechanism. The work that is done is by Earth's orbital energy around the Sun. That is the energy that is less after the spacecraft encounters the Earth than before. The Earth is "dragging" the spacecraft along with it around the Sun. Not trebucheting it.
The gravity of the Earth absolutely does net work done on the probe. That’s an unambiguous mathematical statement. It’s just that this work is frame dependent. It sounds like you want it to frame-independent and to be given by the value in the center-of-mass frame. But it’s not.
> The gravity of the Earth absolutely changes the speed of the probe.
but instead
> The gravity of the Earth absolutely changes the characteristic energy with respect to the sun of the probe.
Would you be happy?