That's a pretty easy track though - a high contrast shape that wasn't changing shape (only position in the frame) that much.

When it did change i.e. due to perspective and angle to the camera, the tracker lost it.

Planar tracking would allow tracking with perspective and size changes.