I guess the easiest way to measure this would be to run some gamepad test program on the target device, and then to record a video from your phone close to the gamepad but still with the screen visible. Then flick a button hard with your finger so that the gamepad button makes a click sound. Then load the video into a video editing software, zoom-in, and measure the number of video frames between the click sound and the reaction on the screen.

I agree this is the way to do it, but for an old cheapo gamepad, "it feels fine" was good enough for me at this stage.