It works great! I agree that we're unusual in the weight that we put in our references, but I've never had it be a blocker with an applicant. We run every reference through the same script of questions, for a call that tends to last about 30 minutes. Since many engineers often talk in interviews about what their team did, versus their own contributions, the third-party viewpoint is very helpful for getting that perspective. We've found references to be helpful in differentiating between average, good, great, and exceptional individual performance, and extremely helpful in understanding a candidate's teamwork. I personally find it vastly more useful than whiteboard coding and work samples for predicting how a candidate will actually perform.

Aside from their usefulness in making hiring decisions, as a manager, it has been great for jump-starting my relationship with my new hires with context on how they have worked in the past. That has been really helpful.

I agree with you that references have some limitations. We do, of course, expect that applicants will cherry-pick their references to make themselves shine. Yet, I typically get very candid feedback from the reference providers. I think most people simply aren't built to straight-up lie for someone else, even if they are a friend. References are typically someone in some level of authority someplace else, and they put their word on the line. Most people seem to take that seriously.

Another limitation is that people often can't use their current boss as a reference, and for people on an upward career trajectory, that may exclude someone who is capable of talking about the applicant's greatest career achievements. This was the case for me when I applied to my current role.

Lastly, we have to keep in mind that any single reference is colored by the biases and personality of its provider. And some applicants simply have access to better referrers than others for reasons out of their control. All we can do is make judgment calls when it comes to these things.