Eh. That position is mathematically flawed in a way that is likely making you more pessimistic about tests than is deserved.
When you select a population by a criteria the selection itself eliminates the correlation.
So for example if you only accept high LSAT scores to be lawyers you should expect to find LSAT scores to be largely uncorrelated-- the correlation has already been removed. In fact, you may find it be be _inversely_ correlated because the parties with weaker scores that still made it through had other things going for them that made them successful. And this is true even if the test performance is HIGHLY predictive considering the whole population.
And this is clearly true: if nothing else LSAT tests the ability to understand written English and the kinds of basic logical reasoning that are absolutely required in the law. There are some people who do poorly at those tasks and would score very poorly at the LSAT, and would be huge liabilities if they became lawyers.
Now, perhaps those low performers would be filtered out by other criteria ultimately (say the bar exam) and so the LSAT may be redundant in that sense (except for saving huge costs and time for people who would ultimately flunk out...).
All that said, it's common for tests to get overweighed because they're the number we have. Would you prefer the lawyer that has the 5pt higher LSAT score but never worked a case like your vs one who successfully handles them all the time? Obviously the latter! but many other predictors are often not available and ones that are available are often not reducible to numbers or are situation specific.
In any case reasoning from post selected statistics has produced some disastrous decisions in business. Sadly, most people are not in the position to conduct a controlled study most of the time and so the stats you get are always tainted by post-selection effects.
I know someone whose work conducted basic coding tests on applicants. They found that scores on the test didn't predict performance (or were even somewhat anti-predictive: the people who nailed the tests were sometimes fakers who had managed to study for the test). So they eliminated it and then suffered disaster after disaster. The newer hires were generally not as good at their jobs, entirely contrary to the expectations from their prior results.
When you select a population by a criteria the selection itself eliminates the correlation.
So for example if you only accept high LSAT scores to be lawyers you should expect to find LSAT scores to be largely uncorrelated-- the correlation has already been removed. In fact, you may find it be be _inversely_ correlated because the parties with weaker scores that still made it through had other things going for them that made them successful. And this is true even if the test performance is HIGHLY predictive considering the whole population.
And this is clearly true: if nothing else LSAT tests the ability to understand written English and the kinds of basic logical reasoning that are absolutely required in the law. There are some people who do poorly at those tasks and would score very poorly at the LSAT, and would be huge liabilities if they became lawyers.
Now, perhaps those low performers would be filtered out by other criteria ultimately (say the bar exam) and so the LSAT may be redundant in that sense (except for saving huge costs and time for people who would ultimately flunk out...).
All that said, it's common for tests to get overweighed because they're the number we have. Would you prefer the lawyer that has the 5pt higher LSAT score but never worked a case like your vs one who successfully handles them all the time? Obviously the latter! but many other predictors are often not available and ones that are available are often not reducible to numbers or are situation specific.
In any case reasoning from post selected statistics has produced some disastrous decisions in business. Sadly, most people are not in the position to conduct a controlled study most of the time and so the stats you get are always tainted by post-selection effects.
I know someone whose work conducted basic coding tests on applicants. They found that scores on the test didn't predict performance (or were even somewhat anti-predictive: the people who nailed the tests were sometimes fakers who had managed to study for the test). So they eliminated it and then suffered disaster after disaster. The newer hires were generally not as good at their jobs, entirely contrary to the expectations from their prior results.