In fairness, I wouldn't offer the pedantry of a prescriptionist who was moved to create a video as evidence that the average person doesn't get the sarcasm of "I could care less". I think it's rather more likely that the average person couldn't care less whether the phrase is literally correct, as long as the listener or reader understands its meaning.
I'm sure most people would see the original non-literal features of the phrase if they were to think about it. The point is, they don't! Not because they're dumb but because the entire expression has unit status in their vocabulary. The fact that people do not notice the original non-literalness in the phrase (and indeed understand it as intended) is evidence that it's not non-literal anymore.
I'm harping on about this because it's such a nice poster child for an entrenched (conventionalized) meaning of an entire expression as opposed to just a word, and for the lack of componentiality of meaning in language. In other words, there's more to the meaning of a sentence than just the meaning of its words. Componentiality is one of the points of debate between different schools of thinking in linguistics.
I suppose I just have a hard time getting too exercised about what is, essentially, a banal artifact of a highly idiomatic language. When I find myself getting bogged down over a particular expression, I step back and ask myself: if person A uses this expression, will person B understand what they mean? Really, this is all that really matters.