This is, legally, the correct interpretation of the difference between "stealing" and "infringement", at least in the US.
However, the copyright owner is being deprived of their exclusive right to the content. They have the right to control who uses the content and for what (exclusive of fair use).
Sure. And that's why civil and criminal law provide remedies for wronged content owners.
But no-one's arguing that copying isn't wrong or a civil and possibly even criminal act. I'm explaining the distinction between infringement and theft, as it's far more relevant than the ultimately unknowable "whether or not the unjustly enriched party may have bought a license".
However, the copyright owner is being deprived of their exclusive right to the content. They have the right to control who uses the content and for what (exclusive of fair use).