I'm not a lawyer, but I think we're entering into a legal gray area. There are the existing frameworks of copyright, parody, free speech, slander, libel, etc. that are all somewhat tangential to this.
I believe (I'm not certain) that celebrity voice impersonation is legal as long as it is not used to sell or endorse a product.
Most models are trained on the original speaker's voice, but maybe only a little bit. Models might incorporate learning from many speakers. We might even be able to boil down a speaker representation to a small vector encoding in the future. It'll be interesting if we can capture the representation of a person with just a few numbers.
I don't think the legislature should be overly protective against machine learning. It seems obvious to me that neural networks will play a huge role in creating entirely virtual musicians and influencers. We're already seeing this start to happen. r9y9 on github has published some models that rival Vocaloid in lyrical ability.
At the same time, we don't want these techniques used to commit fraud, slander, or have them be used to falsely accuse someone of committing some act. These are things we might need new legal protections for.
But I don't know what I'm talking about. I'm not a lawyer.
I asked not because I expected an answer, but because I figured you'd have an insightful opinion.
It's essentially the performance of a composition vs the composition question again: at what point am I mimicking someone to the extent they have a valid claim on a portion of my work?
I expect it'll enter the courts a few milliseconds after someone clones a dead actor (without their estate's permission) for a new performance.
There's always been an inherent tension in the US distinction between a law of nature and a creative work though. It seems a bit silly for me to claim patent / trademark on a vector that encodes my likeness.
I suspect there's some plausible deniability built-in that might allow for such matters to be legal.
For example, lots of people sound like Arnold Schwarzenegger. So if you trained a model with tall, deep-voiced Austrian man, you could probably get something that people will immediately associate with Arnold without actually being his voice, or someone emulating him. Because much of what Americans associate with his voice is really a regional accent which is relatively uncommon in the US.
There may be a little bit more difficulty getting away with with someone like Gilbert Gottfried, whose voice is much more unique. But I do think you could get away with creating a voice that people think sounds just like him, but doesn't hold up in a side-by-side comparison.
What I think will happen is celebrities like Morgan Freeman will use their voice to train models like this, then gift these to their estates for use in the future.
> So if you trained a model with tall, deep-voiced Austrian man, you could probably get something that people will immediately associate with Arnold without actually being his voice, or someone emulating him. //
I think "passing off", an unregistered element of trademark laws, may be pertinent here. If the public think that there's an association and you're knowingly trading on that, even if the public are wrong, then you can be 'passing off' your output as someone else's goods/services/[vocal renditions].
It's likely you'd have to be very careful about use of copyright material for training the voice (eg extracting metrics that describe the voice). Fair Use might apply in USA though (even commercially).
>"Most models are trained on the original speaker's voice, but maybe only a little bit."
Really cool that you got this to work. I used to work on TTS (a few years ago, now), and we trained on celebrity voices, but used full audiobooks. https://github.com/Kyubyong/tacotron
It's not settled caselaw so anyone basing a business off this should expect to spend a lot of money defending it in courts
I saw once a company that offered to be the sole purveyor of a celebrity's synthesized voice. I haven't been able to find them again, but that seems like a much safer way to monetize this.
E.g. training off Schwarzenegger and offering an Arnold transform