Fair enough. While I would kind of wish AI could be reliably detected, deep down I know this is impossible and it would be pretty bad if we had, say, a prosecution that succeeded because "this 'provably-non-AI' photo places you at the scene of the crime" because only a few underground people know how to remove a watermark.

You raise an interesting point about the artificial generation of evidence used in court. In 1992, Michael Crichton wrote the book Rising Sun, which centers around the editing of security camera video footage to coverup a murder.

I also wonder if being able to prove that an image or video isn’t AI generated would lend credence to it, while in reality there are other methods to produce falsified video.

Well, you just have to convince the jury. The defense attorney will try to throw up all the reasons it could be falsified, but the prosecutor will say "All of that is unlikely - the defense attorney would like you to believe that farfetched story, but this is still a compelling piece of evidence." This is how it always is with any kind of testimony and evidence.

Not necessarily. Knowing an image for sure is fake has value, even if you can’t guarantee the reverse is true.

What stops someone from adding a watermark to an actually photographed (carefully framed?) picture to discredit it? There is no certainty either way, just suggestions from someone else about what the truth might be.

No need to theorize, this is already happening.

If you want to discredit an imagine, upload a slightly ai-edited copy of it.

For C2PA and exif, these aren't watermarks, just metadata. You could already remove them with exiftool.