I’m still not at all sure about the “billions” invested claim. How much of that is cloud running the models? How much is pre and post training (which may or may not be part of what we’d want to include in accounting). Etc. Does anyone have links to good reporting about this: not blind recitations of numbers, but analysis and thought mixes with investigation?

This is very interesting: it is a lot like the GLP (pun intended) solution: it doesn't try to "cure" the symptom, it just piggybacks onto the existing facts-on-the-ground and tries to make things better from there.

I'm not totally sure I understand the "distribution/registries" thing though? This also related to GitHub and HF, right? And Netflix. (You mentioned Spotify.) How they host stuff and then leverage that position to build add-ons and lock-in, lobby for laws. Companies PAY for this "bandwidth."

I always thought that BitTorrent would eventually take off and make these kinds of sites irrelevant, that it would democratize bandwidth, the last leg of the battle for universal accessibility.

I like your idea. I wish you luck.

We. Are. FUCKED.

THIS is where the regulation needs to start.

Regulations won't be set (serious ones, at least) unless there's some risk to those holding power. Which is the opposite in this case: this tracking helps them to take even more control over society and individuals.

But politics is when people who feel differently try to do something about it

Any regulation will be geared toward protecting the profits of the AI companies, and not toward consumer protection.

Maybe it's time you all voted for people who might change that? It's really amazing to me how on the one hand people in the US seem to crow about democracy all the time, yet also just accept as a fact that their government will never actually work to help them.

I threw money behind getting Bernie nominated over Biden--none of this shit is my fault.

Let's imagine that a model is pulled from HF by order of the new overlords or because of some other kind of censorship. Wouldn't the question of it having a Free license or not potentially become a complex legal issue?

But if the point is to be "censorship-free" then why respect licenses at all? They are among main choke points today. If authoritarians use licenses to censor political, artistic, scientific, etc., speech that they want to block, does that make the censorship more respectable?

When Anthropic sues a Chinese lab for IP infringement and get a court to put a bar on that software, does it THEN get pulled from Pirate Face?

I know that an awful lot of international negotiations have become focused more and more on questions of "IP" - licensing battles are already intensely politicized and it's hard to imagine a future where it doesn't get much much worse. Imagine N Korea coming after you for violating a license that they worked hard to control and leverage.

Can you explain this a bit to a non-expert?

I haven't wrapped my mind around this

This is the original description of abliteration and it's quite approachable and interesting to read: Refusal in Language Models Is Mediated by a Single Direction (https://arxiv.org/abs/2406.11717). Warning: changes to your world view caused by seeing "HarmBench" used to maximise expected harm instead of minimising it may be irreversible.

There's an empirical observation that models often have a single direction in their activation space for "hmm no I shouldn't do this". It forms naturally during pre-training, and is then surfaced during post-training to make the model refuse to engage in certain behaviour.

With a little bit of linear algebra you can zap that direction from the model's activations, and it stops refusing to do things. You can also do the opposite: magnify that direction, and the model refuses to do anything at all.

I'm pretty sure this was achieved with prompting rather than with weights, but there is a chatbot available that tries to maximize the motivated refusals:

https://www.goody2.ai/chat

Damn what's happened since this? Presumably they scramble refusal intentionally somehow now? Like intentionally couple it to "directions" that effect performance if messed with? Or is it more like just don't rely on the model to refuse and instead capture bad responses between generation and delivery?

I've seen attempts to obfuscate the refusal direction, like here: https://arxiv.org/html/2608.18093v1

Also this one was interesting, training the model to give preambles with reasons for the reasons for refusal seems to make it less sensitive to modulating the single refusal direction: https://arxiv.org/html/2505.19056v1

My empirical observation is that when a new model is released on HuggingFace, an abliterated version with < 10/100 refusals (baseline usually 100/100) is uploaded the same day, so either these techniques don't work very well or the open-weight labs aren't applying them.

There's some defense-in-depth, like a lot of the "guardrails" people hit on cloud models are classifiers applied to prompt or output, not a refusal generated by the model. Also closed-weight models obviously try to avoid this by not letting you see or modify the weights.

I'm not active in this space but why do you think anything happened since this? As far as I know, it still works.

Instead of editing the weights so they don't create the refusal signal, just let them do whatever, then delete the refusal signal itself. You don't want to edit quantised weights because it causes a loss of precision that can be pretty bad.

When we sing, draw, speak, play, we are not attacking the soul of someone else!

Culture has always had a kind of violence to it, but not FROM the THIEF: you HAVE to speak the language that was already made; you have to understand and participate in their song-structures, their plot devices, their ways of attesting.

The "IP owners" should be paying ME!!!

Hasn't this crossed over into being a philosophical question? You want to attest to reality or provenance. But then you start making lists of "acceptable" changes: file format; compression; color-correction; size; taking "medium" into account... It then slowly slides into "average human perceptibility"; "irrelevant details"; keeping the "spirit" of the image intact; judging the intention or motive of the user or of the viewer; aligning the image with "values"; appropriate/legal use... etc. Not sure what the end-game is?

We are trying to make images in the AI era be as reliable as they were pre-AI? But they were not reliable pre-AI, it was just more difficult to intentionally doctor them.

I feel like we were already in a weird gray area. Cellphone cameras use all kinds of programmatic tricks to make their output better than the exact information the lens captures. I think people just didn’t realize how much their phones were doing. Is that “fake”?

Yes.

Any capturing device is only every going to offer a symbol of the light that traversed it's aperture.

Even a camera obscura with a strip of film inside contains some adjustment to the "actual" image. If I expose the film for longer/shorter or during differing weather conditions, or near a train, etc, the developed film is going to have an appearance that may tinge the viewers perception of the subject. A short exposure, slightly off tilt, during a partially cloudy day, and a 'proper' exposure, rightly aligned, during full sun at noon, both of the same subject, both taken as soon as the photographer could to meet the deadline, both published on the front page of a newpaper, will give readers wildly different tastes for the subject.

Even two different b/w films will have different brightness/contrast etc, and the color films vary even more, with some punching various red/blue/green levels.

That's all without necessary deliberate intervention of the photographer. Once you get in to artistic license, you've got an even wider range. Taken from a low angle to add gravitas, taken from a high angle to minimize chin and highlight chest, with special lights to deepen shadows, with/without normal/stage makeup to add anything from regality to poverty. Taken in one's very nicest clothing, with family that is never around, while everyone has a congenial expression. (Aww, look at this lovely family)(look at the grand barren desert monument off in the wild dunes that is clearly not littered with tourists and a short walk from downtown cairo) This is easily noticeable if you've seen both a wedding and it's final professional photos. The lights didn't look like that. Where'd the audience around their first dance go? Holy shit, she looked good, but not that good.

Many have often taken photography as if it offers a genuine view of a thing at a time. We even sometimes call historical representations "snapshots". But it's never been the case.

Not sure I'd really call any photo a fake per se, they're all just representations. I suppose the fakeness comes from outside the image: the framing. If one edits a picture of a park and says "come to Always Sunny Perfect Awesomeville Where it Never Rains and is Always Full of Butterflies", then one has lied as it definitely rains there and there's certainly not always butterflies. One producing a deepfake is doing the same: capturing something they'd like captured. If it is represented as a photograph, then, sure, there's a lie; there was no photograph. But both the image with the clothes on (with lighting, editting, and makeup) and without are the same "here's what I thought it should look like" rather than "here's an exact transcript of the actual arrangement of this particular band of the electromagnetic spectrum that would have met my eye at the time"

This is so good. I think this is what Hegel meant by the "absolute" - not some final God's-eye view that he's often caricatured for, but the fact that every objective "substance" is and will always be wrought by "subject", by a POV, a mediation, a representation.

I think for photojournalism it should be original raw image, as captured in camera — zero edits for colour correction/cropping/filesize etc — platforms can link a digital sig from a postprocessed version but you keep the provenance proof tied to a published original.

Are they allowed flash? What about studio lighting? What about commentary? Paid actors?

"Hey, do me a favor, wear this hoodie and go hand this plastic baggie with white powder in it to that guy over there, it's for an art project, he's going to act like he doesn't know you, and then you just run in the opposite direction to where you'll be payed"

Or "here's three photos of XYZ taken on three seperate days at his new favorite coffee shop, notably across from this elementary school. Read on for our take on why he likes this shop so much!"

Or even something as simple as positioning the camera just so that one national head appears shorter than another. Or, in an interview, adjusting lights on the 'hero' to make them look good and highlight the shadows on the 'villain' to make them look ominous.

Use a mirror set up to make it look to the 'villain' that they are making eye contact but are instead looking weirdly askance.

Limitations are the origin of creativity. Force publishing RAW would just force journalists to be more creative in their framing.

Also, no edits for filesize? Lossless RAWs are huge! Especially from professional cameras. You could end up loading a gig of data for a single article with 10 images, or even more for larger frames. The few who care enough to see the 'real' image (that is still tainted by photographer intent regardless of file size and editting) are motivated enough to click the lossy jpg for the raw. Those who do not care aren't going to sit waiting on the order of minutes to see the first image.

> Are they allowed flash? What about studio lighting? What about commentary? Paid actors? > > "Hey, do me a favor, wear this hoodie and go hand this plastic baggie with white powder in it to that guy over there, it's for an art project, he's going to act like he doesn't know you, and then you just run in the opposite direction to where you'll be payed"

what exactly is the 'threat model' that you're trying to protect against here? If you already don't trust John Doe as a credible source, of course then there's a million ways he can frame a scene to tell an unreliable story. That's always been the case.

But to me, what we're looking for now in the age of diffusion models is for someone to be able to say "here is the provenance of this image, it came from John Doe's camera" and that can survive being shared round the internet, provably.

> You could end up loading a gig of data for a single article with 10 images, or even more for larger frames.

I'm not suggesting that the _only_ thing that gets used is a raw image. I'm saying that if you go to the Wall Street Journal, then underneath the photo, beside the credit, they put a link out to some page that hosts the RAW file, the metadata of the camera used and a signature.

If you find the photo unbelievable and think it's bogus, you're able to regenerate the signature from the parts and see that it's valid. (and obviously if they link off to something that's massively different - that isn't just compression, white balance, etc - people hold them to account)

Or what about: the photographer tries to hide so that they don't contaminate the real-life that they are watching, but then the person sort of hears a small sound from the photographer's direction -- they don't actually SEE the photographer, but they get interrupted.

Or: the CCTV is well hidden, but the people who installed it left behind some mud that got on someone's shoes and that made them late...

That's pretty much how the Apple one works. The raw sensor data is signed. But you have to process it because raw sensor data is unviewable.

Provide the raw original, and the exact processing steps used to get the presentable one.

ok, point camera at a screen. let's counter that by also taking 360° image with secondary camera. Ok build some props, let's add gps into it. fake gps. let's add inertial tracking and self-destruct on tamper. Ok, but we really need a fake image, here is our badge, give us signing keys for special use case...

You cannot replace honesty with tech.

no no no you need to buy a new phone!

/s

Would like: * Local web page interface or even browser UI element (since extension needed anyway) * Ability to add notes to history * Flag if bookmarked, allow filtering "bookmarks only" * Keep old versions of pages * Human-readable text diff vs current live page