This is what has me most excited. AI has it's limited uses for now but with the current requirement of handing over all your data to big brother it was not even worth considering. Now that on prem is reasonable and doesn't require you to beg Nvidia for H100s it might actually be usable.
It's virtually impossible to not hand over data now. I'd say at this point it's more likely that any attempt to draw less attention actually has the opposite effect in reality.
The reality is that many datasets include stuff that has been absolutely stolen or haphazardly stored online, stuff like emails, texts, conversations, our credit histories, really any information about people that exists in a large quantity has already made its way into these LLMs one way or another.
An AI trained solely/additionally on specific datasets, with the intent of predicting human behavior exactly, should yield successful and highly accurate predictions from rather general demographic data combined with an individuals location history/current their location, plus really any third substantial thing - could be browsing history, or purchases, activity on social media, that should be all that's necessary to rather accurately put everyone into a box.
The most frustrating thing is that there is a box for the people that refuse to be put into a box.