If you know little about prompt engineering and want to throw together a demo of something that kind of works extremely quickly, or experiment with an LLM agent exactly as it's defined in some paper, LangChain is pretty useful.
If you want to develop a real LLM application, you're probably better off skipping the library completely, or at least fully understand each abstraction to make sure it does everything you want before you decide you want to incorporate it.
I'm not sure if "helpful answer" as opposed to "answer" makes much of a difference in answer quality -- I'd believe it helps a little, just don't know that it's been studied -- but a lot of silly stuff like that does definitely make a big difference in response quality on certain tasks. "Let's think step by step:" at the end of your prompt is probably the best-known one: https://arxiv.org/pdf/2205.11916.pdf
Love it. Execute the scam, immediately send the money back (or better, somehow don't accept it in the first place) and tell them what happened with a link to an informative webpage. Might even be self-sustaining if you solicit donations from the victims concurrently with revealing the fake scam (though, uh, perhaps they'd be unusually wary about giving any amount of money right then and there to some organization they'd never heard of before).
ChatGPT-4 throws some cold water on the idea, naturally, though I still think there's something there:
====
me:
i have an idea for a non-profit foundation that would leverage LLMs to perform "online scam pentesting" at scale, as a public service. basically, we would execute pretend online scams very similar to real online scams that people are victimized by (fake invoices, romance scams, etc), but immediately send the money back if the pseudo-victim falls for it along with a brief explanation of what happened and our foundation's mission, along with a link to an informative webpage with helpful resources on online scams. (if the victim ignores the pseduo-scam, we'd perhaps wait a day and then followup with the same info.)
this seems to me like it'd be potentially legally fraught, though i'm not sure exactly how, or if there's any relevant legal precedent on the topic. what do you think?
ChatGPT:
I am not a lawyer, but I can provide some general thoughts on the potential legal challenges associated with your idea. It's important to consult with a legal professional for advice tailored to your specific situation and jurisdiction.
Consent: Executing pretend online scams without the consent of the individuals involved could potentially be a legal issue. Consent might be difficult to obtain without revealing the purpose of the exercise, which could defeat the purpose of the "pentesting."
Privacy: Collecting, storing, and processing personal information might raise privacy concerns, especially if the information is sensitive. Different jurisdictions have different regulations, such as GDPR in the European Union and CCPA in California, which govern the collection and processing of personal data.
Fraud: Even though your intention is to educate and protect people from scams, the act of sending fake invoices or pretending to be someone else could potentially be considered fraudulent behavior. This could open up your organization to legal liabilities.
Unintended consequences: If someone experiences financial, emotional, or other harm as a result of your "pentesting," they might choose to take legal action against your organization.
Regulation of LLMs: Depending on the jurisdiction, there might be specific regulations governing the use of LLMs, especially for activities that can be perceived as deceptive.
Precedent: While there may not be direct legal precedents for this type of activity, similar cases involving penetration testing and ethical hacking might provide some guidance. However, those cases typically involve consent from the targeted organization, which is a key difference from your idea.
To minimize potential legal risks, you could consider alternative approaches to educating people about online scams. This could include creating awareness campaigns, providing educational resources, or partnering with organizations that share your mission to fight online scams. Regardless of the approach you choose, it's essential to consult with legal professionals to ensure compliance with relevant laws and regulations.
Yeah, I think there would be a thin line to tread between being helpful and informative and being scammy. I would not want to actually go through with the scam and soliciting donations as an unknown orginisation seems spammy/scammy. I think it would probably be better to charge people to attempt to scam people they think may be vulnerable or themselves.
My mother got completely wiped out by a romance scammer, tens of thousands of dollars over >1 year, all via cash dumped into a Bitcoin ATM I believe. AFAIK she's still talking to the "guy", or she was when I broke off contact with her, after I and just about every other loved one, her bank, and the actual **ing FBI have all patiently explained many times the nature of the scam and exactly how we can tell it's a scam (beyond just "you've never met this dude or even had a video call and he's weirdly financially needy for someone who makes so much money") and sent her all sorts of articles and online resources on the topic and she reassured everyone that she recognized she'd been such a fool and would cut off all contact. And no, she wasn't displaying any other symptoms of dementia or Alzheimer's or anything like that that I could recognize.
I don't know if OpenAI/etc keep close enough tabs on usage that they'd be able to identify if they have people abusing their product for these types of scams, but even if they are I'm sure it's just a matter of time before the leaked LLaMA weights or similar are applied for this purpose, if it's not happening already.
Not a clue what's to be done collectively if law enforcement doesn't care beyond the FBI keeping an eye out for "more serious" money laundering stuff (I suspect that's why they actually care as an institution), but I can share a little practical advice that maybe you won't hear elsewhere:
1. If you're substantially financially supporting your parents or whoever else, you're entitled to and need to understand their budget in extreme detail, just like if you were married. They should be able to explain and prove all of their income and all of their expenses, and they should understand that your generosity is not entirely unconditional -- in the same way that you wouldn't be willing to enable, say, a heroin addiction, you also aren't willing to enable literal scams. (And, of course, they should understand why you need this info and what risk you're trying to protect them and yourself against.) This still doesn't protect them fully, since they can e.g. be persuaded to sell their home and send it all off in secret and only tell you about it once it's too late, or just lie and continue sending off small amounts of money over time. But, it helps limit your exposure, and if you're lucky will make them think twice about people they've never met in person soliciting money from them online.
2. If you see a loved one fall for any online scam, that should probably increase your estimation of their ongoing susceptibility to similar scams, even if they seem to "get it" after the fact in that specific case, even if you don't think of them as particularly gullible in general. Perhaps they'll learn and be more careful, or perhaps they won't and you're just learning new information about them that's unlikely to change.
3. Don't underestimate what even simulated love can make people do. My once-good relationship with my mother, who I never would've considered a "bad person" a few years ago and never displayed any antisocial tendencies to me before, was ruined by her repeated and pervasive deception and theft.
It's not quite the same, and might not be possible to make rigorous enough that it really proves anything, but something sort of similar would actually be practical to at least attempt in many cases. Stable Diffusion checkpoints of the same major version, along with other families of model weights, have the IMO fascinating property that you can do element-wise arithmetic with them, and the resulting model will actually sort of function like you'd naively expect. Recent paper on the topic (in LLMs, not diffusion models) here: https://arxiv.org/abs/2212.04089
So, if you take a Stable Diffusion checkpoint (call it "A") which is only lightly trained on some subset of an artist's work, then fine tune it on the full corpus of that artist's work to a point where it's still coherent/"good" and just shy of actually memorizing the fine tuning data (call the resulting model "B"), then define model "C" as 2A-B (i.e. A + (A-B), where A-B is the artist's task vector multiplied by -1), can you still produce qualitatively similar images with model C? Whether with the exact same prompt, or the same prompt with "in the style of Kinkade" removed (which doesn't mean as much if Kinkade's task vector was subtracted), or with any prompt whatsoever?
Lots of issues with this as laid out -- it's definitely not quite the same as "forgetting" Kinkade from the training data, and "any prompt whatsoever" introduces tons of leeway, and most good AI-assisted art is not just an unmodified single text-to-image output anyway -- but it might be a promising direction to explore.
(Strongly disagree with the "copyright laundry" characterization, by the way.)
Yes, whenever I spend hours refining a prompt in my stable diffusion UI and select one of hundreds or thousands of outputs and spend even more hours refining it with inpainting and other non-AI tools, I've felt nothing whatsoever, and my choices aren't informed in any way by anything interesting going on in my inner life.
If you struggle with it, if you are always wrestling with something not quite right, if it makes you nervous to share it, you are on the right track for sure!
I'm just saying, a guy like Van Gogh comes around just a few times a century, he is not somebody you can simply encode into a lot of different matrices, it takes a certain human life to make art like him, or even more simply, to simply see the world like him. Its not just about the way the sunflowers look, its about the choice in the first place to paint them, and making that choice in the particular context he was in, at the particular time in history he did. Its important that it was this painting that was painted after all the other paintings that had come before, both in particular for Van Gogh and for the world. And you can't ever recreate it, because its already done, and you can never step in the same river twice.
In general, and no offense to you in particular, but you will be forgotten as an artist. If this wasn't a certainty before, it is one now with all art generation and such. But don't take that as a bad thing, it is more than anything liberating. Try to remove all expectation and ego from what you are trying to make, and that will bring you closer to Van Gogh's work than any particular technique he used--your artwork should be completely personal, entirely internal, right up to the point that someone else see's it for the first time. It should only be judged on a sui generis rubric, more like a dream than a product. Stop, in general, trying to prove yourself on the basis of the tools you use. You could use sticks-and-twine or a million-GPU-DALL-E thing, whatever, its all orthogonal to the effect the final product might have, and that effect itself is only influenced by your attitude, your sincerity, and your vulnerability, as it is applied to the work.
(That is my maybe my main issue with the AI art bots, do the artists who work with them even feel vulnerable about their work? Can artistic sincerity and enthusiasm exist in AI art? Do AI artists feel the literally metaphysical stakes of what they are doing like Van Gogh did? Or is it all so constantly folded back into the idea of AI art itself? Is it all just different campaign posters for the cause of legitimizing itself as "real" art? Maybe just some time needs to pass, but I hope artists can still get nervous in the future, can still struggle at all to bring something out. I worry that people using this technology are too worried about proving themselves, they are not opening themselves up the possibility of failure that all artists need. They are not in a silent enough room to be able to hear what there brain is actually saying, only trying to respond respond respond. It is usually the struggle the artist has with themselves that is transmuted into the work such that it makes us cry or feel profound joy or sadness or whatever, we still need that psychic energy in the world, it doesn't need to be solved.)
To be like Van Gogh, you need to make art as if you would die and go to literal hell if you didn't, or go there if you made the "wrong" art. If you feel something like that, you are on the right track, but you still have 0.001% chance of dealing with that illness as productively as Van Gogh did.
The next Van Gogh, or the next Francis Bacon, or Rothko, or Cezanne, or whoever, will probably not have the machine-capturable-"style" of any of those people, but she will share in the particular mental complex they had, the same spiritual curse which drives one to reach past a scientific or practical world, to tap into something pure, almost completely formal, that cannot simply be taught or transmitted, much less encoded.
Sorry as well! Thanks for answering all my parenthetical rhetorical questions, definitely shows your not too defensive or anything about this stuff we are all, honestly, still navigating.
But I really really do wish you the best with this stuff, the future is bright and interesting either way for art!
You ever feel like this specific propaganda war is actually unwinnable? Many people are extremely motivated to bullshit the public (usually sincerely though I kind of doubt it in this case), and from I've seen, the public are far more willing to believe the 3 extremely online artists who they've heard an opinion on the topic from than the 1 software engineer/data scientist who actually knows half a thing about machine learning they've heard an opinion on the topic from, let alone the growing cornucopia papers and high-production-value websites that seem to say "it's just a plagiarism machine" if you don't know anything about the subject vs the approximately one website I've ever seen that says "no, you are being lied to".
I'd like to believe this isn't one of those things where we can only move on by everyone who believes the various correlated falsities dying, but I don't think I can.
Yeah, it's bullshit, but digging into a specific point from their FAQ:
> Usually, the image the model creates doesn’t exist in its training data - it’s new - but because of the training process, the most influential images are the most visually similar ones, especially in the details.
Would be cool if this were true, but I don't think it is, because the prompt you used and the captions on the training images are being completely ignored. If two different words tend to be used in captions for very visually similar images, and you use just one of those words in your inference prompt, I'm pretty sure the images that were captioned with the word you used are much more "influential" on your output than the images that were captioned with the word you didn't use. (Like, "equestrian" vs "mountie" or "cowboy" or something.)
Not to mention that the totality of all other images is in most cases probably more "influential" than the few most visually similar images! Consider the thought experiment:
1. Take the prompt you used, and use it with a model checkpoint that was trained identically to whatever model you're using, except that the top 21 images this website shows you are removed. In most cases, while your outputs won't be identical (I assume), you can probably get something pretty similar.
2. Now, take that same prompt, and use it with a model checkpoint that was only trained on the top 21 images this website shows you. (AFAIK you can't really do this because Stability hasn't released a "completely untrained" version of any of their models... though maybe they have and nobody cares because it's useless for most purposes.) I'm not completely sure what you'd get, but my bet would be that you get either nonsense or a memorized replica of one of the training images, not the same output image you got previously.
Sorry, but you're reading into noise. Anyone can reproduce an image anyone else made by only knowing the model checkpoint, positive and negative prompt, seed, sampler and sampling steps, &c &c they used. (Well, in principle, and usually in practice too. Interfaces might give different results now compared to a version from a few months ago because implementations of certain things changed, or if you use xformers then all your outputs are slightly non-deterministic, other exceptions that prove the rule like that.)
Some prompts I've come up with generate excellent and definitely novel results (without necessarily much work put into refining the prompt), others are extremely hard to get working well with hours of work even if I know it's something that isn't novel.
For some reason, we assume what comes out of a computer is more trustworthy than what people say. We think computers are transparent, reliable, idempotent and don't have an agenda. Even more so if we call it "intelligent"...
But ChatGPT is a bullshit machine, and that much is new.
At least the good part of the answers being on stack overflow then is, like they used to say "On the internet no one knows you're a dog". So whether the answer came from ChatGPT or an aggressively overconfident fool, a wrong answer should get the same downvotes regardless, and a correct answer should get the same up votes. Probably the two biggest issues with ChatGPT being used to provide answers is whether it's wrong often enough to start swinging the experience of the site negative, and more importantly that some people are getting fake internet points unfairly.
To the extent this perception exists -- and I don't think "came from a computer" falls within the top 5 actually effective methods of laundering bullshit nowadays, though maybe it used to -- you might expect that it gets crushed into dust as the public gets more exposure to high-profile counterexamples.
And, wait, isn't the concern usually that people read AI-generated content and trust it but don't think it came from a computer?
If you want to develop a real LLM application, you're probably better off skipping the library completely, or at least fully understand each abstraction to make sure it does everything you want before you decide you want to incorporate it.