Very interesting part around 5 mins in where Zuck says that they bought a shit ton of H100 GPUs a few years ago to build the recommendation engine for Reels to compete with TikTok (2x what they needed at the time, just to be safe), and now they are accidentally one of the very few companies out there with enough GPU capacity to train LLMs at this scale.
Are the export controls to China geographically or any Chinese majority-owned entity? Either way, ByteDance has tons of offices everywhere in the world including Singapore, US, etc. Given the money, I don't think GPU access wouldn't be their biggest problem.
The only thing the Reels algorithm is showing me are videos of ladies with fat butts. Now, I must admit, I may have clicked once on such a video. Should I now be damned to spend an eternity in ass hell?
It’s easy to populate your feed with things you specifically want to watch: watch the stuff you’re interested in and swipe on the things that don’t interest you.
Reels don't interest me, they are just showed in my face whenever I use Facebook (or should I say Face-butt?). It's impossible to hide without using a custom script/adblock, which I ended up doing, but the only long term, cross device solution is to simply to delete the Facebook account.
They can be useful. My feed is filled with startup advice as well as Chinese lessons. I think a big part of my Chinese vocab comes from watching reels teaching me chinese.
I could have saved them a lot of money by revealing to them that, yes, heterosexual men tend to gravitate towards ladies with fat butts.
I have a hunch that some of the more professional folks there game the algorithm. If you ever wanna find a place where people share algo optimization secrets, it’s OF creators on reddit.
Seems like a year or two of MMA has done way more for his charisma than whatever media training he's done over the years. He's a lot more natural in interviews now.
Alternatively, he’s completely relaxed here because he knows what he’s doing is genuinely good and people will support it. That’s gotta be a lot less stressful than, say, a senate hearing.
I've noticed the same thing! I think the personal confidence you build training hard MMA is a lot more empowering than the presonal confidence you build from making billions of dollars and being CEO of a gigantic company. For those of us without the money, it seems hard to believe, but people are people even when they're rich, and I've seen MMA change a lot of people in the same way.
There is something especially confidence building about training martial arts, I personally believe it adjusts our fight-flight response, which is also kicking in in social situations.
It’s not just training with other people but becoming used to receiving physical insult, it dampens our baseline fear of physical attack that we all feel in our factory default configuration.
People may not like Joe Rogan but he described BJJ perfectly: 3D chess with consequences. It is a great way to relieve stress and forces you to temporarily forget about whatever is bothering you that day.
That is a strength, not a weakness. It's valuable to see why people, even those with whom we disagree, think the way they do. There's already far too much of a tendency to expel heretics in today's society, so the fact that Lex just patiently listens to people is a breath of fresh air.
How? It's fine to have on people with all different viewpoints, including awful ones, but I think pushing back when they're on some bullshit is good and necessary. Otherwise you're just uncritically spreading fake junk to a huge audience, which leads to more people believing in fake junk.
The trouble is self-styled "both sides" types believe that since they take the both sides approach, they have insulated themselves from the kinds of politicization that compromises the extremes. But the manner in which you position yourself relative to those extremes is every bit as politicized and every bit as liable to the same cognitive biases and rationalizations.
Misinformed climate skeptics often regard themselves in this way, as not taking one side or the other on global warming. They mistakenly believe that this orientation has elevated them above equivalently offensive extremes, but in truth they have compromised their own media literacy by orienting themselves in that manner.
There are numerous instances of this all over the political spectrum, Cornell West talking to left-wing academics in left-wing academic language about how "nobody" thinks Obama is truly left-wing. Journalists during the Iraq war had a both sides approach that cashed out as extremely hawkish and apologetic in defense of the Iraq war.
The Lex Friedman version is a "centrist" in a specific kind of media environment that lends disproportionate visibility towards its own set of boutique topics. The combination of optimism about technology and trends especially around AI and crypto and some libertarian leaning politics surrounding it, which at its periphery finds itself disproportionately saturated by right-wing memeing and politics. And so it's a form of centerism that's in the center of a world as described by those things. But for him and his viewers it's something they consider a perfectly neutral state of nature that's free of any adornment of ideology.
I felt that way until he had Carlson on. Carlson is a grade A TV talking head grifter who just spins up sensationalist narratives to drive views. No background, no expertise, just a guy who mastered which buttons to push to get average joe's raging.
Lex says he wants open honest conversation, but Carlson was just doing the same stunningly dishonest grift he does every time he has a mic in front of him. So dumb.
I do have a few gripes though, which might just be from personal preference. A lot of the time the language used by both the host and the guests is unnecessarily obtuse. Also the host is biased towards being optimistic about LLMs leading to AGI, and so he doesn't probe guests deep enough about that, more than just asking something along the lines of "Do you think next token prediction is enough for AGI?". Most of his guests are biased economically or academically to answer yes. This is then taken as the premise of the discussion following.
Having said that, I do agree that it is much better and deeper than other podcasts about AI.
There's a difference to being a good chatshow/podcast host and a journalist holding someone's feet to the fire!
Dwarkesh is excellent at what he does - lots of research beforehand (which is how he lands these great guests), but then lets the guest do most of the talking, and encourages them to expand on what they are saying.
It you are critisizing the guest or giving them too much push back, then they are going to clam up and you won't get the best out of them.
I decided to listen to a Dwarkesh episode as a result of this thread. I chose the Eliezer Yudkowsky episode. After 90 minutes, Dwarkesh is raising one of the same 3 objections for the n-teenth time, instead of leading the conversation in an interesting direction. If his other AI episodes are in the vein as other comments describe, then this does seem to be plain old positive AGI optimism bias rather than some special interview technique. In addition, he's very ill-prepared in that he doesn't seem to have attempted to understand the reasons some people have for believing AGI to be a threat.
On the other hand, Yudkowsky was a terrible guest, in terms of his public speaking skills. He came across as combative. His answers were terse and he spent little time on background information or otherwise making an effort to explain his reasoning in a way more digestible for a general audience.
I think with any talk show it mostly comes down to how interesting the guests are. I kind of agree with you that Dwarkesh's steering of the conversation isn't the best, but he seems to put his guests at ease and maybe they are more forthcoming as a result. He is also obviously smart, and it seems that encourages his guests to feel compelled to give deeper/more insightful/technical answers than if they had been, say, talking to some clueless journalist. This was notable in his interview with Ilya Sutskever, who otherwise seems to talk down to his interviewers.
The main strength of Dwarkesh is the caliber of guests he is able to attract, especially for being so new to the game. Apparently he'll research a potential guest for a couple of weeks before cold e-mailing them with some of his researched questions and asking if they'll come on his podcast, and gets a very high acceptance rate since the guests appreciate the questions and effort he has put into it (e.g. maybe Zuck enjoying being asked about Augustus, and not just about some typical FaceBook fare).
If you were inclined to give him another try, then I'd recommend the Richard Rhodes or Dario Amodei episodes, not because of any great Dwarkesh interviewing skills, but because of what the guests have to say. If you are a techie then the Sholto + Bricken one is also good - for same reason.
As far as AI optimism, I gather Dwarkesh has moved to SF, so that maybe goes with the territory (and some of his friends - like Sholto + Bricken - being in the AGI field). While arguably being a bit too deferential, he did at least give some pushback to Zuck on AI safety issues such as Meta's apparent lack of any "safe scaling" tests, and questioning how Zucks "increased AI safety via democratization" applied to bio threats (how is putting capability to build bio weapons in hands of a bad actor mitigated by others having AI too).
I haven't listened to Dwarkesh, but I take the complaint to mean that he doesn't probe his guests in interesting ways, not so much that he doesn't criticize his guests. If you aren't guiding the conversation into interesting corners then that seems like a problem.
He does a lot of research before his interviews, so comes with a lot of good questions, but then mostly let's the guests talk. He does have some impromptu follow-ups, but mostly tries to come back to his prepared questions.
I struggle to blame people for speaking in whatever way is most natural to them, when they're answering hard questions off the cuff. "I apologize for such a long letter - I didn't have time to write a short one."
I think AGI is less a "generation" problem and more a "context retrieval" problem. I am an outsider looking in to the field, though, so I might be completely wrong.
I don't know Dwarkesh but I despise Lex Fridman. I don't know how a man that lacks the barest modicum of charisma has propelled himself to helming a high-profile, successful podcast. It's not like he tends to express interesting or original thoughts to make up for his paucity of presence. It's bizarre.
Maybe I'll check out Dwarkesh, but even seeing him mentioned him in the same breath as Fridman gives me pause ...
I mostly agree with you. I listened to Fridman primarily because of the high profile AI/tech people he got to interview. Even though Lex was a terrible interviewer, his guests were amazing.
Dwarkesh has recently reached the level where he's also interviewing these high profile AI/tech people, but it's so much more enjoyable to listen to, because he is such a better interviewer and skips all the nonsense questions about "what is love?" or getting into politics.
The question you should ask is: why are high-profile guests willing to talk to Lex Fridman but not others?
The short answer, imho, is trust. No one gets turned into an embarrassing soundbite talking to Lex. He doesn't try to ask gotcha questions for clickbait articles. Generally speaking "the press" are not your friend and they will twist your words. You have to walk on egg shells.
Lex doesn't need to express original ideas. He needs to get his guests to open up and share their unique perspectives and thoughts. He's been extremely successful in this.
An alternative question is why hasn't someone more charismatic taken off in this space? I'm not sure! Who knows, there might be some lizard brain secret sauce behind the "flat" podcast host.
Yes, of course. His guests love being able to come on and present their view with very little critical analysis of what they are saying. It is fantastic PR for them.
Interviewers shouldn't be aggressive, antagonistic or clickbaity but they should put opposing views to their guests so that the guest can respond. Testing ideas like this is a fundamental way of learning and establishing an understanding of a topic.
My earlier comparison was basically saying now that high-profile guests are talking to a much better interviewer (Dwarkesh), we no longer have to rely on Lex as the only podcast with long-form interviews of these guests.
I would have thought folks wouldn’t care less about superfluous stuff like “charisma” on HN and would like a monotone, calm robot-like man that 95% of podcast just lets their gust speak and every now and then just asks a follow-up/probing question. Thought Lex was pretty good at just going with the flow of the conversation and not sticking too much with the script.
I have never listened to Dwarkesh but I will give him a go. One thing I was a little put off by just skimming through this episode with Zuck is that he’s doing ad-reads in the middle which Lex doesn’t.
I'll agree that "interesting thoughts" may be up to interpretation, but imma fight you on the charisma thing. I looked up "flat affect" in the dictionary and there were no words, only a full-page headshot of Lex Fridman.
I'm simply pointing out the answer to your "I don't understand why people like him" question. If you can't understand why people don't share your hatred for something, then odds are that the disconnect is because they don't share your reasons for hating it.
Yeah, I'm a big fan of Lex because I think he is really good at building connections, staying intellectually curious, and helping peopl open up, but he is absolutely not big with charisma! I don't know if he normally talks so flat or not, but in the podcast I don't think he could be more flat if he tried. He's also not great at asking questions, at least not spontaneously. Seems really good at preparation though.
I listen to Lex relatively often. I think he often has enough specialized knowledge to keep up at least somewhat with guests. His most recent interview of the Egyptian comedian (not a funny interview) on Palestine was really profound, as in one of the best podcasts I’ve ever listened to.
Early on I got really fed up with him when I discovered him. Like his first interview with mark zuckerberg where he asks him multiple times to basically say his life is worthless, his huge simping to Elon musks, asking empty questions repeatedly, and being jealous of Mr Beast.
But yeah for whatever reason lately I’ve dug his podcast a lot. Those less good interviews were from a couple years ago. Though I wish he didn’t obsess so much about twitter
Neglected to include comparisons against GPT-4-Turbo or Claude Opus, so I guess it's far from being a frontier model. We'll see how it fares in the LLM Arena.
They didn't compare against the best models because they were trying to do "in class" comparisons, and the 70B model is in the same class as Sonnet (which they do compare against) and GPT3.5 (which is much worse than sonnet). If they're beating sonnet that means they're going to be within stabbing distance of opus and gpt4 for most tasks, with the only major difference probably arising in extremely difficult reasoning benchmarks.
Since llama is open source, we're going to see fine tunes and LoRAs though, unlike opus.
Not really that either, if we assume that “open weight” means something similar to the standard meaning of “open source”—section 2 of the license discriminates against some users, and the entirety of the AUP against some uses, in contravention of FSD #0 (“The freedom to run the program as you wish, for any purpose”) as well as DFSG #5&6 = OSD #5&6 (“No Discrimination Against Persons or Groups” and “... Fields of Endeavor”, the text under those titles is identical in both cases). Section 7 of the license is a choice of jurisdiction, which (in addition to being void in many places) I believe was considered to be against or at least skirting the DFSG in other licenses. At best it’s weight-available and redistributable.
I appreciate it too, and they're of course going to call it "open weights", but I reckon we (the technically informed public) should call it "weights-available".
It's impossible. Meta itself cannot reproduce the model. Because training is randomized and that info is lost. First samples a coming at random. Second there are often drop-out layers, they generate random pattern which exists only on GPU during training for the duration of a single sample. Nobody saves them, it would take much more than training data. If someone tries to re-train the patterns will be different, which results in different weight and divergence from the beginning. Model will converge to something completely different. With close behavior if training was stable. LLMs are stable.
So, no way to reproduce the model. This requirement for 'open source' is absurd. It cannot be reliably done even for small models due to GPU internal randomness. Only the smallest trained on CPU in single thread. Only academia will be interested.
Interesting. LLAMA is trained using 16K GPUs so it would have taken around a quarter for them. An hour of GPU use costs $2-$3 so training a custom solution using LLAMA should be atleast $15K to $1M. I am trying to get started with this thing. A few guys suggested 2 GPUs were a good start but I think that would only be good for 10K training samples.
Losers & Winners from Llama-3-400B Matching 'Claude 3 Opus' etc..
Losers:
- Nvidia Stock : lid on GPU growth in the coming year or two as "Nation states" use Llama-3/Llama-4 instead spending $$$ on GPU for own models, same goes with big corporations.
- OpenAI & Sam: hard to raise speculated $100 Billion, Given GPT-4/GPT-5 advances are visible now.
- Google : diminished AI superiority posture
Winners:
- AMD, intel: these companies can focus on Chips for AI Inference instead of falling behind Nvidia Training Superior GPUs
- Universities & rest of the world : can work on top of Llama-3
Google's business is largely not predicated on AI the way everyone else is. Sure they hope it's a driver of growth, but if the entire LLM industry disappeared, they'd be fine. Google doesn't need AI "Superiority", they need "good enough" to prevent the masses from product switching.
If the entire world is saturated in AI, then it no longer becomes a differentiator to drive switching. And maybe the arms race will die down, and they can save on costs trying to out-gun everyone else.
AI is taking marketshare from search slowly. More and more people will go to the AI to find things and not a search bar. It will be a crisis for Google in 5-10 years.
I think I agree with you. I signed up for Perplexity Pro ($20/month) many months ago thinking I would experiment with it a month and cancel. Even though I only make about a dozen interactions a week, I can’t imagine not having it available.
That said, Google’s Gemini integration with Google Workplace apps is useful right now, and seems to be getting better. For some strange reason Google does not have Gemini integration with Google Calendar and asking the GMail integration what is on my schedule is only accurate if information is in emails.
I don’t intend to dump on Google, I liked working there and I use their paid for products like GCP, YouTube Plus, etc., but I don’t use their search all that often. I am paying for their $20/month LLM+Google One bundle, and I hope that evolves into a paid for high quality, no ad service.
Only if it does nothing. In fact Google is one of the major players in LLM field. The winner is hard to predict, chip makers likely ;) Everybody jumped on bandwagon, Amazon is jumping...
I often use ChatGPT4 for technical info. It's easier then scrolling through pages whet it works. But.. the accuracy is inconsistent, to put it mildly. Sometimes it gets stuck on wrong idea.
Interesting how far LLMs can get? Looks like we are close to scale-up limit. It's technically difficult to get bigger models. The way to go probably is to add assisting sub-modules. Examples would be web search, have it already. Database of facts, similar to search. Compilers, image analyzers, etc. With this approach LLM is only responsible for generic decisions and doesn't need to be that big. No need to memorize all data. Even logic can be partially outsourced to sub-module.
It takes less than an hour of conversation with either, giving them a few tasks requiring logical reasoning, to arrive at that conclusion. If that is a strong position, it's only because so many people seem to be buying the common scoreboards wholesale.
That’s very subjective and case dependent. I use local models most often myself with great utility and advocate for giving my companies the choice of using either local models or commercial services/APIs (ChatGPT, GPT-4 API, some Llama derivative, etc.) based on preference. I do not personally find there to be a large gap between the capabilities of commercial models and the fine-tuned 70b or Mixtral models. On the whole, individuals in my companies are mixed in their opinions enough for there to not be any clear consensus on which model/API is best objectively — seems highly preference and task based. This is anecdotal (though the population size is not small), but I think qualitative anec-data is the best we have to judge comparatively for now.
I agree scoreboards are not a highly accurate ranking of model capabilities for a variety of reasons.
If you're using them mostly for stuff like data extraction (which seems to be the vast majority of productive use so far), there are many models that are "good enough" and where GPT-4 will not demonstrate meaningful improvements.
It's complicated tasks requiring step by step logical reasoning where GPT-4 is clearly still very much in a league of its own.
Disagree on Nvidia, most folks fine-tune model. Proof: there are about 20k models in huggingface derived from llama 2, all of them trained on Nvidia GPUs.
If anything a capable open source model is good for Nvidia, not commenting on their share price but business of course.
Better open models lower the barrier to build products and drive the price down, more options at cheaper prices which means bigger demand for GPUs and Cloud. More of what the end customers pay for goes to inference and not IP/training of proprietary models
Do they really need “free RLHF”? As I understand it, RLHF needs relatively little data to work and its quality matters - I would expect paid and trained labellers to do a much better job than Joey Keyboard clicking past a “which helped you more” prompt whilst trying to generate an email.
Variety matters a lot. If you pay 1000 trained labellers, you get 1000 POVs for a good amount of money, and likely can't even think of 1000 good questions to have them ask. If you let 1000000 people give you feedback on random topics for free, and then pay 100 trained people to go through all of that and only retain the most useful 1%, you get much ten times more variety for a tenth of the cost.
Of course numbers are pretty random, but it's just to give an idea of how these things scale. This is my experience from my company's own internal -deep learning but not LLM- models to train which we had to buy data instead of collecting it. If you can't tap into data "from the wild" -in our case, for legal reason- you can still get enough data (if measured in GB), but it's depressingly more repetitive, and that's not quite the same thing when you want to generalize.
Which indicates that they get enough value out of logged ~in~ out users. Potentially they can identify you without logging in, no need to. But also ofc they get a lot of value by giving them data via interacting with the model.
They also stated that they are still training larger variants that will be more competitive:
> Our largest models are over 400B parameters and, while these models are still training, our team is excited about how they’re trending. Over the coming months, we’ll release multiple models with new capabilities including multimodality, the ability to converse in multiple languages, a much longer context window, and stronger overall capabilities.
Anyone have any informed guesstimations as to where we might expect a 400b parameter model for llama 3 to land benchmark wise and performance wise, relative to this current llama 3 and relative to GPT-4?
I understand that parameters mean different things for different models, and llama two had 70 b parameters, so I'm wondering if anyone can contribute some guesstimation as to what might be expected with the larger model that they are teasing?
Right because the very little I've heard out of Sam Altman this year hinting at future updates suggests that there's something coming before we turn our calendars to 2025. So equaling or mildly exceeding GPT-4 will certainly be welcome, but could amount to a temporary stint as king of the mountain.
Yes, but the amount they have invested into training llama3 even if you include all the hardware is in the low tens of millions. There are a _lot_ of companies who can afford that.
Hell there are not for profits that can afford that.
Where are you getting that number? I find it hard to believe that can be true, especially if you include the cost of training the 400B model and the salaries of the engineers writing/maintaining the training code.
I mean anyone can throw out self evident general truisms about how there will always be new models and always new top dogs. It's a good generic assumption but I feel like I can make generic assumptions and general truisms just as well as the next person.
I'm more interested in divining in specific terms who we consider to be at the top currently, tomorrow and the day after tomorrow based on the specific things that have been reported thus far. And interestingly, thus far, the process hasn't been one of a regular rotation of temporary top dogs. It's been one top dog, Open AI's GPT, I would say that it currently is still, and when looking at what the future holds, it appears that it may have a temporary interruption before it once again is the top dog, so to speak.
That's not to say it'll always be the case but it seems like that's what our near future timeline has in store based on reporting, and it's piecing that near future together that I'm most interested in.
>We’re rolling out Meta AI in English in more than a dozen countries outside of the US. Now, people will have access to Meta AI in Australia, Canada, Ghana, Jamaica, Malawi, New Zealand, Nigeria, Pakistan, Singapore, South Africa, Uganda, Zambia and Zimbabwe — and we’re just getting started.
No EU initially - I think this is the same with Gemini 1.5 Pro too. I believe it’s to do with the various legal restrictions around AI which iirc take a few weeks.
LLM chat is so compute heavy and not bandwidth heavy that anywhere with reliable fiber and cheap electricity is suitable. Ping is lower than average keystroke delay for most who haven't undergone explicit speed typing training (we're talking 60~120 WPM for between intercontinental to pathological (other end of the world) servers).
Bandwidth matters a bit more for multimodal interaction, but it's still rather minor.
They just said laws, not privacy - the EU has introduced the "world's first comprehensive AI law". Even if it doesn't stop release of these models, it might be enough that the lawyers need extra time to review and sign off that it can be used without Meta getting one of those "7% of worldwide revenue" type fines the EU is fond of.
Am I reading that right? It sounds like they’re outlawing advertising (“Cognitive behavioural manipulation of people”), credit scores (“classifying people based on behaviour, socio-economic status or personal characteristics”) and fingerprint/facial recognition for phone unlocking etc. (“Biometric identification and categorisation of people”)
Maybe they mean specific uses of these things in a centralised manner but the way it’s written makes it sound incredibly broad.
Facebook has shown me ads for both dick pills and breast surgery, for hyper-local events in town in a country I don't live in, and for a lawyer who specialises in renouncing a citizenship I don't have.
At this point, I think paying Facebook to advertise is a waste of money — the actual spam in my junk email folder is better targeted.
Claude has the same restriction [0], the whole of Europe (except Albania) is excluded. Somehow I don't think it is a retaliation against Europe for fining Meta and Google. I could be wrong, but a business decision seems more likely, like keeping usage down to a manageable level in an initial phase. Still, curious to understand why, should anyone here know more.
The same reason that Threads was launched with a delay in EU. It simply takes a lot of work to comply with EU regulations, and by no surprise will we see these launches happen outside of EU first.
In the case of Switzerland, the EU and Switzerland have signed a series of bilateral treaties which effectively make significant chunks of EU law applicable in Switzerland.
Whether that applies to the specific regulations in question here, I don't know – but even if it doesn't, it may take them some time for their lawyers to research the issue and tell them that.
Similarly, for Serbia, a plausible explanation is they don't actually know what laws and regulations it may have on this topic–they probably don't have any Serbian lawyers in-house, and they may have to contract with a local Serbian law firm to answer that question for them, which will take time to organise. Whereas, for larger economies (US, EU, UK, etc), they probably do have in-house lawyers.
It is because of regulations. Nothing is trivial and anything has a cost. Not only it impacts existing businesses, it also make it harder for a struggling new business to compete with the current leaders.
Regulations in the name of the users are actually just made to solidify the top lobbyists in their positions.
The reasons I hate regulations is not because billionaires have to spend an extra week on some employee's salary, but because it makes it impossible for me tiny business to enter a new business due to the sheer complexity of it (or force me to pay more for someone else to handle it, think Paddle vs Stripe thanks to EU VATMOSS)
I'm completely fine with giving away some usage data to get a free product, it's not like everyone is against it.
I'd also prefer to be tracked without having to close 800 pop-ups a day.
Draconian regulations like the EU ones destroy entire markets and force us to a single business model where we all need to pay with hard cash.
> It is because of regulations. Nothing is trivial and anything has a cost. Not only it impacts existing businesses, it also make it harder for a struggling new business to compete with the current leaders.
But, in my experience, it is also true that "regulations" is sometimes a convenient excuse for a vendor to not do something, whether or not the regulations actually say that.
Years ago, I worked for a university. We were talking to $MAJOR_VENDOR sales about buying a hosted student email solution from them. This was mid-2000s, so that kind of thing was a lot less mainstream then compared to now. Anyway, suddenly the $MAJOR_VENDOR rep turned around and started claiming they couldn't sell the product to us because "selling it to a .edu.au domain violates the Australian Telecommunications Act". Never been a lawyer, but that legal explanation sounded very nonsensical to me. We ended up talking to Google instead, who were happy to offer us Google Apps for Education, and didn't believe there were any legal obstacles to their doing so.
I was left with the strong suspicion that $MAJOR_VENDOR didn't want to do it for their own internal reasons (product wasn't ready, we weren't a sufficiently valuable customer, whatever) and someone just made up the legal justification because it sounded better than whatever the real reason was
You didn't provide the source for the claim though. You're saying you think they made that choice because of regulations and what your issues are. That could well be true, but we really don't know. Maybe there's a more interesting reason. I'm just saying you're really sure for a person who wasn't involved in this.
You also said that when Meta delayed the Threads release by a few weeks in the EU. I recommend reading the princess on a pea fairytale since you seem to be quite sheltered, using the term draconian as liberally.
The text of the law says that the actual criteria can change to be whatever they think is scary:
As regards stand-alone AI systems, namely high-risk AI systems other than those that are
safety components of products, or that are themselves products, it is appropriate to classify
them as high-risk if, in light of their intended purpose, they pose a high risk of harm to the
health and safety or the fundamental rights of persons, taking into account both the severity
of the possible harm and its probability of occurrence and they are used in a number of
specifically pre-defined areas specified in this Regulation. The identification of those
systems is based on the same methodology and criteria envisaged also for any future
amendments of the list of high-risk AI systems that the Commission should be
empowered to adopt, via delegated acts, to take into account the rapid pace of
technological development, as well as the potential changes in the use of AI systems.
And there's also a section about systemic risks, which llama definitely falls into, and which mandates that they go through basically the same process, with offices and panels that do not yet exist:
I'd call that the "anywhere but US" phenomena. Pretty much 100% of the times I see any "deals"/promotions or whatnot on my google feed, it's US based. Unfortunately I live nowhere near to the continent.
What a silly, provocative comparison. China is a suppressive state that strives to control its citizens while the EU privacy protection laws are put in place to protect citizens. If you cannot access websites from "the free world" because of these laws, it means that the providers of said websites are threatening your freedom, not providing it.
> China suppresses citizens while EU protects citizens!
Lol this is the real silly provocative comparison.
China bans sites & apps from the West that violate their laws - the ad tracking, monitoring, censorship & influencer/fake news we have here... the funding schemes and market monopolizing that companies like Facebook do in the West is just not legal there. Can you blame them for not wanting it? You think Facebook is a great company for citizens, yet TikTok threatens freedom? Lol it's like I'm watching Fox News.
Companies that don't violate Chinese laws and approach China with realistic deals are allowed to operate there - you can play WoW in China because unlike Facebook it's not involved in censorship, severe privacy violations etc. and Blizzard actually worked with China (NetEase) to bring their product to market there instead of crying and trying to stoke WW3 in the news like our social media companies are doing. Just because Facebook and Google can do whatever they want unchecked in America and its vassal the EU, doesn't mean other countries have to allow it. I applaud China for upholding their rule of law and their traditions, and think it's healthy for the real unethical actors behind our companies to get told "No" for once in their lives.
US and its puppet EU just want to counter-block Chinese apps like TikTok in retaliation for them upholding their own rule of law. Sounds like you fell for the whole "China is a big scary oppressor" bit when the West is an actual oppressor - we have companies that control the entire market and media narrative over here - our companies and media can control whether or not white people can be hired, or can predict what you'll buy for lunch. Nobody has a more dangerous hold on citizens than western corporations.
> China is a suppressive state that strives to control its citizens
China's central government also believes it is protecting its citizens.
> while the EU privacy protection laws are put in place to protect citizens
The fact that they CAN exert so much power on information access in the name of "protection" is a bad precedent, and opens the door to future, less-benevolent authoritarian leadership being formed.
(Even if you think they are protecting their citizens now, I actually disagree; blocking access to AI isn't protecting its citizens, it's handicapping them in the face of a rapidly-advancing world economy.)
>China's central government also believes it is protecting its citizens.
Anyone who's taking a course in epistemology can tell you that there's more to assessing veracity of a belief than noting its equivalence to other beliefs. There can be symmetry in psychology without symmetry in underlying facts. So noting an equivalence of belief is not enough to establish an equivalence in fact.
I'm not even saying I'm for or against the EU's choices but I think the purpose of analogies to China is kind of rhetorical purpose of warning or a comparison intended to reflect negatively on the EU. I find it hard to imagine one would make a straight faced case that they are in fact equivalent in scope or scale or ambition or equivalent and their idea of the relation of their mission to their values for core liberties.
I think the difference is here are clear enough that reasonable people should be able to make the case against AI regulation without losing grasp of the distinction between European and Chinese regulatory frameworks.
The previous poster said that the EU is not restricting the freedom of its citizens, but protecting them (from themselves?). I fail to see how one can say that with a straight face. If you had a basic understanding of history of dictorships you would know that every dictatorship starts off by "protecting" its citizens.
> The fact that they CAN exert so much power on information access in
They don't have any power on information access. They just require their citizen can decide what you do with it. There is no central system where information is stored that can be used in future by authoritarian leadership. But the information stored about American by American companies can be use in such a way if there one day an authoritarian leadership in America.
In my opinion this is a thought stopping cliche that throws the concept of differences of scale out the window, which is a catastrophic choice to make when engaging in comparative assessments of policies in different countries. Again just my opinion here but I believe statements such as these should be understood as a form of anti-intellectualism.
EU? I live in south america and don't have access either, Facebook is just showing what the US plans to do, weaponize AI in the future and give itself accesss first.
Also added Llama 3 70B to our coding copilot https://www.double.bot if anyone wants to try it for coding within their IDE and not just chat in the console
Double seems more like a feature than a product. I feel like Copilot could easily implement those value-adds and obsolete this product.
I also don't understand why I can't bring my own API tokens. I have API keys for OpenAI, Anthropic, and even local LLMs. I guess the "secret" is in the prompting that is being done on the user's behalf.
I appreciate the work that went into this, I just think it's not for me.
Why does Meta embed a 3.5MB animated GIF (https://about.fb.com/wp-content/uploads/2024/04/Meta-AI-Expa...) on their announcement post instead of much smaller animated WebP/APNG/MP4 file? They should care about users with low bandwidth and limited data plan.
I'm based on LLaMA 2, which is a type of transformer language model developed by Meta AI. LLaMA 2 is a more advanced version of the original LLaMA model, with improved performance and capabilities. I'm a specific instance of LLaMA 2, trained on a massive dataset of text from the internet, books, and other sources, and fine-tuned for conversational AI applications. My knowledge cutoff is December 2022, and I'm constantly learning and improving with new updates and fine-tuning.
I suppose it could be hallucinations about itself.
I suppose it's perfectly fair for large language models not necessarily to know these things, but as far as manual fine tuning, I think it would be reasonable to build models that are capable of answering questions about which model they are, their training date, their number of training parameters, and how they are different from other models, etc. Seems like it would be helpful for it to know and not have to try to do its best guess and potentially hallucinate. Although in my experience Llama 3 seemed to know what it was, but generally speaking it seems like this is not necessarily always the case.
I haven't tried Llama 3 yet, but Llama 2 is indeed extremely "safe." (I'm old enough to remember when AI safety was about not having AI take over the world and kill all humans, not when it might offend a Puritan's sexual sensibilities or hurt somebody's feelings, so I hate using the word "safe" for it, but I can't think of a better word that others would understand).
It's not quite as bad as Gemini, but in the same class where it's almost not useful because so often it refuses to do anything except lecture. Still very grateful for it, but I suspect the most useful model hasn't happened yet.
If the model doesn't refuse to produce output, it's not censored anymore for any practical purpose. It doesn't really matter if there are "censorship neurons" inside that are routed around.
Sure, it would be nice if we didn't have to do that so that the model could actually spent its full capacity on something useful. But that's a different issue even if the root cause is the same.
I think the point is Silicon Valley is such a place.
"Visible nipples? The defining characteristic of all mammals, which infants necessarily have to put in their mouths to feed? On this website? Your account has been banned!"
Meanwhile in Berlin, topless calendars in shopping malls and spinning-cube billboards for Dildo King all over the place.
GPT-3.5 rejected to extract data from a German receipt because it contained "Women's Sportswear", sent back a "medium" severity sexual content rating. That was an API call, which should be less restrictive.
When it comes to LLMs, metrics are misleading and easy to game. Actually talking to it and running it through novel tasks that require ability to reason very quickly demonstrates that it is not on par with GPT-4. As in, it can't solve things step-by-step that GPT-4 can one-shot.
This was exactly my experience. I have very complex prompts and I test them on new models and nothing performs as well as GPT-4 that I've tried (Claude 3 Opus included)
Could you expand on this? Is the idea to embed paragraphs (or some other arbitrary subsection) of text, and then semantic search for the most relevant paragraphs, and then only summarize them?
Yes that's exactly right, but it presumes you know what to look for and what you want in your summary. Our use case is to pick out action items or next steps from meeting notes so this can work. But not for all use cases - i.e. summarize this paper.
The whole idea that the copyright holder needs to opt-out is seriously broken. Copyright is not 'opt-out'. Stable diffusion has set themselves up for trouble, this should have been opt-in.
Lately I've been walking 20 km per day (~12.5 mi). Takes at least 3.5 hours. I try not to listen to any audio -- I just let my mind wander.
To step it up a notch, I put on a weighted backpack (i.e. ruck). I'm typically carrying 30 lbs, sometimes 50. This raises the heart rate and roughly doubles the caloric burn. Low injury risk, too. Michael Easter's book "Comfort Crisis" has a nice chapter on rucking.
I just wish it was easier to contact nature on my walks. I live in a big urban centre, so it's not always easy. But at least urban centers can facilitate social walking -- I've found some fantastic Meetup groups where I can partake in social hikes.
Can't say enough good things about walking/hiking/rucking. It's foundational to the human experience. Get outside, everyone :)
That is some serious distance. I remember thinking I would be able to do at least 25-35 km per day, a few years ago when I walked parts of the camino de Santiago in Spain. Blisters and destroyed feet in just a few days, not ever reaching 30 km per day I think.. How long did it take you to get used to this kind of distance daily?
I have heard of the camino de Santiago from someone planning to do it. I just looked it up and it says you can choose one of many routes. Camino Frances being the most popular. Feel like this will be on my bucket list if I can ever get that fit :-).
> I just looked it up and it says you can choose one of many routes.
Yes, I've done small parts of the route on two different occasions. First time I quit due to problems with my feet, the second time I quit after problems with the heat :-)
I never planned to go the full camino, as most of these routes are quite long. The one that starts in France is about 800 km (500 miles) I think, so that will be a full summer holiday and probably a few weeks off work at best. Not ideal to walk during the height of summer either, as the heat can be brutal.
Thanks for the advice. In the army we got the opposite advice, they said we must never use woolen socks when we did the 30 km march, always use cotton they said...
Yes, but in my experience the quality of your socks matters much more. A single wrinkle is gonna give you a nice blister within 5km. Get nice socks and pack blister bandages. Those things are like magic.
As an anecdote, I stopped buying "normal" socks from regular clothing stores completely.
All I wear is hiking labeled socks from sports stores. Greatly reduces wrinkles even if you go for the cheap ones. (Decathlon's store brand for example, but that's EU only i think).
From my adjacent experience (ultrarunning / multi-day races), after trying a bunch of different brands, I've settled on Drymax. By far the best blister resistance! They're really good at keeping feet, well, dry, which helps a lot against chafing.
If you have particular hot spots, I'd also recommend applying some foot cream like Trail Toes. I always use it for runs over ~30km.
Socks with a high percentage of wool are amazing if you've got even remotely sweaty feet. You can be having a complete swamp in your shoes and still have warm feet.
When I (not parent) went to the Camino de Santiago, I was already an experienced hiker (I could certainly walk 30kms in a day and I had done it many times) and I brought a good pair of shoes with me (that I used before without getting any blisters).
But my feet were just not ready to take 30km per day for many, many days in a row. It's one thing to do it just during weekends, doing it every day is a different beast.
Eventually it got better, but it took a while for me to get used to it (and a whole lot of Compeed plasters).
My advice for anyone who is going, would be to start gradually (maybe 10-15 km/day) and then over time go for longer and longer walks.
Edit: I totally agree with the sibling comment about socks being just as important, if not more, than shoes
This was something I heard very often from fellow pilgrims on the camino. That the constant abuse, really is something else. There is a ton of shoe stores along the camino, and plenty of people that "just have to wait a bit" for their feet to catch up. I guess both shoes and socks are important; but certainly never do what I did and go to Spain with brand new shoes...
Absolutely. Not only the size, but the general shape and the technologies of the shoe. For walking/running (not hiking as the parent post is discussing) I like the Ultra Boost line of shoes with their "torsion system", breathable knit upper material, and lightweight foam outer sole. Comfiest shoes I own.
I'm going to go against the grain here but probably not. I walked 500+ km (300 miles) on the Camino Del Norte in a pair of 20£ under armour running shoes I bought 3 years ago. I only had one blister that went away when I put a blister plaster on it.
I think in my experience the size of the shoe matters hugely, more than the quality. Slightly too loose or slightly too tight will lead to blisters if you do an unusual (for you) amount of walking.
I'm doing 20-30 km on free days and 1-10 km on most work days for the last 2 years, but not nearly as fast (my average speed went from 4.5 to 5.1 km/h in that time). Are you sure about the doubling of caloric burn?
Cause I started when I was 130 kg and now I'm 90-100 kg and I measured calories in and how much I walked, and calculated the calories burned/km walked depending on weight and it's basically linear for me (roughly 60 kcal/km at 100 kg). It would be very weird if just 30 kg doubled the caloric burn, maybe the speed makes the difference?
At any rate I agree it's great, it completely changed my life. It's very hard to be frustrated after a 5-hour walk.
I started doing this and got to walking ~10-15km per session, but boy does it take a long time. It gets old fast. Music helps but at some point the time cost is just too high. I started swimming instead and it feels like 30 minutes in the water is roughly equivalent to 2 hours walking, though I'm not a very good swimmer.
There is definitely a risk for injury walking. Aside from the obvious risk of tripping over, you can also mess up your feet and your back, especially when carrying weight. I didn't have any knee problems but apparently that's another risk.
I'm not sure my body could handle that (my hips would need a day to recover from a 20 km walk).
I'm in my mid 50s, so I'd like to believe my age is part of that.
I do about 7 km / day, which still takes time.
I've probably done everything mentioned here (thinking, audiobooks, music, walking with my kids, etc.).
Lately, I've been simply listening for birds. I've gotten pretty good at identifying several species by vocalization and it has the added benefit of putting me "in tune" with my environment.
> (my hips would need a day to recover from a 20 km walk).
>
> I'm in my mid 50s, so I'd like to believe my age is part of that.
Nah, it's just lack of practice. I turned fifty during my walk across Europe from Dublin to Istanbul. I'm 55 now and can happily cover 30 km every few months.
Try going for an extra ten or twenty minutes on your regular walk, and when that feels good, add another ten or twenty minutes. Keep going until you can knock out 30 km in a day's walk, with a break for lunch. Good luck, and enjoy it!
I'm an avid walker (I guess rucker, considering I always wear my backpack). My worst is when passing cars stop and offer me a ride. Apparently it's socially acceptable/normal to run, though.
You are very lucky. After walking about 6km every day for few months arches of my feet started to hurt. Basically overuse injury. I probably should be varying shoes and places (surfaces) I walk on.
People who spend a lot of time in mindless remote meetings (the kind that proliferated during the pandemic) can take most of them while walking. There’s a ton of resources on walking meetings going back many years, some are geared towards just grabbing the attendees and all going for a walk, whereas the tendency these days is for remote / video / audio meetings.
There are even tools that transcribe meeting audio and try to extract summaries, agendas and actions from the discussed material.
