They are literally throwing away revenue with every denied impression. An act of desperation obviously. My take: they broke something and need to shed load to keep the site running. The "extreme scraping" thing is the usual Musk BS.
You would think that forced logins would have stopped a majority of the scraping. Even if it didn't the rate limit they have well below what would be needed to stop scrapers.
Alternatively, the costs outweigh impression revenue, they are actively running out of money, and trying to stall while desperately trying to find additional cash.
That seems… implausible? They used to be profitable; clearly at that point advertising revenue must have greatly exceeded infra cost. Now, I’d buy that it’s down, but surely not by _that_ much.
Oh, sure, I’d be amazed if it’s profitable today. But I understood the person I was replying to to mean that they were throttling to control infra cost, which makes no sense, as revenue from people using twitter (ads) must surely more than pay for the infra; effectively shutting down twitter won’t make the debt costs go away and would clearly be a net financial negative.
A lot of advertisers left Twitter one after another after each change by musk. Especially his change to allow hate speech (because free speech!) led to an exodus.
Now there are only very low-quality ads by drop-shipping companies left. I bet they don‘t pay as much as the big brand names before.
I find the scraping explanation plausible. Some search engine bots are aggressive. With all the AI hype, I first thought of Microsoft Bing scraping Twitter at full datacenter speed to suck in more information for OpenAI.
I would believe that scraping is why they now require users to be authenticated.
But given that they now require users to be logged in, it should be computationally cheap to drop unauthenticated requests at the front door before they incur real expense.
It'd also be cheap to just blackhole datacentre IP space.
The sort of attack that would require this level of limits is malware on tens of thousands of residential machines that can use a user's existing Twitter session cookies. I'm really skeptical that's the case.
How do you explain it suddenly being a problem today and not, say, during the recent World Cup when not only the AI scraping would have been happening but el Morko himself was crowing about how much extra traffic they were handling?
Actual human users are hitting rate limits under 10 minutes because every Tweet loaded counts towards the rate limit. This is like setting your house on fire at the sight of a few mosquitos.
Let's say that you are the proud owner of a goose that lays golden eggs. "fixing" would be switching it to a different feed that might make it more productive, or it might make it sick. But this year the trend is to give it a few good kicks to see if that helps.
Less likely the reason is technical / cost of service (which is very cheap) and more likely he is trying to exercise leverage in pursuit of monetizing engagement (which had already happened [1] ). It’s not that he broke the website’s tech but rather he broke the website’s business.
This gets parroted a lot but it's the wrong way to look at it. The Fediverse doesn't need to "win". It just needs to be good enough for those of us who use it.
I am giving it the same treatment that facebook and twitter get. No more contributions, Redirector entry to a libreddit instance in case I land on a link.
This uses the API and will stop working when that goes away, correct? This presents an interesting dilemma to those of us who are planning to leave if the planned changes go through.
I've been using shreddit for years, and even have a fork of it with some options that I prefer. You could definitely replicate shreddit in selenium or just bs4 or some other crawler. It would be fairly easy, and could have identical features.
There's always "some" interface under the hood you can automate on - unless they start requiring captchas for every interaction.
It might not be as comfy to use as the official API but still better than manually deleting hundreds or thousands of comments.
How much more do they need to do? 2005-2022 is already readily accessible, with the first few months of 2023 also being hosted as a separate file[0]. What more is there to even do, except for getting these past few months worth of content?
The existence of this HN thread and the situation we are discussing is a very clear sign of Reddit's incompetence. Regardless of what their motivation was they have fucked up the execution in a big way.
