As much as I'd like to believe their numbers, something seems off.
If you look "underground" you'll find hundreds of thousands of forum posts selling keys that are from, you guessed it, HumbleBumble; they're also not shy about citing their sources. In fact, I'd say that more than 75% of all "carded" steam keys are from HumbleBundle, if not more.
Nobody else has the ease of ordering and uses Stripe (pathetic antifraud — which this post alludes to. More about that in my comment history); SMS verification isn't all that grandiose either.
Does this stop the "buys cards casually, doesn't make a career out of it" carder? Sure does. But they're not the ones companies and individuals need to worry about. It's the guys who are making $250, $500, $1000, $2500 a day that you need to worry about.
I'll say this every time the subject of fraud comes up: Do not trust your processor to do anything for you. They have little-to-no interest in protecting you. Hire a nerd to school you on fraud; if you have massive transaction volume, hire that nerd to help train some models on fraud. But do not, and I mean do not fucking trust your processor.
Can you email me at jeff@humble.com? I would love to see what you are talking about. We do have an awesome engineering team that works on fraud and "trusting the processor" is actually the last step in our defense.
I build/train models for high transaction volume. The real struggle for gaming fraud specifically is that the data is typically non-stationary. I came into my job without formal training in machine learning so I may have the terminology wrong but essentially the machine learning models are typically learning distributions over time. Meaning that this combination of features typically has this ratio of fraud to legitimate orders and assumes those ratios will hold in the future.
For example a model trained using historical data will flag too many orders during a sale that brings a spike in legitimate order volume. This can be mitigated somewhat by feeding into the model volume indicators such as time of day and day of week.
For gaming however the organized fraud rings typically hit en-masse. The largest ring that I saw went from zero to 3000 to 4000 attempts a day in a week. A model tuned at the peak would reject too many orders on a typical day and vice-versa.
The other challenge is that all statistical models rely on IID assumptions which means that the attacker isn't supposed to "learn" between attacks. For the typical smash and grab jobs seen with physical goods this (roughly) holds true but completely falls down with organized fraud rings in gaming. Any competent attacker will quickly see when his success rate drops and change tactics or increase attacks when the success rates rise.
The result is that a model that takes a week to build can decay in a matter of days or hours. I use DataRobot which can automate model building and you can combine short term and long term models in your strategy but it's still a struggle.
Historically the patch has been to limit velocity based on a specific data point that was hard to change but one-by-one they have fallen. Credit cards, email addresses, ip addresses, device IDs and now phone numbers. Each is successful for a while but it's an arms race. For example the largest attacks that I've seen utilized a 100,000+ computers over a three month period and 300,000+ credit cards. The attackers had the ability to login to the machines using remote-desktop like software to evade device ID limits.
Getting good results against these types of attacks requires a multi-layered defense but if there was a magic bullet it wouldn't be with classifiers but with anomaly detection. The problem domain is closer to detecting a hacker inside a network or a disease outbreak.
This particular problem is hard and DARPA has thrown lots of money at a lot of people looking for solutions. At the turn of the century it was intrusion detection and after 9/11 it was bio-terror. After years of research none of these have resulted in commercial products because the false positive rates are always to high.
I second not trusting your payment partners to manage fraud for you. For low price games it's possible to be fined and lose your merchant account even when your internal chargeback reports don't show a problem. In some cases the card issuing bank may not issue a chargeback (and absorb the loss) but will still report it to Visa/MasterCard.
> The largest ring that I saw went from zero to 3000 to 4000 attempts a day in a week.
> which means that the attacker isn't supposed to "learn" between attacks
Those are key takeaways and I'm glad someone else (on this side of the job) understands it.
It's a hard problem for anyone to solve. Not to self-promote, but I'm working on something that doesn't rely on machine learning; instead, it's focusing on patterns.
Because I used to be that guy that you worried about. Now, I'm the guy that the guys that you worry about worry about.
I wish you luck and if you succeed I'm sure that there will be some three letter agencies knocking on your door. I've had some luck using off-the-shelf clustering algorithms but they are too CPU intensive to run real time and require an investigator to interpret (great productivity boost though).
If you look "underground" you'll find hundreds of thousands of forum posts selling keys that are from, you guessed it, HumbleBumble; they're also not shy about citing their sources. In fact, I'd say that more than 75% of all "carded" steam keys are from HumbleBundle, if not more.
Nobody else has the ease of ordering and uses Stripe (pathetic antifraud — which this post alludes to. More about that in my comment history); SMS verification isn't all that grandiose either.
Does this stop the "buys cards casually, doesn't make a career out of it" carder? Sure does. But they're not the ones companies and individuals need to worry about. It's the guys who are making $250, $500, $1000, $2500 a day that you need to worry about.
I'll say this every time the subject of fraud comes up: Do not trust your processor to do anything for you. They have little-to-no interest in protecting you. Hire a nerd to school you on fraud; if you have massive transaction volume, hire that nerd to help train some models on fraud. But do not, and I mean do not fucking trust your processor.