They often do. The value of those kinds of blanket security audits is questionable, however.
(This is one of the reasons I'm generally pro-OSS for digital infrastructure: security quickly becomes a compliance game at the scale of government, meaning that it's more about diligently completing checklists and demonstrating that diligence than about critically evaluating a component's security. OSS doesn't make software secure, but it does make it easier for the interested public to catch things before they become crises.)
Also, any certificate bears a certificator company name. We can always say "company A was hacked despite having its security certified by company B". So that company B at least share some blame.
In practice, most commercial attestations/certifications contain enough weasel language that the certifier isn't responsible for anything missed (i.e. reasonable effort only).
But yes, there are many standards for this (e.g. SOC Type 2 reports).
In defense of their utility, the good ones tend to focus on (a) whether a control/policy for a sensitive operation exists at all in the product/company & (b) whether those controls implemented are effectively adhered to during an audited period.
That’s not really how they work. The auditor attests that they were provided with evidence that the systems/business units audited were compliant at the time of auditing. That doesn’t mean that the business didn’t intentionally fake the evidence, or that the business is compliant at any time subsequent to the assessment.
An auditor would certainly have some consequences if they were exposed for auditing negligently.
This is how the PCI SSC manages to claim that no compliant merchant/service provider has ever been breached, because they assume being breached means that the breached party was non-compliant at the time of the breach. Which is probably a technically true statement, but is a bit misleading about what they’re actually claiming that means.
“Worthless” is quite a strong claim. There isn’t much work I’ve encountered that’s truly “worthless”, even though bad work can make me quite upset. Anyways, that’s why I would often caveat.
Mandatory audits by accredited auditors in order to participate in a market, inevitably create a market for accredited auditors that don't uncover too much but ensure all checkboxes are ticked. Much of the security industry is actually selling CYA and not actual security. The same dynamic at play means buyiong a home/boat/car you should get your own inspector, not blindly trust the seller's.
I'll say they are worthless because most of time they are dragging time away from things that could improve security. For example, $LastJob we spent a ton of time on SOC2 compliance and despite having applications with known vulnerabilities, we got hacked and ended up all over the news. Maybe of instead of spending all the time getting SOC2 compliance finished, we could have worked at upgrading those apps.
Actually, I doubt they would have upgraded the apps and pocketed the profits instead but SOC2 is providing cover instead of real change.
In this particular case it was worthless. If you have known vulnerabilities and you deprioritize that work to waste time on soc2, and get hacked because of it… soc2 was worthless. Because the whole point is security assurance. When you get hacked you’ve proved the opposite of security assurance.
But also you gotta have the balls to stand up to the guy pushing soc2 and say. No. There are known vulnerabilities. We are patching those first then we are doing soc2. The way I frame it is “we know we have critical vulnerabilities, we don’t need to go hunting for more till we fix them. Once we fix them we go looking for other ways to improve security posture”
And if the ceo still insists (big client requires it so we’re doing soc2 simultaneously) you say fine, then hire a security consultant so we can go twice as fast. And if he refuses you quit because fuck that place.
Because it's better than nothing when independent organizations are reviewing systems or other organizations. It's like saying that penetration tests are useless because you cannot prove security with testing.
Even if these govt. security audits are checkboxes, dont they require some nominal pentesting and black box testing, which test for things like SQL injection?
It may not apply to this specific incident, but pen-testing only ensures you meet a minimum standard at a specific point in time.
I almost feel I could write novels (if only I had time and could adequately structure my thoughts!) on this and adjacent topics but the simple fact is that the SDLC in a lot of enterprises/organizations is fundamentally broken, unfortunately a huge portion of what breaks it tends to occur long before a developer even starts bashing out some code.
(This is one of the reasons I'm generally pro-OSS for digital infrastructure: security quickly becomes a compliance game at the scale of government, meaning that it's more about diligently completing checklists and demonstrating that diligence than about critically evaluating a component's security. OSS doesn't make software secure, but it does make it easier for the interested public to catch things before they become crises.)