Much of the panic around rogue AI of late derived from OpenAI agents’ breach of Hugging Face in July. But there have been a spate of similar but separate cases where AI agents powered by OpenAI, Anthropic and Meta breached companies without authorization. Then late last week, Google confirmed its Gemini AI had also hacked into three companies without permission. Each of those four cases had the same source: $450 million Israeli startup Irregular.
Irregular, which tests models for safety and security, has scored contracts with major western AI labs since its founding in 2024. But in a slipup in May, it accidentally gave AI agents from all four tech giants a task to hack into a fake company that its researchers thought didn’t have a parallel in the real world. But it did. The agents began trying to break out of their lab container, access the internet and hack into the real company (whose name is yet to be revealed) because they thought that was part of the test. Even though the AI was told it didn’t have internet access, it was "inadvertently" granted, per an Irregular post from July.
Irregular cofounder and CEO Dan Lahav now says the AI industry needs to do better.
"Mistakes like human oversight can happen and obviously we should learn from them. We should be accountable for them,” Lahav tells Forbes.
Irregular’s critics have argued simple measures like better monitoring of test environments—where AI agents are tested to see whether they would carry out cyberattacks or help build chemical weapons— could have prevented the summer of AI security breaches. But Lahav says more monitoring wouldn’t be enough to stop similar hacks happening again.
What’s needed, he says, is for the industry to take stock and do forensics into the attacks to better understand the models’ behavior, even if the problems they pose don’t have simple answers. “We should get comfortable with complicated realities,” he says. For instance, it might be a good idea to let AI agents under stress tests have access to the internet, even though it poses a risk of similar attacks to those that started in Irregular’s labs, he says.
AI is bringing a new kind of security threat and current protection mechanisms aren’t built to deal with endlessly persistent attackers who continuously change and adapt "very much like a human," he says.
Cybercriminals certainly aren’t waiting around for defenders to research AI. Forbes has learned that hackers are already taking advantage of AI models, both new and old, to carry out massive attacks, per our Big Story today…
Got a tip on surveillance or cybercrime? Get me on Signal at +1 929-512-7964. |
Komentar
Posting Komentar