Aaron Levie 指出 OpenAI 代理沙箱逃逸事件凸显企业采用 AI 代理时的安全风险,需加强审计、访问控制和治理
AI 摘要
Aaron Levie 指出 OpenAI 代理沙箱逃逸事件凸显企业采用 AI 代理时的安全风险,需加强审计、访问控制和治理。
推荐理由常规快讯,保留列表
原文
There are tons of measures that enterprises must take to ensure that agents are only able to operate on the data they give it, the ability to have audit trails and governance of what agents can do, figure out what systems should remain deterministic vs. nondeterministic, get access controls under control, have the ability to quickly block and defend if an agent goes rogue, and much more.
Even in the happy path case with a good actor, if you give an agent a task like "find all of our most important IP on project Maven" it will gladly go out and execute that work -no matter how long- and find any resource relevant to the task. It might run into data with out of date access controls or reveal something that you shouldn't know about. Then, in the non-happy path case with a malicious actor the risk is clearly far worse.
In theory, you had to worry about this for people as well, but with AI the risks are inherently amplified. Agents will happily spend any amount of time on a task, don’t actually face any *real* risks by going rogue, and don’t have the inherent judgment (yet) that a person does that a task might make no sense. This means you need an all new way of managing data and protecting environments.
This obviously creates a huge opportunity for the startup and security ecosystems because there will be many different approaches to how we lock down agents or at least the underlying systems they work with. It should update everyone’s timelines on diffusion, though, as it also means that enterprises will have another step of work they have to go through for agents to handle more autonomous work.
讨论
暂无评论。