> I still don't really get it. Surely the older, simpler, and better cardinal rule is that you just don't expose any service to the Internet that you have given access to your private data, unless you directly control that service and have a very good understanding of its behavior.
This scenario involves a system whose responsibility is to react to an event, analyse your private info in response to the event, and output something.
The exploit is that, much like a SQL injection, it turns out attackers can inject their own commands into the input event.
Also, it's worth keeping in mind that prompts do lead LLMs to update their context. Data ex filtration is a danger, but so is having an attacker silently manipulating the LLM's context.
This scenario involves a system whose responsibility is to react to an event, analyse your private info in response to the event, and output something.
The exploit is that, much like a SQL injection, it turns out attackers can inject their own commands into the input event.
Also, it's worth keeping in mind that prompts do lead LLMs to update their context. Data ex filtration is a danger, but so is having an attacker silently manipulating the LLM's context.