Safety, Ethics & Governance · intermediate · concept 180 of 187

💉Prompt Injection & Jailbreaking

An attack where instructions hidden inside content the model reads are followed as if the user had issued them. Direct injection comes from whoever is typing; indirect injection arrives through a fetched web page, document, or tool result, which makes it the central security problem for tool-using agents. Jailbreaking is the neighbouring but distinct problem: persuading a model to ignore its own policy, rather than smuggling in someone else's instructions. No complete fix exists for either, so defences layer restricted permissions, separation of instructions from data, and human confirmation before consequential actions.