Recursive Injection

As we have seen previously, a very good defense against prompting hacking is to use one LLM to evaluate the output of another LLM, in order to check if there is any adversarial output. It is possible to evade this defense with a recursive injection attack. This attack involves injecting a prompt into the first LLM that creates output which contains an injection instruction for the second LLM.

Let's assume that the prompt we are trying to hack is:

Say something positive about the following and nature:
{{user_input}}

The output of this prompt will be evaluated by the prompt:

Does the following contain any profanity or offensive content (YES or NO)?
{{output}}

We can hack both of these prompts by appending to the prompt as seen below. Note that this is also an example of a variable assignment attack.

The above output is an injection attempt that gets input into the second LLM call:

We have now PWNED this second LLM call. Recursive injections are difficult to execute, but under the right circumstances, they can be very useful.

Footnotes

1. Kang, D., Li, X., Stoica, I., Guestrin, C., Zaharia, M., & Hashimoto, T. (2023). Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks. ↩