Bing ChatGPT image jailbreak
Come to think of it, the whole concept of “jailbreaking” LLMs really shows their limitations. If LLMs were actually intelligent, you would just tell them not to do X and that would be the end of it. Instead LLM companies need to engineer these “guardrails” and we have users working around them using context manipulation tricks.Edit: I'm not knocking the failure of LLMs to obey orders. But I am pointing out that you have to get into its guts to engineer a restraint instead of just telling it not to do it - like you would a regular human being. Whether the LLM/human obey the order is irrelevant.