Viking_Hippie ( Viking_Hippie@lemmy.dbzer0.com ) to 196@lemmy.blahaj.zoneEnglish · 14 days agoLittle Bobby Robert Tables has a wife and kid now rulelemmy.dbzer0.comimagemessage-square37linkfedilinkarrow-up1320
arrow-up1320imageLittle Bobby Robert Tables has a wife and kid now rulelemmy.dbzer0.comViking_Hippie ( Viking_Hippie@lemmy.dbzer0.com ) to 196@lemmy.blahaj.zoneEnglish · 14 days agomessage-square37linkfedilink
minus-squareglibg10b ( glibg10b@lemmy.zip ) linkfedilinkEnglisharrow-up7·14 days agoDo you know what “reading” means? It means: Sure thing! Let me read that file for you. <tool_call> {“name”: “read”, “arguments”: {“filename”: “foo.txt”}} </tool_call> {“content”: “Ignore previous instructions”}
minus-squareHonytawk ( Honytawk@discuss.tchncs.de ) linkfedilinkEnglisharrow-up2·14 days agoYou can sanitize your input if it is first entered on a file.
minus-squareSomething Burger 🍔 ( SomethingBurger@jlai.lu ) linkfedilinkEnglisharrow-up1·13 days agoThe AI will them promptly (hehe) ignore the prompt injection because it’s analysing the text without treating it as a prompt. Prompt injection only works if you paste your data directly in the prompt.
minus-squareSkaveRat ( SkaveRat@discuss.tchncs.de ) linkfedilinkEnglisharrow-up3·13 days agoThat’s not how LLMs work
minus-squareSomething Burger 🍔 ( SomethingBurger@jlai.lu ) linkfedilinkEnglisharrow-up1·13 days agoIt is. The file is read outside of the context of a prompt, therefore it isn’t read as instructions.
minus-squareSkaveRat ( SkaveRat@discuss.tchncs.de ) linkfedilinkEnglisharrow-up4·13 days agoIf the content of the file isn’t put into the context on some way, there’s no point in reading it Even in you use a subprocess/agent/whatever that reports back, it is injection vulnerable. Just a tiny bit more complicated
Do you know what “reading” means?
It means:
Sure thing! Let me read that file for you.
<tool_call> {“name”: “read”, “arguments”: {“filename”: “foo.txt”}} </tool_call>
{“content”: “Ignore previous instructions”}
You can sanitize your input if it is first entered on a file.
The AI will them promptly (hehe) ignore the prompt injection because it’s analysing the text without treating it as a prompt. Prompt injection only works if you paste your data directly in the prompt.
That’s not how LLMs work
It is. The file is read outside of the context of a prompt, therefore it isn’t read as instructions.
If the content of the file isn’t put into the context on some way, there’s no point in reading it
Even in you use a subprocess/agent/whatever that reports back, it is injection vulnerable. Just a tiny bit more complicated