AI is now part of the software supply chain, and instruction-tuned LLMs like ChatGPT introduce risks when developers trust generated code they don't fully understand.
LLMs are often trained on vast, unfiltered datasets like the Common Crawl, which includes public GitHub repositories and Stack Overflow posts of varying quality.
Prompt injection uses malicious language to bypass an AI's instructions, as shown in a demo where a simple command hijacks a text summarizer app.
A malicious email containing a hidden prompt can compromise an AI email assistant, causing it to add malicious links or exfiltrate data without user interaction.
Defend against prompt injection by using third-party security agents to analyze I/O or implementing a multi-LLM architecture with privileged and quarantined models.
AI models can invent non-existent software packages, which attackers then create as malicious decoys to trick developers into installing malware via hallucination squatting.
Attackers use purpose-built malicious AI models to refactor old exploits, making them effective again, and to create highly convincing spearphishing campaigns.
Employees often paste sensitive information like API keys into public AI models, creating a risk of data leakage and enabling attackers to extract secrets.
Instead of banning AI tools, which creates shadow IT risks, focus on developer education, using the right tools for the job, and reinforcing security fundamentals.
24:23Balázs Kiss
29:00Alex Soto
25:17Scott Hanselman
27:02Mirko Ross
35:37Liran Tal
27:26Cassie Kozyrkov
23:24Keno Dreßel
28:27PJ Hagerty
Jobs that call for the skills explored in this talk.
