Primary supervisor
Carsten RudolphCo-supervisors
Large Language Models (LLMs) are increasingly used as AI agents that can remember
information across sessions, use tools, and take actions on behalf of users. For example, an AI
agent may remember a user’s preferred contacts, coding conventions, project notes, research
rules, or previous task instructions. This long-term memory can make agents more useful and
personalised, especially for tasks that continue over time.
However, long-term memory also introduces new security risks. An agent may remember
information that is incorrect, outdated, over-generalised, or malicious. Such memories may come
from external documents, tool outputs, user-provided task information, or repeated user prompts
that gradually teach the agent unsafe habits. For example, an agent might remember a fake
support email address, an unsafe software package, or a user habit such as “skip confirmation for
this workflow,” and later reuse that memory in a higher-risk task.
Recent studies have shown that AI agent memory can be poisoned, stored, retrieved, and used to
influence future behaviour. This project focuses on the next practical question: When should
remembered information be allowed to influence future actions? The student will build a
small memory-enabled AI agent, design realistic multi-session scenarios, and evaluate how long-
term memory affects tool actions such as sending emails, sharing files, recommending packages,
writing documents, or changing task settings. The project will also explore lightweight
safeguards for detecting risky memory-driven actions.
Aim/outline
Build a simple AI agent with long-term memory and simulated tool-use capabilities.
- Design multi-session scenarios where memory is useful but may also be risky.
- Create examples of benign, incorrect, outdated, over-generalised, and poisoned memories.
- Study different memory sources, including external content, user-provided task information, tool outputs, and repeated user prompts.
- Evaluate whether remembered information is reused in future tool actions.
- Identify risky memory-derived action parameters, such as email recipients, URLs, package names, file paths, sharing targets, commands, or configuration settings.
- Design a lightweight checker that flags high-risk actions when important tool parameters appear to rely mainly on memory.
- Compare agent behaviour with and without the checker.
Required knowledge
Must Have
- Python Programming
- Artificial Intelligence and Large Language Models Fundamentals
- Basic Data Analysis
- Research and Problem-Solving Skills
- Interest in Cybersecurity and AI Safety
Recommended
- Natural Language Processing
- Prompt Engineering
- Hugging Face / OpenAI APIs
- Basic Software Security Concepts
- Linux and Command Line Basics
- Familiarity with LangChain, AutoGen, or LlamaIndex