How the fk did we end up here?

I started researching prompt injections. Somehow, that has led me to thinking about the ownership, architecture and security of entire AI systems. Along the way there’s been Injectionator, Surfaces, and now work on an AI Surface Model.

If you’ve been following any of this and wondering how those things connect, fair question. I’ve occasionally needed to remind myself.

It started with wanting to understand how you could get a language model to follow instructions it wasn’t supposed to follow. What could you put into a prompt, or into something the model would read, that would send it off in the wrong direction?

That was interesting enough on its own. It gave me something to investigate and experiment with, and it was the starting point for Injectionator.

But the further I went, the less comfortable I became with stopping at the model’s response.

There was rather a lot around the LLM

The prompt-injection work had focused my attention on the LLM. When I stepped back and looked at the application using it, there was much more to account for.

There was code deciding what to send to the model. There could be retrieved documents, stored conversations, tools, databases and connections to other services. Somewhere, permissions decided what could be read or changed. Somewhere else, a person was expecting the whole arrangement to do a useful job.

Getting the model to say something unexpected was one result. Understanding what the application could then do with that output was a much bigger investigation.

Imagine an assistant that can search company documents. A model refusing to reveal confidential information sounds reassuring. I’d still want to know why the application could retrieve those documents for that user in the first place.

That line of thinking became A Chatbot Is Not the Surface. I was starting to follow the request through the system, from the prompt to whatever happened next.

Some of this looked very familiar

Looking at the surrounding components brought me back to familiar security questions. Authentication. Authorisation. Credentials. Input handling. The permissions given to an integration because that was the easiest way to get it working.

In the things I was examining, I kept finding security gaps around the model too. I couldn’t assume that a component had all of this covered just because it was part of an AI application.

I also found myself making an increasingly useful distinction: a control could be described in the documentation, available in the software, enabled in a configuration, or actually used on the path I was investigating. Those weren’t interchangeable.

That fed into my thinking about how AI systems are assembled. Choosing components was only part of the work. I wanted to understand what happened when they were connected, and how much of that arrangement someone had actually checked.

Then came the ownership question

This was the bit that kept bothering me.

There could be people responsible for individual components, each doing useful work within their own scope. But who was looking at the complete system? Who could explain how the user’s identity, the model’s instructions, the application’s permissions and the connected tools fitted together?

I struggled to see clear ownership of that whole picture in what I was looking at. Every time I widened the investigation, there seemed to be another boundary to cross and another assumption to check.

I don’t mean that nobody anywhere is doing this work. I mean that I had started with a fairly specific security question and ended up asking a much more awkward organisational one.

Who is responsible for how all of this behaves together?

Which brings me to here

Surfaces is where I’ve been developing that wider perspective at Viewyonder: looking at the connected system and what it can access, influence and do.

For Injectionator, it also created a practical problem. If I wanted to investigate a system, change something and repeat an experiment, I needed a useful description of what I was testing. Otherwise, a result could quickly become detached from the setup that produced it.

That’s the connection to the AI Surface Model and n8r-surface work: recording the components, their connections, the evidence and the bits I still don’t know. It’s experimental work, and I’m still finding out what makes those descriptions useful.

I certainly didn’t have all of this mapped out when I started looking at prompt injections. Each question exposed something else I needed to understand.

So that’s how I got here. I’m still interested in what you can persuade an LLM to do. These days, I’m just as interested in everything connected to it—and who is looking after the whole thing.