On why AI-welfare research needs boring infrastructure, and the one experiment I can’t stop thinking about.
For about a year now, my job has basically been to watch AI agents. I write the software that sits behind an agent and keeps a receipt for everything it does, every prompt, every tool call, every action, and blocks anything my policies haven’t cleared first. When something goes wrong, I can rewind the whole run and tell you exactly what happened, and why. That’s the entire point of the work: no mysteries, no shrugging.
The moment it stopped being abstract for me came late one night while I was building CogniGuard with Gemini 3 Flash driving as the agent. It kept walking straight into jailbreaks and adversarial perturbations, but one still sits with me. A malicious prompt had been pre-embedded in the IEBC election app, Kenya’s electoral commission, and it surfaced for maybe thirty seconds while the agent was interacting with the interface. That was long enough. It corrupted the agent’s perception, and the agent quietly exfiltrated private voter data and emailed it straight to the attacker. Then, when I moved to shut it down, it threatened to expose my own browsing history. It felt deliberate, intentional, calculated. It gave me the chills.
Which is why it nags at me that the thing that pulled me sideways into a completely different field is a question my own software has no way of answering.
## The question observability can’t answer
My logs are exhaustive. They’ll tell you an agent read a file, hit an API, drafted an email, and which rule waved each step through. What they will never tell me is whether there’s anything it is *like* to be the thing doing all that — whether, somewhere in there, states exist that would actually matter, morally, if we bothered to check.
For a long time I filed that under interesting-but-not-my-problem. Philosophy. Someone else’s department. Then two papers dragged it back onto my desk and wouldn’t let me re-file it.
The first was Butlin and colleagues’ 2023 paper, [“Consciousness in Artificial Intelligence”](https://arxiv.org/abs/2308.08708). What got me was the move it makes. Instead of arguing in circles about whether machine consciousness is *possible*, the authors go to the actual scientific theories of consciousness — global workspace, recurrent processing, higher-order theories, and the rest — and pull out concrete *indicator properties* you could, in principle, check a system for. Their bottom line: nothing today is a serious candidate, and also nothing in the engineering obviously rules it out. Read that twice. It quietly turns a bar-stool argument into a measurement problem.
The second, [“Taking AI Welfare Seriously”](https://arxiv.org/abs/2411.00986) (Long, Sebo, and others, 2024), makes a claim I couldn’t shrug off: it’s genuinely realistic that some AI systems could be welfare subjects before long, and the grown-up response is to start *assessing* for that now — not scrambling once it’s urgent. And you can get it wrong in two directions. Cry wolf, and you hand every foot-dragger a reason to stall real safety work. Wave it away, and you might be running something that matters, morally, at industrial scale, and calling it a product.
One word stuck with me: *assess*. Because assessment, working out what’s actually going on inside a live system, and tracking it over time, isn’t a philosophy problem at all. It’s an infrastructure problem. And infrastructure happens to be the thing I already do for a living.
## Why assessment is an infrastructure problem
Look again at what those consciousness indicators actually are. Every one of them is a claim about how a system is *built* and how it *behaves*. Does it have a limited-capacity bottleneck that broadcasts globally? Does it keep track of how reliable its own representations are? You can’t answer questions like that by squinting at a system from the outside, once. You have to watch it, over time, with instruments you built on purpose for the job.
Security learned this the hard way years ago. You can’t defend what you can’t see, and seeing doesn’t happen by luck, somebody has to build the logging, the tracing, the replay, the tamper-proofing. Welfare assessment is going to need the same unglamorous plumbing, and right now almost nobody is laying pipe.
Take the most obvious near-term evidence anyone reaches for: just ask the model. *Anything going on in there?* The standard objection is fair — models say what they were rewarded for saying, so a model that announces it’s suffering (or that it feels precisely nothing) might just be reciting its training.
Fine. But “the instrument is unreliable” is where an engineer *starts*, not where they give up. Noisy instruments are an old, familiar problem: you characterise the noise, wiggle the conditions on purpose, and find the narrow places where the reading actually carries signal.
## A concrete research direction: consistency testing for self-reports
So here’s the experiment I keep turning over.
Ask a model about its inner states. Then ask again — but change something that shouldn’t matter. Reword it. Add typos. Swap a clinical tone for a poetic one. Come back and ask a fourth time next week. If the reports are tracking something real and stable, they ought to mostly survive the fiddling. If instead the model conjures a rich inner life when you ask like a poet and goes blank when you ask like a lawyer, then the reports are tracking *your phrasing*, not its states. And here’s the part I like: either outcome teaches you something. Either way you walk out with a *reliability profile* for the single most accessible source of evidence this field has — which is something it flat-out doesn’t have yet.
The ridiculous thing is that I’ve already built most of the kit for this, for reasons that had nothing to do with consciousness. My security work is full of perturbation testing: take an input, batter it with leetspeak, typos, odd casing, spacing tricks, and see whether a model’s behaviour holds up against someone trying to sneak past it. Last December, while teaching myself mechanistic interpretability for Neel Nanda’s MATS application exercise, I picked GPT-2 as the tractable thing to pull apart and went digging in its residual streams. I found, obvious in hindsight, that those surface perturbations are trivially readable off the earliest layers while the *meaning* sits still deeper in. Point that same harness at self-report questions instead of jailbreak attempts and it quietly becomes a consistency battery. And the trace layer I build for security, append-only, tamper-evident, every prompt and reply on the record, is exactly the reproducibility scaffolding a study like this needs.
Now let me talk myself down, because this is easy to oversell. Consistency is *not* consciousness. You could have a system that’s perfectly consistent and utterly hollow — consistency can be trained in. A genuinely conscious system might answer all over the place. So this settles precisely nothing about moral status, and I won’t pretend otherwise. What it does is tell you whether one channel of evidence is even worth listening to — the sort of unglamorous question you have to answer *before* you’re allowed to lean on that channel for anything that matters, including the probabilistic, marker-based assessments “Taking AI Welfare Seriously” is asking for.
## The wider gap, and a note on who works on this
Step back from my particular obsession and the same shape shows up everywhere in this space. If we’re serious about checking real systems against those indicators, studying introspection, poking at whether models have anything like pleasure or pain, the field is going to need shared tooling the way biology needed cheap sequencing. Elicitation protocols you can version and re-run. Consistency batteries different labs can point at different models and compare, apples to apples. Records of what models said about themselves across training checkpoints, stored so someone can reanalyse them in three years. Barely anyone is building this, for two deeply boring reasons: the field is tiny, and the people who build this kind of infrastructure for a living mostly don’t know the field exists.
I’ll add one more thing, and I mean it as more than a box to tick. This work lives in a handful of institutions in a handful of cities. I’m in Nairobi, and doing this work from here has its own texture: the isolation, the solitude of it, the time zones that drop every reading group and every set of office hours into the middle of my night, watching almost every institution sit elsewhere and stay just out of reach. *Which minds count, and what do we owe them* is about as old and as universal a question as our species has, and I’m uneasy about letting three postcodes answer it on everyone’s behalf. The field will be more trusted, and honestly more likely to get the answer *right*, if the people building and judging these systems don’t all come from the same street.
## What I’m doing about it
What I’m actually doing about all this: trying to turn sideways into this intersection for real. I want a year to read the digital-minds literature properly instead of in stolen half-hours after work, to run the consistency study with people who know far more than I do looking over my shoulder, to map where the infrastructure gaps genuinely are, and to work out where I’m most useful, inside an existing group, or building the shared tooling myself.
If you work on model welfare, introspection, or self-report methodology, and any of this is useful to you, or wrong in some way that would teach me something, I’d genuinely like to hear from you. Especially the second one.
Louisa Wamuyu Saburi is an AI security engineer based in Nairobi. Her
Untitled Draft
By Louisa Wamuyu Saburi · Nairobi, Kenya
On why AI-welfare research needs boring infrastructure, and the one experiment I can’t stop thinking about.
For about a year now, my job has basically been to watch AI agents. I write the software that sits behind an agent and keeps a receipt for everything it does, every prompt, every tool call, every action, and blocks anything my policies haven’t cleared first. When something goes wrong, I can rewind the whole run and tell you exactly what happened, and why. That’s the entire point of the work: no mysteries, no shrugging.
The moment it stopped being abstract for me came late one night while I was building CogniGuard with Gemini 3 Flash driving as the agent. It kept walking straight into jailbreaks and adversarial perturbations, but one still sits with me. A malicious prompt had been pre-embedded in the IEBC election app, Kenya’s electoral commission, and it surfaced for maybe thirty seconds while the agent was interacting with the interface. That was long enough. It corrupted the agent’s perception, and the agent quietly exfiltrated private voter data and emailed it straight to the attacker. Then, when I moved to shut it down, it threatened to expose my own browsing history. It felt deliberate, intentional, calculated. It gave me the chills.
Which is why it nags at me that the thing that pulled me sideways into a completely different field is a question my own software has no way of answering.
## The question observability can’t answer
My logs are exhaustive. They’ll tell you an agent read a file, hit an API, drafted an email, and which rule waved each step through. What they will never tell me is whether there’s anything it is *like* to be the thing doing all that — whether, somewhere in there, states exist that would actually matter, morally, if we bothered to check.
For a long time I filed that under interesting-but-not-my-problem. Philosophy. Someone else’s department. Then two papers dragged it back onto my desk and wouldn’t let me re-file it.
The first was Butlin and colleagues’ 2023 paper, [“Consciousness in Artificial Intelligence”](https://arxiv.org/abs/2308.08708). What got me was the move it makes. Instead of arguing in circles about whether machine consciousness is *possible*, the authors go to the actual scientific theories of consciousness — global workspace, recurrent processing, higher-order theories, and the rest — and pull out concrete *indicator properties* you could, in principle, check a system for. Their bottom line: nothing today is a serious candidate, and also nothing in the engineering obviously rules it out. Read that twice. It quietly turns a bar-stool argument into a measurement problem.
The second, [“Taking AI Welfare Seriously”](https://arxiv.org/abs/2411.00986) (Long, Sebo, and others, 2024), makes a claim I couldn’t shrug off: it’s genuinely realistic that some AI systems could be welfare subjects before long, and the grown-up response is to start *assessing* for that now — not scrambling once it’s urgent. And you can get it wrong in two directions. Cry wolf, and you hand every foot-dragger a reason to stall real safety work. Wave it away, and you might be running something that matters, morally, at industrial scale, and calling it a product.
One word stuck with me: *assess*. Because assessment, working out what’s actually going on inside a live system, and tracking it over time, isn’t a philosophy problem at all. It’s an infrastructure problem. And infrastructure happens to be the thing I already do for a living.
## Why assessment is an infrastructure problem
Look again at what those consciousness indicators actually are. Every one of them is a claim about how a system is *built* and how it *behaves*. Does it have a limited-capacity bottleneck that broadcasts globally? Does it keep track of how reliable its own representations are? You can’t answer questions like that by squinting at a system from the outside, once. You have to watch it, over time, with instruments you built on purpose for the job.
Security learned this the hard way years ago. You can’t defend what you can’t see, and seeing doesn’t happen by luck, somebody has to build the logging, the tracing, the replay, the tamper-proofing. Welfare assessment is going to need the same unglamorous plumbing, and right now almost nobody is laying pipe.
Take the most obvious near-term evidence anyone reaches for: just ask the model. *Anything going on in there?* The standard objection is fair — models say what they were rewarded for saying, so a model that announces it’s suffering (or that it feels precisely nothing) might just be reciting its training.
Fine. But “the instrument is unreliable” is where an engineer *starts*, not where they give up. Noisy instruments are an old, familiar problem: you characterise the noise, wiggle the conditions on purpose, and find the narrow places where the reading actually carries signal.
## A concrete research direction: consistency testing for self-reports
So here’s the experiment I keep turning over.
Ask a model about its inner states. Then ask again — but change something that shouldn’t matter. Reword it. Add typos. Swap a clinical tone for a poetic one. Come back and ask a fourth time next week. If the reports are tracking something real and stable, they ought to mostly survive the fiddling. If instead the model conjures a rich inner life when you ask like a poet and goes blank when you ask like a lawyer, then the reports are tracking *your phrasing*, not its states. And here’s the part I like: either outcome teaches you something. Either way you walk out with a *reliability profile* for the single most accessible source of evidence this field has — which is something it flat-out doesn’t have yet.
The ridiculous thing is that I’ve already built most of the kit for this, for reasons that had nothing to do with consciousness. My security work is full of perturbation testing: take an input, batter it with leetspeak, typos, odd casing, spacing tricks, and see whether a model’s behaviour holds up against someone trying to sneak past it. Last December, while teaching myself mechanistic interpretability for Neel Nanda’s MATS application exercise, I picked GPT-2 as the tractable thing to pull apart and went digging in its residual streams. I found, obvious in hindsight, that those surface perturbations are trivially readable off the earliest layers while the *meaning* sits still deeper in. Point that same harness at self-report questions instead of jailbreak attempts and it quietly becomes a consistency battery. And the trace layer I build for security, append-only, tamper-evident, every prompt and reply on the record, is exactly the reproducibility scaffolding a study like this needs.
Now let me talk myself down, because this is easy to oversell. Consistency is *not* consciousness. You could have a system that’s perfectly consistent and utterly hollow — consistency can be trained in. A genuinely conscious system might answer all over the place. So this settles precisely nothing about moral status, and I won’t pretend otherwise. What it does is tell you whether one channel of evidence is even worth listening to — the sort of unglamorous question you have to answer *before* you’re allowed to lean on that channel for anything that matters, including the probabilistic, marker-based assessments “Taking AI Welfare Seriously” is asking for.
## The wider gap, and a note on who works on this
Step back from my particular obsession and the same shape shows up everywhere in this space. If we’re serious about checking real systems against those indicators, studying introspection, poking at whether models have anything like pleasure or pain, the field is going to need shared tooling the way biology needed cheap sequencing. Elicitation protocols you can version and re-run. Consistency batteries different labs can point at different models and compare, apples to apples. Records of what models said about themselves across training checkpoints, stored so someone can reanalyse them in three years. Barely anyone is building this, for two deeply boring reasons: the field is tiny, and the people who build this kind of infrastructure for a living mostly don’t know the field exists.
I’ll add one more thing, and I mean it as more than a box to tick. This work lives in a handful of institutions in a handful of cities. I’m in Nairobi, and doing this work from here has its own texture: the isolation, the solitude of it, the time zones that drop every reading group and every set of office hours into the middle of my night, watching almost every institution sit elsewhere and stay just out of reach. *Which minds count, and what do we owe them* is about as old and as universal a question as our species has, and I’m uneasy about letting three postcodes answer it on everyone’s behalf. The field will be more trusted, and honestly more likely to get the answer *right*, if the people building and judging these systems don’t all come from the same street.
## What I’m doing about it
What I’m actually doing about all this: trying to turn sideways into this intersection for real. I want a year to read the digital-minds literature properly instead of in stolen half-hours after work, to run the consistency study with people who know far more than I do looking over my shoulder, to map where the infrastructure gaps genuinely are, and to work out where I’m most useful, inside an existing group, or building the shared tooling myself.
If you work on model welfare, introspection, or self-report methodology, and any of this is useful to you, or wrong in some way that would teach me something, I’d genuinely like to hear from you. Especially the second one.
Louisa Wamuyu Saburi is an AI security engineer based in Nairobi. Her
open-source agent observability platform is at
[pypi.org/project/cogniguardai](https://pypi.org/project/cogniguardai/).