When I began researching how employees were adapting to AI at work, I was not thinking of it as an AI safety project.
Drawing on my neuroscience, HR and organisational development background, I was curious to learn how employees were experiencing AI adoption within their organisations, beyond the common narrative around productivity.
So I ran a research study with 299 employees in UK organisations that had formally introduced AI tools, with representation from six key sectors: tech, financial services, professional services, healthcare, education and public sector.
The question became bigger than the workplace
As I developed and shared this work, I began connecting with people in the AI safety space. Curious to learn more, I applied to and was accepted onto the BlueDot AGI Strategy course.
This opened up wider perspectives and a new set of questions about AI and society. It also led me down multiple rabbit holes and got me excited about new areas like mechanistic interpretability and digital minds, but those deserve articles of their own.
During the course, there was this one concept in particular that stood out: gradual disempowerment. It changed how I looked at my workplace findings and I began to wonder if some of the patterns I had observed might represent early, small-scale signs of a much larger problem.
What is gradual disempowerment?
The concept was introduced by Jan Kulveit and colleagues in a 2025 paper and refers to the possibility that humanity could progressively lose influence and meaningful control as AI becomes embedded in decision making and essential societal functions.
It does not require one big, dramatic moment when AI takes over as a result of a sudden increase in its capabilities. Instead, it could happen incrementally as human influence over systems such as the economy, culture and governments starts to weaken.
The paper’s argument is that many of these systems currently require human cognition and labour to function. They may become less reliant on people as AI alternatives become more competitive and, in turn, less responsive to what people need.
What the research found
The study found evidence that AI may be affecting how employees approach cognitive tasks:
56% felt confident with an AI output after a brief review
45% tended to go with the AI output rather than their own approach
47% scrutinised AI outputs less than a colleague’s work
41% found tasks they had previously completed independently harder without AI
I also explored whether these patterns varied according to how frequently they used AI. Daily users reported the strongest patterns across all four items.
The findings suggest that confidence after a brief review, deference to AI outputs and difficulty performing tasks independently may form part of the same emerging pattern of cognitive dependency.[1]
How these findings might connect to gradual disempowerment
The gradual disempowerment paradigm is relatively new and it mostly still sits at a systemic, macro level.
These results suggest the potential behavioural micro-level mechanisms that could contribute to these systemic impacts in the long term. The cognitive dependency patterns identified here suggest how at an individual level, humans could gradually lose agency by outsourcing judgement to AI.
Effective human oversight requires more than formal accountability for decision making. If over time people become less able to challenge AI outputs, being “in the loop” might not necessarily equate to meaningful human control.
Neuroergonomics, the study of the brain at work, provides a useful precedent for thinking about this. Years of research, much of it on automation from the aviation industry and other safety critical domains, have shown that humans can struggle to monitor an automated process they are not actively involved in1. Add to this automation bias2 - the tendency to trust automated outputs over our own judgement—along with our brain’s preference for minimising cognitive effort3, and meaningful oversight gets harder to sustain. If it cannot be assumed, it has to be actively designed for and built into workflows.
Currently these patterns are visible at the workplace level. The question is what happens if they become the default across wider societal systems, such as governments, courts, the media or science. As people outsource more judgement, institutions may find it easier to reduce human involvement, supporting the original paper’s concern that these systems could become less dependent on human cognition and labour.
Why this technology is different
One of the challenges I often hear is that we have been through this before with Google and other technologies and turned out fine. And yes, cognitive offloading to external tools is not new, they can form part of what cognitive scientists call ‘distributed cognition’. According to this, thinking and memory are not confined to the individual brain, but rather distributed across interactions with other people, physical environments and like in this case, external tools.
Unlike some of the previous technologies where we were offloading narrow, discrete cognitive tasks such as navigation or information retrieval, AI takes this further by producing complete analyses and recommendations across a wide range of tasks. This makes it easier to start outsourcing part of the interpretation and judgement that follow.
The ‘black box’ nature of large language models also makes this particularly challenging. Users are presented with a very confident sounding answer, while being unable to understand how this was arrived at and where uncertainty lies.
What the study does not show
The study captured employees’ self-reported experiences at a single point in time and it did not objectively assess whether their ability to complete tasks independently had declined.
The finding that the majority of respondents felt confident after a brief review should not automatically be interpreted as insufficient verification, since “brief” was self-defined by respondents. A brief review may be entirely appropriate for some tasks.
The study therefore does not prove that AI causes cognitive dependency, but it identifies a set of related, self-reported cognitive patterns that warrant further investigation.
What’s next
What would make cognitive dependency an early warning sign?
The first step is testing whether this hypothesis stands. Is cognitive dependency truly an early indicator of gradual disempowerment or will it resolve like previous adaptations to technology have?
To answer this, research would need to move beyond self-reported measures to assess whether repeated AI use affects people’s ability to challenge outputs or complete tasks independently. Longitudinal behavioural experiments would be especially helpful, potentially supported by techniques such as eye tracking or EEG/fNIRS to investigate changes in attention and cognitive engagement. The evidence base is still developing, with more rigorous longitudinal research needed.
2. What forms of cognitive offloading become harmful?
Something else I have been considering lately is at which point cognitive offloading becomes unhelpful. As mentioning, offloading is not inherently harmful and especially in the short term, it can free up space in our working memory and reduce cognitive overload. When does it become risky? Further research on this can help us understand what cognitive processes can be safely delegated, under which conditions and ultimately what human capabilities need to be preserved going forward.
3. When do individual effects aggregate into reduced institutional and societal influence?
Once we better understand cognitive dependency at an individual level, the next question is how these effects aggregate at an institutional and societal level. Do they become embedded in ways that leave humans still formally accountable but less able to exercise meaningful control?
This would require developing indicators to track how often consequential decisions are shaped by AI, how much influence humans retain, what thresholds apply and whether any feedback loops emerge at a systemic level.
Reasons to be hopeful
I am an optimist by nature so why am I writing about gradual disempowerment?
Partly, because connecting the dots felt too important to leave unsaid. But more importantly, the value of identifying a possible early warning sign is that it gives us the opportunity and time to respond.
Nothing here is set in stone. AI can expand human capability, but only if we are deliberate about how we use it. What we need to figure out is how to realise those benefits while preserving the human judgement and agency to remain meaningfully in control.
References
1 NASA (2023). ’A meta-analytic approach to investigating the relationship between trust in automation and attention allocation. NASA Technical Report 20230008573. NASA Ames Research Center.
Schaefer KE, Chen JY, Szalma JL, Hancock PA (2016). ‘A Meta-Analysis of Factors Influencing the Development of Trust in Automation: Implications for Understanding Autonomy in Future Systems’. Hum Factors. 58(3):377-400. doi: 10.1177/0018720816634228.
2Goddard K, Roudsari A, Wyatt JC (2012).’ Automation bias: a systematic review of frequency, effect mediators, and mitigators’. J Am Med Inform Assoc.19(1):121-7. doi: 10.1136/amiajnl-2011-000089.
3Christie ST and Schrater P (2015). ‘Cognitive cost as dynamic allocation of energetic resources’. Front. Neurosci. 9:289. doi: 10.3389/fnins.2015.00289.
Kulveit, J., Douglas, R., Ammann, N., Turan, D., Krueger, D., & Duvenaud, D. (2025). ‘Systemic Existential Risks from Incremental AI Development’. arXiv: 2501.16946.
These measures formed part of a four-item scale showing acceptable internal consistency (Cronbach’s alpha = 0.79). ‘Cognitive dependency’ is used as an interpretative label for the observed pattern. Related concepts in cognitive science and human-AI interaction research include cognitive offloading and automation bias.
Could cognitive dependency be an early warning sign of gradual disempowerment?
I started with a workplace question
When I began researching how employees were adapting to AI at work, I was not thinking of it as an AI safety project.
Drawing on my neuroscience, HR and organisational development background, I was curious to learn how employees were experiencing AI adoption within their organisations, beyond the common narrative around productivity.
So I ran a research study with 299 employees in UK organisations that had formally introduced AI tools, with representation from six key sectors: tech, financial services, professional services, healthcare, education and public sector.
The question became bigger than the workplace
As I developed and shared this work, I began connecting with people in the AI safety space. Curious to learn more, I applied to and was accepted onto the BlueDot AGI Strategy course.
This opened up wider perspectives and a new set of questions about AI and society. It also led me down multiple rabbit holes and got me excited about new areas like mechanistic interpretability and digital minds, but those deserve articles of their own.
During the course, there was this one concept in particular that stood out: gradual disempowerment. It changed how I looked at my workplace findings and I began to wonder if some of the patterns I had observed might represent early, small-scale signs of a much larger problem.
What is gradual disempowerment?
The concept was introduced by Jan Kulveit and colleagues in a 2025 paper and refers to the possibility that humanity could progressively lose influence and meaningful control as AI becomes embedded in decision making and essential societal functions.
It does not require one big, dramatic moment when AI takes over as a result of a sudden increase in its capabilities. Instead, it could happen incrementally as human influence over systems such as the economy, culture and governments starts to weaken.
The paper’s argument is that many of these systems currently require human cognition and labour to function. They may become less reliant on people as AI alternatives become more competitive and, in turn, less responsive to what people need.
What the research found
The study found evidence that AI may be affecting how employees approach cognitive tasks:
56% felt confident with an AI output after a brief review
45% tended to go with the AI output rather than their own approach
47% scrutinised AI outputs less than a colleague’s work
41% found tasks they had previously completed independently harder without AI
I also explored whether these patterns varied according to how frequently they used AI. Daily users reported the strongest patterns across all four items.
The findings suggest that confidence after a brief review, deference to AI outputs and difficulty performing tasks independently may form part of the same emerging pattern of cognitive dependency.[1]
How these findings might connect to gradual disempowerment
The gradual disempowerment paradigm is relatively new and it mostly still sits at a systemic, macro level.
These results suggest the potential behavioural micro-level mechanisms that could contribute to these systemic impacts in the long term. The cognitive dependency patterns identified here suggest how at an individual level, humans could gradually lose agency by outsourcing judgement to AI.
Effective human oversight requires more than formal accountability for decision making. If over time people become less able to challenge AI outputs, being “in the loop” might not necessarily equate to meaningful human control.
Neuroergonomics, the study of the brain at work, provides a useful precedent for thinking about this. Years of research, much of it on automation from the aviation industry and other safety critical domains, have shown that humans can struggle to monitor an automated process they are not actively involved in1. Add to this automation bias2 - the tendency to trust automated outputs over our own judgement—along with our brain’s preference for minimising cognitive effort3, and meaningful oversight gets harder to sustain. If it cannot be assumed, it has to be actively designed for and built into workflows.
Currently these patterns are visible at the workplace level. The question is what happens if they become the default across wider societal systems, such as governments, courts, the media or science. As people outsource more judgement, institutions may find it easier to reduce human involvement, supporting the original paper’s concern that these systems could become less dependent on human cognition and labour.
Why this technology is different
One of the challenges I often hear is that we have been through this before with Google and other technologies and turned out fine. And yes, cognitive offloading to external tools is not new, they can form part of what cognitive scientists call ‘distributed cognition’. According to this, thinking and memory are not confined to the individual brain, but rather distributed across interactions with other people, physical environments and like in this case, external tools.
Unlike some of the previous technologies where we were offloading narrow, discrete cognitive tasks such as navigation or information retrieval, AI takes this further by producing complete analyses and recommendations across a wide range of tasks. This makes it easier to start outsourcing part of the interpretation and judgement that follow.
The ‘black box’ nature of large language models also makes this particularly challenging. Users are presented with a very confident sounding answer, while being unable to understand how this was arrived at and where uncertainty lies.
What the study does not show
The study captured employees’ self-reported experiences at a single point in time and it did not objectively assess whether their ability to complete tasks independently had declined.
The finding that the majority of respondents felt confident after a brief review should not automatically be interpreted as insufficient verification, since “brief” was self-defined by respondents. A brief review may be entirely appropriate for some tasks.
The study therefore does not prove that AI causes cognitive dependency, but it identifies a set of related, self-reported cognitive patterns that warrant further investigation.
What’s next
What would make cognitive dependency an early warning sign?
The first step is testing whether this hypothesis stands. Is cognitive dependency truly an early indicator of gradual disempowerment or will it resolve like previous adaptations to technology have?
To answer this, research would need to move beyond self-reported measures to assess whether repeated AI use affects people’s ability to challenge outputs or complete tasks independently. Longitudinal behavioural experiments would be especially helpful, potentially supported by techniques such as eye tracking or EEG/fNIRS to investigate changes in attention and cognitive engagement. The evidence base is still developing, with more rigorous longitudinal research needed.
2. What forms of cognitive offloading become harmful?
Something else I have been considering lately is at which point cognitive offloading becomes unhelpful. As mentioning, offloading is not inherently harmful and especially in the short term, it can free up space in our working memory and reduce cognitive overload. When does it become risky? Further research on this can help us understand what cognitive processes can be safely delegated, under which conditions and ultimately what human capabilities need to be preserved going forward.
3. When do individual effects aggregate into reduced institutional and societal influence?
Once we better understand cognitive dependency at an individual level, the next question is how these effects aggregate at an institutional and societal level. Do they become embedded in ways that leave humans still formally accountable but less able to exercise meaningful control?
This would require developing indicators to track how often consequential decisions are shaped by AI, how much influence humans retain, what thresholds apply and whether any feedback loops emerge at a systemic level.
Reasons to be hopeful
I am an optimist by nature so why am I writing about gradual disempowerment?
Partly, because connecting the dots felt too important to leave unsaid. But more importantly, the value of identifying a possible early warning sign is that it gives us the opportunity and time to respond.
Nothing here is set in stone. AI can expand human capability, but only if we are deliberate about how we use it. What we need to figure out is how to realise those benefits while preserving the human judgement and agency to remain meaningfully in control.
References
1 NASA (2023). ’A meta-analytic approach to investigating the relationship between trust in automation and attention allocation. NASA Technical Report 20230008573. NASA Ames Research Center.
Schaefer KE, Chen JY, Szalma JL, Hancock PA (2016). ‘A Meta-Analysis of Factors Influencing the Development of Trust in Automation: Implications for Understanding Autonomy in Future Systems’. Hum Factors. 58(3):377-400. doi: 10.1177/0018720816634228.
2Goddard K, Roudsari A, Wyatt JC (2012).’ Automation bias: a systematic review of frequency, effect mediators, and mitigators’. J Am Med Inform Assoc.19(1):121-7. doi: 10.1136/amiajnl-2011-000089.
3Christie ST and Schrater P (2015). ‘Cognitive cost as dynamic allocation of energetic resources’. Front. Neurosci. 9:289. doi: 10.3389/fnins.2015.00289.
Kulveit, J., Douglas, R., Ammann, N., Turan, D., Krueger, D., & Duvenaud, D. (2025). ‘Systemic Existential Risks from Incremental AI Development’. arXiv: 2501.16946.
These measures formed part of a four-item scale showing acceptable internal consistency (Cronbach’s alpha = 0.79). ‘Cognitive dependency’ is used as an interpretative label for the observed pattern. Related concepts in cognitive science and human-AI interaction research include cognitive offloading and automation bias.