When it comes to Artificial Intelligence what worries many is not so much how smart it might become but instead what it might do with the intelligence it learns. The “do” part is a function of its agency and is perhaps the greatest source of risk and concern facing today’s society.
Agency is the power to act in the world. In its narrow definition agency is intentional but may lack moral considerations. However, having agency without moral capacity is a serious problem and something where applied ethics (AI Ethics) is needed.
Before we explore the topic of AI Agency we need consider the difference between autonomy and agency. Autonomy has more to do with the right to make decisions free from external control or unwarranted interference. Autonomy is the right of self-governance.
In this context, autonomous vehicles are better described as driving agents as they are acting on behalf of the driver’s intention. They do not have the right of self-governance or act on its own intention. However, when it comes to AI agency and autonomy these are often used interchangeably often describing aspirational goals of the creators rather than describing the AI technology itself.
Agency is what turns our possibilities into realities, and therein lies the rub.
Agency is what turns descriptions of our world into something we experience. Having smart people is important, but it's what is done with this knowledge that we are more concerned about. It's the application of knowledge (engineering) which builds our realities.
When it comes to AI, without agency:
Intelligence is just advice,
Information is just data, and
Knowledge is just a database.
You could also say, engineering is just technology. Having smarter machines is not the problem. It is a means to an end. The question is – what end?
For humans, informed by knowledge of our past and future desires, agency turns possibilities into present day realities. What those realities are depend very much (but not entirely) on one’s intentions.
Agency provides the means to transcend a world defined by a future described as unfolding, controlled by deterministic laws of nature, and stochastic uncertainty to a future that is becoming chosen by the decisions and actions we make.
Agency gives us the power to choose our future.
That’s why agency without good judgment is not desirable as it creates the opportunity for risk. We usually limit the amount of agency based on moral capacity and the level of accountability.
Just as with our children, we expect them to behave morally, however we do not hold them accountable in the same way as we do adults. As such we limit what they can do and the choices they can make.
When we are young our foolish choices are tolerated and at times encouraged to provide fodder for learning. However, as adults, foolishness is frowned upon in preference of those who demonstrate wisdom, good judgment, and sound choices.
To act in the world brings with it the responsibility to decide between bad and good, useless and useful, and what can harm and what can heal. Ethics provides the framework for these decisions to be made. In many ways, applied ethics is the application of wisdom to the domain of agency.
If AI is to have agency it must at least have the capacity to make moral decisions. This requires at a minimum ethical subroutines, something that is currently not available. Even if they were, this would need to be accompanied by accountability.
Agency always brings with it a measure of culpability.
Agency and accountability are two sides of the same coin. Agentic AI must be answerable for the decisions it makes. This in turn will require more than just explanation for what it has done.
As humans are more than an embodiment of intelligence, we need another name to describe artificial intelligence with agency having ethical subroutines, and is accountable for its actions.
Here are a possible names for each type:
AI Machines - AI systems without agency (advisory, decision support, analysis, etc..)
AI Agents - AI Machines with agency but without moral capacity and limited culpability
AI Ethical Agents - AI Agents with moral capacity and full culpability
AI Machines can still have agency (self-referencing machines)
In theory, machines have a measure of agency to the degree they interact in the world. In the classical sense, machines may adapt to their environment based on pre-defined rules. However, when it comes to AI Machines the ability to adapt is enhanced by machine learning.
AI Machines of this kind are self-referencing and are not an impartial observer in the classical sense. The output generated interferes with the future they are trying to represent which forms a feedback loop. AI in this scenario is better described as an observer-participant which gives it a greater measure of agency than classical machines. This is agency without purpose or intention manifesting as a vicious or virtuous cycle towards some unknown end.
Perhaps, this is what is meant by autonomous AI. These are AI machines that no longer act on behalf of its creator, but instead acts on its own towards some unknown goal, perhaps even to itself. No wonder this is creating significant angst in the population. We have created an open-loop system with the capacity to act in the world and to decide but lacking moral capabilities.
Agency is not the only concern
AI Agency is a source of significant risk. Some of the categories discussed are currently possible and others are still in our future. Nonetheless, guidelines and guardrails can and should be developed to properly regulate AI Agency proportionate to the level of risk.
AI has other risks beyond its capacity to act in the world and to decide. However, AI Agency is where the greatest benefits will come and with it the greatest risks to society.
