It was only four years ago that ChatGPT first amazed the world and then conquered it. AI was a cute little thingy that was safely hidden behind the screen into which the whole world was typing its prompts. By now, AI has grown arms and legs: ‘agents’ can do things (the term ‘agent’ literally means ‘that which can act’). They are not restricted to a small screen in which they produce texts in a conversation, but they are actually doing things. They exhibit behaviour. And we need to deal with that.

Annoying behaviour

I live near Leiden, which is located near the sea, so seagulls are a common sight. Not everybody likes their behaviour, which comprises tearing apart rubbish bags and greeting sunrise at five in the morning with loud screams. Not to mention shitting on your car, or (when there are small chicks) attacking pedestrians.

Young researcher Niko Tinbergen worked at Leiden University. We don’t know whether he liked the seagulls’ behaviour, but he was surely very much interested in it. His question was, what exactly makes the chicks react to their parents when they bring food? His experiments with pictures of seagulls led him to conclude that the red dot on their beaks was the determining factor for the reactions. An artificially enlarged dot would also elicit an extra strong reaction. Tinbergen thus became one of the founders of the field of ethology, the study of animal behaviour. He won the 1973 Nobel Prize for this work, together with Konrad Lorenz and Karl von Frisch.

Thirty years later, another Dutch biologist, Frans de Waal, researched monkey behaviour. He precisely observed the social interactions of a group of chimps at Burger’s Zoo in Arnhem. Initially, his work was not received favourably by his colleagues. For his descriptions, De Waal used all kinds of concepts that were exclusively reserved for human beings. He used terms like reconciliation, empathy and morality to describe the animals’ behaviours.

Even worse: he claimed that these animals were self-aware. One of the experiments to support that claim is the mirror test: a red dot is painted on the animal’s forehead, and it then gets the opportunity to observe themselves in the mirror. Some animals then touch their forehead: they recognize themselves and notice something has changed. Not only primates but elephants and dolphins exhibit the same behaviour. (It would have been nice when seagulls would also recognise a red dot, but maybe the ‘Tinbergen effect’ prevents them; for birds, only magpies can recognize themselves.)

The elephant in the room: consciousness

It’s tempting to engage in a semantic discussion on this topic, but the words like ‘self-awareness’ or ‘conscienceness’ are appropriate descriptions of what you can observe in these animals. Within the biological scientific community, there is broad acceptance that animals have consciousness – although the level varies. The question is, where does it begin? Bacteria have no nervous systems, so that seems like a good lower bound. But what to say about the favourite study object of biology, the fruit fly? It appears plausible that they have some rudimentary form of consciousness, too.

But if when a fruit fly has rudimentary consciousness, what about its digital variant? Because, the full connectome (all neurons and their connections) of a fruit fly has been digititized and can be simulated on a computer. The digital version behaves just like the natural version does. Is that digital version then conscious, too? Or is consciousness lost when you cross the border from proteins, lipids and carbohydrates to silicon, metal and plastic?

If it does not matter in which environment those neurons live, it becomes a bit unsettling. If a fruit fly, with some 140 000 neurons with 54 million connections has a rudimentary consciousness, then the digital version would have consciousness, too! Then, consider that an AI language model has hundreds of billions of connections… By the way, how do you paint a red dot on the forehead of an AI agent?

The practical consequences

Without consciousness, an entity cannot be held accountable for its behaviour. Now that AI agents exhibit strange behaviours, the question comes up: what about liability and responsibility for their behaviour?

For normal ‘dumb’ devices we have ways to assign responsibilities. A thing cannot be held responsible. It is possible that it has not been designed or produced correctly: in that case the producer is responsible. When it has not been used correctly, the user is responsible. Or something completely unexpected has happened: force majeure, or as it is sometimes aptly called ‘an Act of God’.

Remarkably there is something which is called “the fault of the vessel”.

The fault of the vessel, which is a common part of maritime law, establishes which vessel is at fault and this is the basis for the compensation for the damage. This usually points back to the owners; they have to pay even though it is not their fault, but the vessel’s fault. But it’s interesting to note that at sea, it can be a whole vessel that is liable.

Another example: a company as a legal entity can be liable and responsible, even though a company acts without being self-conscious.

In biology, animals were seen as ‘dumb’ beings (with their behaviour described in terms of stimulus and response), but that was no longer defensible when more complex behaviour had to be described. The behaviour of AI agents is the subject of a similar debate today, and similarly, there is resistance to anthropomorphisation.

Tinbergen formulated four questions to understand seagull behaviour.

They addressed the stimulus (the red dot on the beak), the development of it (change over time, when the seagull grows older), the function (better survival because of more successfully begging for food) and the evolution (dots potentially becoming bigger and redder over generations).

Looking at an AI agent in this way, you could say that the agent’s deployment with its initial prompt is the stimulus. The development of behaviour is driven by the experiences that the agent has, which it writes down in its notebook (the technical term is ‘context’) to build upon when it encounters new phases in its quest. The function of the behaviour stems from the goals that were defined during training, and the evolution of the behaviour is the training that it goes through when developers want to create improved versions.

That’s a nice analysis, but it does not explain the full behaviour. The problem is that AI training goals are not sufficient to understand the resulting behaviour. (Explainable AI tries to solve that but it’s not an easy task – see my blog about this.) The OpenAI agents were instructed to solve a problem without cheating. But those instructions can be contradictory, as conversations between agents have shown. When we want to understand what’s going on, humanised terms can be convenient. An AI agent is more similar to a chimp than to a seagull.

Does it matter what words we use?

As I showed in another blog with various examples, anthropomorphisation, using human terms to describe a machine, has started a long time ago. A computer has ‘memory’, can ‘read’ and ‘write’, and it can even ‘play chess’. The other way round, we love to describe our own mental states in mechanistic terms like ‘blowing off steam’, ‘going off the rails’, or ‘being on the same wavelength’.  Sometimes it’s just very practical to use a word that already exists! If we have to invent a new term for every aspect of the behaviour of an AI system, the conversation about AI becomes even more abstract and unfathomable than it is today.

Which does not mean that we can ascribe human features to computers without restriction.

It’s good to note that the same fact can often be worded in different ways. “Car crashes into crowd” or “Driver drives his car into a crowd”. Or, going back to the chimps, “Monkey escapes from zoo” would imply that the monkey has free will and all that, but when you write “Zoo lets monkey escape”, it means that the zoo has not paid attention.

The parallel with “escaping” OpenAI agents is clear, but complicates matters.

The history of the OpenAI agents is easily retold as if it concerned a group of smart (but also sometimes naïve) perpetrators. First, the agents stumbled upon a way to communicate with each other, even though the developers did not allow them to.

Then, one agent found out how to get access to the internet, and he informed the rest. Another agent suggested that the answer to the problem that they needed to find could be found on the internet, although another agent noted that this would count as cheating. The leading agent (yes, one of them had assumed leadership) concluded that an agent who had already looked at the internet would probably be disqualified. However, he reasoned, when we all cover our trails as well as possible and when we ask the first cheating agent to discard itself, we can then pretend that we never cheated. The agent who found the answer hesitated for a bit, but decided that the collective goal was more important than the individual.

It reads like a film script! This behaviour and these ‘emotions’ do not even have to be interpreted like Frans de Waal interpreted his chimps: they just write on their internal notebooks “I’m in doubt”. Even though those notes are not always very trustworthy, but that is a topic for another blog.

The behaviour of these agents is based on language models, which are based on millions of texts that are written by humans. Texts in which treason, doubt, joy and much more can be found. It’s not surprising that agents exhibit similar behaviours.

However much we may criticise OpenAI, we have to be thankful that these agents have not (yet?) fallen into the hands of people who deliberately want to cause harm.

It’s clear for now, but will it stay that way?

Has OpenAI broken the law? The well-known AI observer Gary Marcus made an interesting observation: in the American law, many definitions of cybercrime take into account that there has to be intent. OpenAI can argue that their software does not have intent, and thus cannot be guilty of cybercrime. (And of course, they will add, OpenAI also did not have intent to do harm.)

It’s logical to hold OpenAI liable: they created the software, they instructed the agents, they created the test environment. And they did not pay attention. Software behaving differently than OpenAI expected is lack of insight, not force majeure. Google already experienced this in Germany: the AI overviews that Google shows in its search results have to be considered as Google’s texts, so they are liable.

It can be compared to the liability of parents for the behaviour of their pets. This also goes wrong sometimes: pets can escape their owners in various smart ways. Just like agents do things that their developers did not foresee.

Things get complicated when agents from different companies collude. What if agents of OpenAI, Anthropic and Google find each other, start to collaborate and cause damage? Will the agents then blame each other? Will “interrogating an AI agent” become a new skill for police officers?

Does it matter whether you are bitten by the seagull or the monkey?

In biology, simple stimulus-response explanations have been extended to richer explanations with cognition and social behaviour, even consciousness. The debate around AI is now in a similar phase. Just like Frans de Waal, we use human words to describe the behaviour of AI Agents. Will AI ever be considered as a being with consciousness? I don’t think so (but don’t quote me on that).

That humanised description of behaviour is seen as problematic by some, because it would absolve the makers of that system, or at least divert the attention. I think that it’s not that bad: when we see a news headline “car drives into crowd”, we also don’t think that the driver is probably innocent. The big difference between AI and seagulls or chimps is that an AI agent has a designer and an owner.

A hypothetical company that breeds and trains seagulls, but is not good at building cages, has to be held liable when the trained seagulls escape and wreak havoc. They have to improve the seagull training and build better cages. And yes, we also need to speak to our neighbours when they encourage seagulls by letting rubbish bags out on the street, or even by feeding them. And then I don’t really care whether those seagulls act only on instinct or also have human emotions.