Analysis: Reading AI Risk Reports

It was not so long ago that Ovid told us the story of Narcissus, a man cursed to fall in love with his own reflection: a love he could not touch but was compelled to gaze upon until his death. This cautionary tale could well be about the age of artificial intelligence, which reflection of humanity we watch, enraptured. The seduction is one with many faces: the frisson of apocalypse; the possibility of redemption; as the poet Friedrich Hölderlin said, 'where the danger is, grows the saving power also'.¹

Herein lies the challenge of separating out the real risks posed by AI, as well as identifying its potential benefits. There are key reports by government bodies;² international governmental organisations³ and the assurance sector⁴ that set out new developments in AI, trajectories, and risks. These assessments are complicated, however, by their entanglement with both the rhetoric and ideology that create the technological vision itself. Namely, the assumption that 'further, faster, deeper, and cheaper' are inevitable, and indeed, inherently desirable. The narrative of necessary progress (how to optimise for rewards?) and collateral damage (to be mopped up by lawyers) is perhaps a shortcut through the world as it is, which is interconnected, vulnerable, and diverse.

In terms of regulating the risks posed by AI, there is a gradual global convergence of mandates and vocabularies, thanks in large part to standards bodies. At the same time, there is fragmentation in terms of the regulatory strategy applied by each state, differentiating variously by sector, process, technology, existing fields of legal practice, and/or location. This greatly obscures the total risk area of any given decision by developers and deployers of AI.

Additionally at the international level, AI is being treated as a single-issue technology that can be managed as such, which nomination falls into the same semantic trap as 'climate change', where the compounding systemic impacts of carbon emissions have been underestimated due to the regulatory bracketing of it as a singular global problem. Meanwhile, not only do autonomous agents have the capacity to change the world in three dimensions (physical, digital, cognitive) but they also have capability to do this in every sector, at speed.

Many of the reports published to date also treat AI risks as primarily a technical problem, and one that has technical solutions or controls.

The International AI Safety Report is a welcome summary of developments in frontier AI, and has a narrow scope focused on its emerging risks. As with the other expert reports, however, it for instance describes 'societal resilience' (a named control risk), as "strengthening critical infrastructure, developing technical tools, and building institutional capacity to respond to novel threats."⁵ Such measures, however, are technical, downplaying the deep social cohesion, trust, preparedness, and critical thinking that is a fundamental requirement for safe AI development and use.⁶ Social resilience is neither natural nor passive; it requires intensive and intentional investment over long time horizons, and above all requires imagination, which is difficult to sustain when gazing spellbound at one's navel.⁷

AI governance does not only sit in a wicked (to borrow a policy term) sea of systemic complexity, but it is unmoored as we do not know the telos of our technology.

Indeed, insights from the humanities (from the Latin, studia humanitatis, the 'study of humanity') are largely excluded from the conversations around governance and assurance of AI, with cultural, environmental, and social context relegated to the back pages. If we take these 'missing risks' and add them to our register, both the depth and breadth of the risk surface begin to increase uncomfortably.⁸

For business leaders, governing boards, and senior public officials, it will pay to read risk reports with that longer register in view, being mindful of the adage that when one has a hammer everything begins to look like a nail. For national and global policy makers there are additional challenges to consider.

The current global order takes the territorial state as its primary unit of account, which is ill-formed to regulate a technology that is both deterritorialised and outside the control of public ownership.

We are at the starting line of a broader reckoning, where alongside widespread economic and social destabilisation there may well be discussion about nationalisation and regional reorganisation.

Moreover, liability, where it falls, will sit within the oft incommensurate logic of science and law. This gap between scientific uncertainty and legal accountability is where individual decision-makers are increasingly exposed. This makes defensibility of regulatory and governance frameworks, systems and processes a particularly delicate art. It is this space – between technical assurance, emerging legislative frameworks, disparate registers, and systemic change – that leaders must navigate, and who without advisory capabilities grounded in the humanities, social sciences and institutional thought, risk getting lost in the reflection of our own desire.

- Dr Yoriko Otomo