There is a paradox at the heart of generative artificial intelligence. For some time it has been clear that compared with humans, large language models are far more capable of completing very challenging tasks.
More than two years ago, we had evidence that OpenAI’s ChatGPT can comfortably pass difficult exams such as the notorious US Law School Admission Test and Ivy League MBA finals. The latest models consistently turn out high-quality written work, producing essays that educators are unable to distinguish from those written by postgraduate students.
Yet there has been, to date, little evidence of AI causing large-scale disruption to the labour market, even in occupations that are reportedly at very high risk. What’s going on?
Two new pieces of work shed light on the puzzle of AI’s simultaneous excellence and apparent toothlessness in attacking human jobs, while also showing where and why the first large-scale losses may now be under way.
The first is mine. Building on previous work by the Brookings Institution and OpenAI, I have carried out a fine-grained analysis of US employment data, comparing recent trends in job numbers against a list of occupations identified as at especially high risk of automation.
The daily tasks undertaken by accounting clerks, insurance underwriters, travel agents and legal secretaries all overlap almost entirely with LLMs’ capabilities. But the numbers of workers in these roles have all remained within their usual range even as generative AI has proliferated.
There are two notable exceptions, however. Writers — of words, not code — and software developers both show tell-tale signs of LLM-related disruption, with employment falling sharply away from trend in the past two years.
And this is not just a function of wider economic trends in those sectors. Job numbers elsewhere in the computing, publishing and marketing industries show no such sudden downturn.
So there is a marked contrast in the fortunes of those in occupations thought to be at similar risk. This finding meshes neatly with a new study from San Francisco-based AI research company METR, which offers a new framework for understanding AI’s strengths, weaknesses and rate of progress.
It finds that LLMs’ ability to perform a given task is a function not so much of how intellectually challenging the same job would be for you or me, nor of the level of specialist skill required, but of how long it would take a human and how “messy” or unstructured the workflow.
So carrying out the duties of an executive assistant, travel agent or bookkeeping clerk — all computer-based jobs requiring entry-level skills — is still beyond the capabilities of even cutting-edge AIs. They struggle to keep track of multiple streams of information, respond to a dynamic environment, work with unclear or changing goals and multitask. These unstructured workflows are a far cry from coding tests and essay questions.
This is not to say these occupations will remain a human domain. METR’s research finds AI is making strong and steady progress across a wide range of tasks, regardless of complexity, duration or “messiness”. It’s just that administrative assistants might have a year or two’s head start.
What sets programmers and writers apart is that these are occupations where the entire job from start to finish — not just discrete constituent parts — is about as close as possible to what AI excels at: nice, clean, linear and sequential tasks, exam-style questions and essay assignments. Notably, both jobs also have high rates of contracting or freelancing. So an AI assistant such as Anthropic’s Claude can be swapped in for a non-staff copywriter without HR getting involved.
Another way of thinking about it is that the protective “messiness” in some jobs comes from the back-and-forth and unpredictability inherent in interacting with other people. There is a certain irony in the realisation that the mantra of rugged self-reliance and workflow optimisation common in Silicon Valley may have made tech roles more, not less fragile.
The occupation you really wouldn’t want to be in right now is one where your activities consist of a predictable recurring linear task. Writing code to analyse data and then synthesising the results into an article of a fixed length, say. Oh dear. The future for someone in this type of work — a data-driven columnist — looks bleak.
AI is transforming the world of work, are we ready for it? | FT Working It
Letter in response to this article:
I just don't get how people believe this. I've been in data/software for 12 years now, and every time I ask an LLM to do something relatively simple in this area it fails massively.
1. Coronavirus over hiring for a future of mainly remote work didn’t transpire and the job market is now regulating.
2. Overinvestment in AI not leading to immediate profit, it’s been two years and no company yet would still release an AI-created product (bar AI companies whose product is AI) to market and stand over it without human review first. Shareholders need to see return for their buck and the easiest way to achieve this is to reduce numbers while waiting for the AI investment eggs to hatch.