Statement on LLMs

Increasingly, many of the most visible people at the top of mathematics are using large language models (LLMs) to help them produce results, and writing about it. Here are a few recent examples: Tim Gowers, an array of people, Manolescu and Rozenblyum.

The messages seem to be: (a) we need to engage with these tools, (b) we need to provide AI companies with benchmark problems to demonstrate that they aren't yet able to replace us,

Here is a countervailing view; it is my own view and, I suspect, shared by many others. It is not as prominently voiced, but I think the time has come to voice it more loudly. We need to stop feeding the bin-fire. The disruption these tools may bring is likely to be largely negative in the long term. The companies that are developing the tools have no accountability, as we see when we look at the medium-term effects of social media on society and on individuals. Anything we do to engage will simply push up the already-inflated share prices of these companies and accelerate the damage they are able to do.

Providing companies with benchmarks does not mean they will exercise caution. It means that when some expert mathematicians are able to guide an LLM to solve one of the benchmark problems (not necessarily a particularly difficult one) the tech companies will be able to shout about the abilities of their latest LLM. You think you are acting as a check/balance on power, but the shareholders and the general public do not understand the distinction between "Expert mathematicians show that an LLM can now be used to solve minor research-level problems by putting together ideas from across the literature" and "Expert mathematicians can now use LLMs to solve all their problems". A food safety inspector awards a restaurant one star; the restaurant tells its customers they have been awarded a gold star.

The counterargument I have heard from several people (from postdocs through to professors) is that LLMs will have a democratising influence, or help with inclusivity. The argument runs as follows. I have a problem which requires input from a different, neighbouring field. Maybe an expert in this field could solve my question in minutes. But maybe the expert is busy, or uninterested, or maybe the expert is unlikely to respond because the person asking is a lowly PhD student. There could be any number of reasons you might not want to ask, ranging from anxiety through embarrassment. But the LLM is always there and can answer without any of these interpersonal complications. Moreover, you don't have to go to the trouble of trying to communicate with someone in a different area, who might take some time to figure out what you're asking, because of slight differences between areas.

I think there are two strong reasons why this is a dangerous and flawed narrative.

First, I wouldn't trust the democratisation of knowledge to the companies that run these models. They have no interest in it. Their interests are in maximising shareholder value and developing a position of market dominance. If they are helped by mathematicians to produce a tool which becomes widely adopted by the research community then, in the long run, they will have power over us, and be able to charge our universities exorbitant amounts to use the most cutting-edge versions. The obvious precedent is academic publishing companies, who once provided a valuable service in making papers more widely available, and who now act as parasitic leaches on library budgets.

The second reason is this: if we replace human interaction with machine interaction, we will exacerbate the big problems our fragmented discipline already experiences. Meaningful interaction between different areas of mathematics is hard, because different people think in different way, and struggle to communicate with one another. If we don't make an effort to do this, then we will become increasingly solipsistic. If the computer just provides the answer to the question we have asked, then we won't have anyone to push back and say "have you thought about it like this instead?". The different perspective other experts provide can make us aware of new, more promising avenues of thought, or make us aware of the narrowness of our scope. And just as importantly, it can make them aware that someone from a different area is interested in what they are doing, for a reason they may not have realised. Communication between different parts of mathematics (and beyond) is central to the health of the discipline, and I worry that turning to LLMs will only worsen the situation.

You don't need to look far for examples where very fruitful connections have been uncovered by mathematicians who were able to bring together people from different areas. Atiyah in the 1980s made an effort to communicate between geometers and physicists, and this led to one of the most exciting and fertile periods in geometry, with the discoveries of Donaldson, Witten and many others. More recently, around the time when I was a PhD student, there was a lot of activity around extremal Kaehler metrics, on the interface between differential geometry and algebraic geometry, which has led to the community of people working in K-stability, and important advances in moduli theory. This has been described as "one of the most precious gifts differential geometers brought to algebraic geometers." To a non-mathematician, these probably don't even sound like different fields of mathematics, which goes to show how fragmented our discipline has become, and how valuable it can be to communicate across the boundaries we have imposed for ourselves.

I believe that, instead of engaging with this new technology, we should be reaching out more to one another, establishing new human-to-human connections and making progress in mathematics in new and unexpected ways. We should be asking our colleagues what they are doing, and reading one anothers' papers, rather than asking a computer how to solve our pet problems and getting it to produce papers at a rate that nobody will be able to read.

Instead of telling them "it's inevitable, use it or fall behind", we should be encouraging young people in the field to grapple long and hard with their own problems. The long gestation process of a train of thought, with all its diversions and missteps, with the practice of new techniques and the working out of many examples, is as much the point of research as the final written result. The "output" is not just a paper, but the changes that happened in the brain along the way. The "output" is a person who understands something better, and can take that understanding forward in their research and their teaching.

For these reasons, I won't be using LLMs to help me do mathematics and I will be encouraging my colleagues and students to question their own use of it.