As someone who has a 18 yo son who wants to study math, this has me (and him)
... worried ... about becoming obsolete?
But I'm wondering what other people think of this analogy.
I used to be a bench scientist (molecular genetics).
There were world class researchers who were more creative than I was. I even had a Nobel Laureate once tell me that my research was simply "dotting 'i's and crossing 't's".
Nevertheless, I still moved the field forward in my own small ways. I still did respectable work.
So, will these LLMs make us completely obsolete? Or will there still be room for those of us who can dot the "i"?--if only for the fact that LLMs don't have infinite time/resources to solve "everything."
I don't know. Maybe I'm whistling past the graveyard.
I was just thinking about this. I already posted a comment here, but I will say that as a mathematician (PhD in number theory), that for me, AI signficantly takes away the beauty of doing mathematics within a realm in which AI is used.
The best part of math (again, just for me) is that it was a journey that was done by hand with only the human intellect that computers didn't understand. The beauty of the subject was precisely that it was a journey of human intellect.
As I said elsewhere, my friends used to ask me why something was true and it was fun to explain it to them, or ask them and have them explain it to me. Now most will just use some AI.
Soulless, in my opinion. Pure mathematics should be about the art of the thing, not producing results on an assembly line like it will be with AI. Of course, the best mathematicians are going into this because it helps their current careers, not because it helps the future of the subject. Math done with AI will be a lot like Olympic running done with performance-enhancing drugs.
Yes, we will get a few more results, faster. But the results will be entirely boring.
There are many similarities in your comment to how grandmasters discuss engines. I have a hunch the arc of AI in math will be very similar to the arc of engines in chess.
I agree with that, in the sense that math will become more about who can use AI the fastest to generate the most theories, which sort of side-steps the whole point of math.
As a chess aficionado and a former tournament player, who didn’t get very far, I can see pros & cons. They helped me train and get significantly better than I would’ve gotten without them. On the other hand, so did the competition. :) The average level of the game is so much higher than when I was a kid (30+ years ago) and new ways of playing that were unthinkable before are possible now. On the other hand cheating (online anyway) is rampant and all the memorization required to begin to be competitive can be daunting, and that sucks.
Hey I play chess too. Not a very good player though. But to be honest, I enjoy playing with people who are not serious because I do think an overabundance of knowledge makes the game too mechanical. Just my personal experience, but I think the risk of cheaters who use programs and the overmechanization of chess is not worth becoming a better player. (And in fact, I think MOST people can gain satisfaction by improving just by studying books and playing. But I do think that a few who don't have access to opponents benefit from a chess-playing computer).
Presumably people who get into math going forward will feel differently.
For myself, chasing lemmas was always boring — and there’s little interest in doing the busywork of fleshing out a theory. For me, LLMs are a great way to do the fun parts (conceptual architecture) without the boring parts.
And I expect we’ll such much the same change as with physics: computers increase the complexity of the objects we study, which tend to be rather simple when done by hand — eg, people don’t investigate patterns in the diagrams of group(oids) because drawing million element diagrams isn’t tractable by hand. And you only notice the patterns in them when you see examples of the diagrams at scale.
Just a counterpoint, but I wonder how much you'll really understand if you can't even prove the whole thing yourself. Personally, I learn by proving but I guess everyone is different.
My hunch is it won't be much different, even when we can simply ask a machine that doesn't have a cached proof, "prove riemann hypothesis" and it thinks for ten seconds and spits out a fully correct proof.
As Erdos(I think?) said, great math is not about the answers, it's about the questions. Or maybe it was someone else, and maybe "great mathematicians" rather than "great math". But, gist is the same.
"What happens when you invent a thing that makes a function continuous (aka limit point)"? "What happens when you split the area under a curve into infinitesimal pieces and sum them up"? "What happens when you take the middle third out of an interval recursively"? "Can we define a set of axioms that underlie all mathematics"? "Is the graph of how many repetitions it takes for a complex number to diverge interesting"? I have a hard time imagining computers would ever have a strong enough understanding of the human experience with mathematics to even begin pondering such questions unprompted, let alone answer them and grok the implications.
Ultimately the truths of mathematics, the answers, soon to be proved primarily by computers, already exist. Proving a truth does not create the truth; the truth exists independent of whether it has been proved or not. So fundamentally math is closer to archeology than it may appear. As such, AI is just a tool to help us dig with greater efficiency. But it should not be considered or feared as a replacement for mathematicians. AI can never take away the enlightenment of discovering something new, even if it does all the hard work itself.
> I have a hard time imagining computers would ever have a strong enough understanding of the human experience with mathematics to even begin pondering such questions unprompted, let alone answer them and grok the implications.
The key is that the good questions however come from hard-won experience, not lazily questioning an AI.
Even current people will feel differently. I don't bemoan the fact that Lean/Mathlib has `simp` and `linarith` to automate trivial computations. A "copilot for Lean" that can turn "by induction, X" or "evidently Y" into a formal proof sounds great.
The the trick is teaching the thing how high powered of theorems to use or how to factor out details or not depending on the user's level of understanding. We'll have to find a pedagogical balance (e.g. you don't give `linarith` to someone practicing basic proofs), but I'm sure it will be a great tool to aid human understanding.
A tool to help translate natural language to formal propositions/types also sounds great, and could help more people to use more formal methods, which could make for more robust software.
We also seem to suffer these automation delusions right now.
I could see how AI could assist me with learning pure math but the idea AI is going to do pure math for me is just absurd.
Not only would I not know how to start, more importantly I have no interest in pure math. There will still be a huge time investment to get up to speed with doing anything with AI and pure math.
You have to know what questions to ask. People with domain knowledge seem to really be selling themselves short. I am not going to randomly stumble on a pure math problem prompt when I have no idea what I am doing.
I agree wholeheartedly about the beauty of doing mathematics. I will add though that the author of this article, Kevin Buzzard, doesn't need to do this for his career and from what I know of him is somebody who very much cares about mathematics and the future of the subject. The fact that a mathematician of that calibre is interested in this makes me more interested.
I think it will become apparent how bad they are at it. They’re algorithms and not sentient beings. They do not think of themselves, their place in the world, and do not fathom the contents of the minds of others. They do no care what others think of them.
Whatever they write only happens to contain some truth by virtue of the model and the training data. An algorithm doesn’t know what truth is or why we value it. It’s a bullshitter of the highest calibre.
Then comes the question: will they write proofs that we will consider beautiful and elegant, that we will remember and pass down?
Or will they generate what they’ve been asked to and nothing less? That would be utterly boring to read.
If you looked at how the average accountant spent their time before the arrival of the digital spreadsheet, you might have predicted that automated calculation would make the profession obsolete. But it didn't.
This time could be different, of course. But I'll need a lot more evidence before I start telling people to base their major life decisions on projected technological change.
That's before we even consider that only a very slim minority of the people who study math (or physics or statistics or biology or literature or...) go on to work in the field of math (or physics or statistics or biology or literature or...). AI could completely take over math research and still have next to impact on the value of the skills one acquires from studying math.
Or if you want to be more fatalistic about it: if AI is going to put everyone out of work then it doesn't really matter what you do now to prepare for it. Might as well follow your interests in the meantime.
It's important to base life decisions on very real technological change. We don't know what the change will be, but it's coming. At the very least, that suggests more diverse skills.
We're all usually (but not always) better off, with more productivity, eventually, but in the meantime, jobs do disappear. Robotics did not fully displace machinists and factory workers, but single-skilled people in Detroit did not do well. The loom, the steam engine... all of them displaced often highly-trained often low-skilled artisans.
If AI reaches this level socioeconomic impact is going to be so immense, that choosing what subject you study will have no impact on your outcome - no matter what it is - so it's a pointless consideration.
That's just about the silliest thing I've read in a long time.
We've had changes before, the most recent one being the rise of computers and then the internet, and before that, manufacturing automation. In all cases, some people were better prepared for change, and some less so.
The general consensus is that diverse skills and foundational skills (e.g. math, communication) best prepare people for transitions, relative to specialized skills (e.g. one technology). In addition, many careers are likely to be less impacted, such as plumbing.
If we reach superhuman AGI the best preparation you can make is weapons, combat training and maybe look to build/join a militia to do a quick takeover while it's still ramping up. Society is built on cooperation outperforming violence - there's 0 chance that holds past real AGI - game theory works very different when we aren't living in an iterated prisoners dilemma.
Let's put it this way, from another mathematician, and I'm sure I'll probably be shot for this one.
Every LLM release moves half of the remaining way to the minimum viable goal of replacing a third class undergrad. If your business or research initiative is fine with that level of competence then you will find utility.
The problem is that I don't know anyone who would find that useful. Nor does it fit within any existing working methodology we have. And on top of that the verification of any output can take considerably longer than just doing it yourself in the first place, particularly where it goes off the rails, which it does all the time. I mean it was 3 months ago I was arguing with a model over it not understanding place-value systems properly, something we teach 7 year olds here?
But the abstract problem is at a higher level. If it doesn't become a general utility for people outside of mathematics, which is very very evident at the moment by the poor overall adoption and very public criticism of the poor result quality, then the funding will dry up. Models cost lots of money to train and if you don't have customers it's not happening and no one is going to lend you the money any more. And then it's moot.
I think there's a pretty good case to be made that LLMs paired with automated theorem provers will become a useful tool to working mathematicians in the next few years. Another thread here links to a lecture from a professor of mathematics who makes this point about halfway in, based solely on Alpha Proofs current abilities (https://www.youtube.com/watch?v=vYCT7cw0ycw [54min]). Terence Tao, well-known mathematician at UCLA has been saying similar things for years. He's blogged about LLMs helping to learn new tools (like Lean) and occasionally helping with brainstorming.
At this stage, the point they're making isn't 'OMG AGI!!!' but rather something like 'having an enthusiastic, often wrong undergrad assistant who's available 24/7 can be useful, if you use it carefully.'
Well said. As someone with only a math undergrad and as a math RLHF’er, this speaks to my experience the most.
That craving for an understanding an elegant proof is nowhere to be found with verifying an LLM’s proof.
Like sure, you could put together a car by first building an airplane, disassembling all of it minus the two front seats, and having zero elegance and still get a car at the end. But if you do all that and don’t provide novelty in results or useful techniques, there’s no business.
Hell, I can’t even get a model to calculate compound interest for me (save for the technicality of prompt engineering a python function to do it). What do I expect?
This is a great point that nobody will shoot you over :)
But the main question is still: assuming you replace an undergrad with a model, who checks the work? If you have a good process around that already, and find utility as an augmented system, then get you’ll get value - but I still think it’s better for the undergrad to still have the job and be at the wheel, and does things faster and better when leveraging a powerful tool.
Shot already for criticising the shiny thing (happened with crypto and blockchain already...)
Well to be fair no one checks what the graduates do properly, even if we hired KPMG in. That is until we get sued. But at least we have someone to blame then. What we don't want is something for the graduate to blame. The buck stops at someone corporeal because that's what the customers want and the regulators require.
That's the reality and it's not quite as shiny and happy as the tech industry loves to promote itself.
My main point, probably cleared up with a simple point: no one gives a shit about this either way.
What LLMs can do is limited, they are superior to wet-wear in some tasks like finding and matching patterns in higher dimensional space, they are still fundamentally limited to a tiny class of problems outside of that pattern finding and matching.
LLMs will be tools for some math needs and even if we ever get quantum computers will be limited in what they can do.
LLMs, without pattern matching, can only do up to about integer division, and while they can calculate parity, they can't use it in their calculations.
There are several groups sitting on what are known limitations of LLMs, waiting to take advantage of those who don't understand the fundamental limitations, simplicity bias etc...
The hype will meet reality soon and we will figure out where they work and where they are problematic over the next few years.
But even the most celebrated achievements like proof finding with Lean, heavily depends on smart people producing hints that machines can use.
Basically lots of the fundamental hints of the limits of computation still hold.
Model logic may be an accessable way to approach the limits of statistical inference if you want to know one path yourself.
A lot of what is in this article relates to some the known fundamental limitations.
Remember that for all the amazing progress, one of the core founders of the perceptron, Pitts drank him self to death in the 50s because it was shown that they were insufficient to accurately model biological neurons.
Optimism is high, but reality will hit soon.
So think of it as new tools that will be available to your child, not a replacement.
"LLMs, without pattern matching, can only do up to about integer division, and while they can calculate parity, they can't use it in their calculations." - what do you mean by this? Counting the number of 1's in a bitstring and determining if it's even or odd?
The point being that the ability to use parity gates is different than being able to calculate it, which is where the union of the typically ram machine DLOGTIME with the circuit complexity of uniform TC0 comes into play.
PARITY, MAJ, AND, and OR are all symmetric, and are in TCO, but PARITY is not in DLOGTIME-uniform TC0, which is first-order logic with Majority quantifiers.
Another path, if you think about symantic properties and Rice's theorem, this may make sense especially as PAC learning even depth 2 nets is equivalent to the approximate SVP.
PAC-learning even depth-2 threshold circuits is NP-hard.
For me thinking about how ZFC was structured so we can keep the niceties of the law of the excluded middle, and how statistics pretty much depends on it for the central limit and law of large numbers, IID etc...
But that path runs the risk of reliving the Brouwer–Hilbert controversy.
Highly recommend this lecture by a working mathematician shared above (https://www.youtube.com/watch?v=vYCT7cw0ycw [54min]). It's very much grounded in history and experience, much more so than in speculation. I wrote a brief summary of some main points in that thread.
But specifically to your worry about humans just dotting i's and crossing t's, he predicts that exactly the opposite will happen. At the end he emphasizes that the ultimate goal of mathematics is more about human understanding than proving theorems.
Most likely AI will be good at some things and not others, and mathematicians will just move to whatever AI isn't good at.
Alternatively, if AI is able to do all math at a level above PhDs, then its going to be a brave new world and basically the singularity. Everything will change so much that speculating about it will probably be useless.
I used to do bench top work too; and was blessed with “the golden hands” in that I could almost always get protocols working. To me this always felt more like intuition than deductive reasoning. And it made me a terrible TA. My advice to students in lab was always something along the lines of “just mess around with it, and see how it works.” Not very helpful for the stressed and struggling student -_-
Digression aside, my point is that I don’t think we know exactly what makes or defines “the golden hands”. And if that is the case, can we optimize for it?
Another point is that scalable fine tuning only works for verifiable stuff. Think a priori knowledge. To me that seems to be at the opposite end of the spectrum from “mess with it and see what happens”.
Another PhD in maths here and I would say not to worry. It's the process of doing and understanding mathematics, and thinking mathematically that is ultimately important.
There's never been the equivalent of the 'bench scientist' in mathematics and there aren't many direct careers in mathematics, or pure mathematics at least - so very few people ultimately become researchers. Instead, I think you take your way of thinking and apply it to whatever else you do (and it certainly doesn't do any harm to understand various mathematical concepts incredibly well).
What part do you think is going to become obsolete? Because Math isn't about "working out the math", it's about finding the relations between seemingly unrelated things to bust open a problem. Short of AGI, there is no amount of neural net that's going to realize that a seemingly impossible probabilistic problem is actually equivalent to a projection of an easy to work with 4D geometry. "Doing the math" is what we have computers for, and the better they get, the easier the tedious parts of the job become, but "doing math" is still very much a human game.
> What part do you think is going to become obsolete?
Thank you for the question.
I guess what I'm saying is:
Will LLMs (or whatever comes after them) be _so_ good and _so_ pervasive that we will simply be able to say, "Hey ChatGPT-9000, I'd like to see if the xyz conjecture is correct." And then ChatGPT-9000 just does the work without us contributing beyond asking a question.
Or will the technology be limited/bound in some way such that we will still be able to use ChatGPT-9000 as a tool of our own intellectual augmentation and/or we could still contribute to research even without it.
Hopefully, my comment clarifies my original post.
Also, writing this stuff has helped me think about it more. I don't have any grand insight, but the more I write, the more I lean toward the outcome that these machines will allow us to augment our research.
As amazing as they may seem, they're still just autocompletes, it's inherent to what an LLM is. So unless we come up with a completely new kind technology, I don't see "test this conjecture for me" becoming more real than the computer assisted proof tooling we already have.
The mathematicians of the future will still have to figure out the right questions, even if llms can give them the answers. And "prompt engineering" will require mathematical skills, at the very least.
Evaluating the output of llms will also require mathematical skills.
But I'd go further, if your son enjoys mathematics and has some ability in the area, it's wonderful for your inner life. Anyone who becomes sufficiently interested in anything will rediscover mathematics lurking at the bottom.
I was referring to Linus's harmful and evil promotion of Vitamin C as the cure for everything and cancer. I don't think Linus was attaching that garbage to any particular Nobel prize. But people did say to their doctors: "Are you a Nobel winner, doctor?". Don't think they cared about particular prize either.
Which is "harmful and evil" thanks to your afterknowledge. He had based his books on the research that failed to replicate. But given low toxicity of vitamin C it's not that "evil" to recommend treatment even if probabilistic estimation of positive effects is not that high.
Sloppy, but not exceptionally bad. At least it was instrumental in teaching me to not expect marvels coming from dietary research.
If Pauling's eugenics policies were bad, then the laws against incest that are currently on the books in many states (which are also eugenics policies that use the same mechanism) are also bad. There are different forms of eugenics policies, and Pauling's proposal to restrict the mating choices of people carrying certain recessive genes so their children don't suffer is ethically different from Hitler exterminating people with certain genes and also ethically different from other governments sterilizing people with certain genes. He later supported voluntary abortion with genetic testing, which is now standard practice in the US today, though no longer in a few states with ethically questionable laws restricting abortion. This again is ethically different from forced abortion.
FWIW my understanding is that the policies against incest you mention actually have much less to do with controlling genetic reproduction and are more directed at combating familial rape/grooming/etc.
Not a fun thing to discuss, but apparently a significant issue, which I guess should be unsurprising given some of the laws allowing underage marriage if the family signs off.
Mentioning only to draw attention to the fact that theoretical policy is often undeniable in a vacuum, but runs aground when faced with real world conditions.
This is mentioned in my link: "According to Pauling, carriers should have an obvious mark, (i.e. a tattoo on the forehead) denoting their disease, which would allow carriers to identify others with the same affliction and avoid marrying them."
The goal wasn't to mark people for ostracism but to make it easier for people carrying these genes to find mates that won't result in suffering for their offspring.
But I'm wondering what other people think of this analogy.
I used to be a bench scientist (molecular genetics).
There were world class researchers who were more creative than I was. I even had a Nobel Laureate once tell me that my research was simply "dotting 'i's and crossing 't's".
Nevertheless, I still moved the field forward in my own small ways. I still did respectable work.
So, will these LLMs make us completely obsolete? Or will there still be room for those of us who can dot the "i"?--if only for the fact that LLMs don't have infinite time/resources to solve "everything."
I don't know. Maybe I'm whistling past the graveyard.