The Navier–Stokes Problem and the Agents

0:00 This podcast is presented by NXAI, your partner for time series foundation models and physical AI. Hello everybody! And welcome to a new episode of our Industrial AI Podcast. my name's Hubert Weber and in Linz Günther Klambauer is waiting for me. Günter Welcome to the podcast. thanks for the invitation Robert.

0:29 it's great to be here again happy to talk about AI and with you and mathematics. Yeah, exactly because we wanted to record an episode on AI LLM stuff in Mathematics And he said We should invite a very special A very prominent voice In this area Gonzalo Cualabora. I have two more days To do that. You said i will write him a message.

0:55 Here we are Gonzalo welcome to the podcast. Hello, thank you very much for the invitation and it's a pleasure to be here. Yeah thanks a lot! You're based in Lausanne right? Yes I'm based in Losan Switzerland. Okay perfect before we start talking about AI and mathematics can you please introduce yourself briefly to the listeners Gonzalo?

1:12 So i am a mathematician by training And I have lived there In the US for awhile doing my PhD at MIT Also living in New York For A bit Basically now working in EPFL in Switzerland. all my life trying to understand partial differential equations that model evolution of fluids, evolution of gases. And I'm also very interested in the development of AI techniques... ...to advance understanding these equations.

1:42 Günther, short introduction? Yes! I am a professor at Johannes Kepler University for artificial intelligence and yeah.. ..I speak about all topics related to AI Very prominent And you like the format, right? You'd like to talk in a podcast. Yeah Yes exactly that's what I wanted. That's why we invite you always yeah.

2:10 So Gonzalo first question The easiest question did OpenAI actually solve the Navian Stokes problem? yes so i think i believe this thing should be carefully checked and i don't to have a final write-up that every mathematician would agree with. But I think everyone in the community is very inclined to believe at this moment, That This Is True but of course spending official final confirmation.

2:37 What's your opinion? My Opinion...is that The proof actually develops Very interesting new ideas And it's incredible..the New Ideas that having developed by suggested proof. On the other hand I should also say that the writing is extremely poor and it's not presented in a way, there are professional mathematician good.

2:59 That's also the reason why he's taking us so long to the mathematical community To actually understand and confirm that this proof is right. But you're a bit skeptical here? And i wouldn't say sceptical because...I mean..i have gone through The Proof , i've spent A big part of last month trying to understand It ..and am fairly confident that its correct.

3:21 What I'm is not satisfied with the presentation of it. There is a very interesting paper of Peter Konstantin, Mikhail Lagnatov and Vlad Vickle where actually they do some further work on this paper. And it's really interesting to read their AI statement because we have significantly used AI in writing these papers.

4:04 but by far the task that has required us to use most AI was just to decipher the proof released by AI. Are large-language models becoming great mathematicians or are they just become extremely good at manipulating mathematical language? What is your opinion. Yes, I think a bit of both. so it's first of all... ...I think that becomes very good because there's lots of material to train as the mathematics community was open and shared a lot of manuscripts And you can always present something like new mathematical problems to the system and it learns to solve them based on what they have.

4:44 And then also this lean, I think that is something... This lean programming language is something in favor of large-language models. There are these drawbacks with readability which has multiple reasons. So first off all training reinforcement learning posttraining goes in a direction where you have to solve correctly And not that it has to transport itself well into the community and so on.

5:11 So this is one thing I think doesn't really, It's really not reinforced To write this in a very good method and understandable manner. The other things also are. kind of brain of the LLMs works bit differently than ours because they have this KV cache. They basically can store lots words from context and look back memorize almost all the things and results that they have already written, whereas our memory is a bit leaky.

5:42 We don't have everything exactly. so when we write or represent paper's publications... For example if you repeat concepts of pickup constructions in certain ways for many other reasons than mathematical publications. in general publications are better written or differently written by humans. And I fully understand concerns by Gonzalo and the mathematical community, and also have very ambiguous feelings towards this not only the Navier stokes but also toward other problems that have allegedly solved.

6:21 Gonzalo you mentioned proving right? So the proof? can I discover mathematics or is it just getting better at proving its? so can you explain that a little bit? what's your opinion. So i think for the moment he's getting better and proven things. under has been out of talk about these. come back school idea which basically means, of the mathematical literature that no mathematician would be able to know all those different ideas because and then combine them in a very, very smart way which already allows you to develop incredible proofs.

7:04 But of course this is very uncertain because given their rate of progress it's pretty much likely than in one year they are even still much better unable. that as far as I know. and collaborators, no they are completely new. So already they don't appear in previous mathematical literature. Yeah good you want to add something?

7:27 Yes so first of all i think this is a very good thought by Gonzalo. different mathematical tricks. So I have an undergraduate that's studying mathematics, so i dare to say something about it! You often have some tricks. you introduce variables or a concept or transform variables into another space and there are different kinds of trick in each mathematical community.

7:52 maybe each mathematician has his own set of tricks and of course an AI can see them all and use them in different contexts. That's what Gonzalo means, And I think that is very valuable to connect this. Yeah the other question is...and i'm very curious here. So I think the initial idea Gonzalo was that this Navier-Stokes proof follows something like What they call it?

8:17 The Spanish program or Something Like That? It doesn't appear so Is This? What's Your Take on This? that's mathematically duty for me and I cannot follow so it would be great to get some insights from you there. builds much more heavily, I would say in the Cordoba-Martinezoro approach. And then OpenAI also recognized that they... In order to solve Navier's talks They feed it with a proof of Euler.

9:05 So there is some reminiscence for ideas and you could see where. But if not i wouldn't go as far As saying this directly because There are also incredible amount new ideas. For example Half of the paper goes into a different strategy that then is merged with this strategy. So, that help has nothing to do and in other half you can really see where these ideas could come from the Spanish program.

9:32 Günther I want to come back for our listeners because they are not so deep on the whole topic but it's maybe the domain mathematics perfect domain research AI driven, research AI discoveries because it's a domain with clear structure and clear processes. Do you know what I mean? So that mathematics is maybe the best domain to prove something there.

10:02 Yeah! Maybe together with coding... That one of the domains can profit from these post training strategies. so give an overview for people that not only OpenAI has announced they solved this Navier Stokes problem, but also they've uploaded seven hundred twenty-two additional manuscripts where it claimed to have solved some other math problems.

10:28 They were open and there was three of the manuscripts they had to withdraw. so I think readability is difficult as well. formalized lean proof, some have not. This has led to a big discussion I think in the mathematical community and generally. what's the impact of this? And how will this field change?

10:54 why is it working so well? i think because you can always present kind of new problems. So these language models learn first natural languages but also programming language. They can also learn on mathematical papers formal that we repeat stuff and introduce concepts, so on. And now think a bit back to your high school.

11:18 you also got math small math from really simple ones to solve but basically your teacher could always come up with new ones at these numbers or in the text change slides things. So this is very good for AI That you have basically infinite amount of training data. You cannot memorize anything. you always get a new problem.

11:40 And I think that's what they're doing with this, for this model is it unreleased? That they presented a lot of mathematical problems and then he tries to solve it. What the good thing is You can verify if the solution is correct. right. If It's another field For example write an article about blah-blah-blAH!

11:57 You cannot verify yes or no if The article Is correct Or not. but here if the mathematical Problem has been solved or whether the program in some whatever Python code or C++ code, you can check if the thing is correct. Also with the help of Lean for example and that's a very good training signal for the neural network to learn from because these are two domains where you have verifiable rewards Gonzalo, from your point of view.

12:34 How much of the discovery came from AI and how much dependent on human mathematical intuition? Could a powerful language model have achieved the same results independently? What is your opinion on that?". So what it's easy to think about here are counterfactual. where AI isn't developed at the current level I think that would take decades for us, i don't see any way mathematicians could be able to get there soon.

13:06 It's more difficult to judge whether AI without previous mathematical literature will be able To arrive at this proof because of course AI is trained on millions of papers. so then Which papers do we remove from the literature? Is a hard question. Yeah, I think it's. if you hadn't seen any or any of the recent results.

13:26 I think It would not be possible. and we see also that he can solve certain problems And some problems are. yeah That didn't progress. that was there? Also fields our areas where it worked better and less so and also The energy invested was gigantic. So i think at least twenty three million euros Of compute were invested for this navier stokes stuff and yeah the price is one million euro by the way, not euros but dollars.

13:59 Yeah I think it will have an impact on mathematics. how they would proceed. I think Terry Tao said this will become more industrialized in larger groups working maybe together with AI. We've seen other scientific fields like molecular biology where that is more industrialized. There are big consortia working on problems and also in physics, you have CERN.

14:24 so they can make progress by making big industrialised research some way because there's a big consortium big machines basically. So maybe this is a trend, but yeah I'm again that's an interesting question. do we will see in industrializing or industrialization of mathematics Gonzalo? Yes i pretty. it's the only possibility thing because basically anything that becomes automatized immediately loses its value you know and therefore mathematicians will dedicate themselves to actually understand these proofs.

14:59 trying And that job, at least for now. the LLMs are much worse than doing proofs. Of course it's uncertain what capabilities they will have in a couple of years but Can you say something, why it's so difficult to read? So I think its really good this point that you raised earlier about the memory fact.

15:30 Like LLMs are able to memorize incredible amounts of data and for example when do look at the Navier-Stokes proof there is an incredible amount of different variables. no mathematician would write a paper like that. There're actually five pages Of the papers That just table with all the different notations.

15:50 Of course, this is something that for artificial intelligence probably it's not a problem. But for mathematics and you just read about any technology when you are on the second page You have argued at AI as becoming remarkably good at construction explicit mathematical examples but still struggles was developing entirely new theories.

16:10 Is That The Fundamental Limitation of Todays AI Gonzalo? It's hard to talk about fundamental limitation. I think it's a limitation of today's AI. I wouldn't say fundamental because maybe we talk one year from now and this limitation is removed, but as the AI that exists nowadays on specifically when we can access not the internal models?

16:34 I think That's our limitation. Mm-hmm What does your opinion go into? yeah i think especially like Yeah It's also very smart combination sometimes have different tricks that in that probably no human can grasp across the fields and by that, combining things often something new comes out. So I think it's especially connecting between different fields is good with AI.

17:00 but what I see maybe not so much in mathematics very disruptive ideas. they are not represented well or an LLM will not do this if somebody has a really a counter-opinion like that. I mean, for Navier stokes it's probably hard to... Maybe in other fields. think back For example In the Middle Ages when the common opinion was That yeah The Earth is in the middle.

17:30 Then and then the Sun rotates around earth And LLM would have probably believed very strongly based on the literature this disruptive idea that the heliocentric world model, and this is a remarkable thing by Galileo like Galilee. And so these counter opinions or unpopular opinion very rare opinions I think will be hard for AI...I don't know if it's relevant from mathematics but In machine learning, for example it would always recommend probably to approach new problems with transformer architectures because they're so powerful.

18:12 Whereas in some areas. this is probably not an end and fields experts in the field will have no I will approach these problem completely different architecture or even within non-machine learning approach. And yeah maybe that's such a disruptive idea but very unpopular ideas. and these are often very trans The most important ideas that transform or open new fields of sciences, right?

18:36 If you think about that. And I mean also Einstein's ideas were very unpopular and criticized when he came up with it. so but i could not come up with a good example here in mathematics where somebody came up something very unpopulated then proved. Gonzalo do have an idea. Yes! I had some ideas in mind.

18:57 specifically look at new understanding that simplifies previously existing proofs. So say that now you think, okay the proof has been done of Navier-Stokes is extremely complicated. Let's try to simplify this approach. I don't think the current AI is good at that and i think part of the problem Is that it just led To the old approach because he's the one That has learned.

19:22 but probably You want to simplify This? And probably also But you cannot verify whether an Approach is simpler than another At least when try to work with LLMs, they are not extremely good at differentiating what is a simplified version of the same argument. So I could see how in this kind of works where you actually tried to simplify an existing method... ...the artificial intelligence isn't as good.

19:47 Gonzalo! At the end if you were starting your career mathematics today would be excited or variate? I think it will be more exciting maybe because i think that situation is very uncertain right now But it will stabilize. So if you're starting your career now, probably you are starting a PhD. You have four or five years and I think the situation of mathematics Will be much more clear in for our five years?

20:12 i think The people that feel bad for other ones That are applying for jobs this year For example there is an extremely uncertain position to begin. What happened in the next five years. Definitely, proofs will become completely automatized and I don't think mathematicians will be proving theorems anymore.

20:32 probably we still want to encourage mathematicians in their training phase to be proven theorems. just the same way we teach people to multiply in schools but of course this can be automatized. And i think mathematician's understanding of mathematics, it will become less specialized because we'll be able to digest mathematics in many more areas with the help of LLMs.

20:59 And... We also probably center ourselves on trying what are good questions and how do we advance mathematical knowledge? What's the application for that as well? Yes, I have the good questions. Now we think another effect of artificial intelligence is that people are less willing to share good questions and That's something i'm not especially happy about.

21:29 but I think it will happen Because now everyone can try to answer those questions with AI. so therefore It becomes a problem because if you just shared your question anyone may actually get this solution in couple days. Well, in the pre-ai era normally you would be very happy to share your questions because You had already been working of them for one year or two years and you knew that other people were not going to try to scoop you Or trying to answer those question before you.

21:57 If you start a career in mathematics today. Would you be excited over it? Yeah I never had a career. I think i would be excited, but also a bit worried where this field is going. But at times of uncertainty it's just the time for uncertainty. how this is transforming... ...but not only in mathematics and people that are now finishing school or studies.

22:31 there're lots of uncertainties as well. Not only in Mathematics but other fields. will AI take my job? Should I study at all?". And yeah, I think the transformation that AI brings to general all jobs basically. We will see it in next years and then when the dust has settled we'll see what important skills are you have?

22:58 analytical skills, the mathematical skills that we teach people in school and on universities. This will still prevail maybe to work efficiently with AI together and so on but I think these things are... Will remain important. Nathalo what is your research agenda for next coming months? Give us a short overview!

23:20 So i think one question that am specifically interested this okay and what new questions Do the Navier Stokes blow up give us? And I think there are several important questions. Can you share one? so for example, One thing that i'm interested in is if You could adopt this technology to try To develop a singularity as the navier stokes proof In different physical situations but basically using The same ideas from open AI very in A Different configuration.

23:51 This Could Give You Perhaps blow up that is more physical. So one thing there's not very satisfactory of the proof they released, it seems to be extremely unphysical. you tune your force in a very delicate way and probably try removing them. Yeah, many things. My agenda is still to have more efficient alternatives to the transformer of course... ...to finally kick that thing out off the AI systems and many other research ideas in not only regarding LLMs, but also AI systems.

24:32 In other fields like always molecular biology biochemistry on. so I don't want to share here because otherwise it gets cool. What is the next domain they are focusing? It's a bit more difficult, of course where you cannot deduct stuff by combining previous results. Some things we just don't know about diseases and how they develop for some at least And there is no way to combine the existing knowledge You have make an experiment or something like that in a way harder for AI to make progress without experimentation.

25:34 The physical world and also I highly, highly recommend the episode with the dunya guys from Berlin. they build Jim To train agents into trained II on experiments. so how do you recommend the episodes? I think four weeks yeah. which is there? They can predict Which experiment to do next. we all see that in physics And under which conditions and so on.

25:56 So i'm very excited For these phases, of course also this egentic systems are able to do it. Also certain ways. some labs in biochemistry already a bit automated with robotic arms and so on. So yeah I think they will see lots progress Yeah but whether the LLAMs or something like that They're trained all over the internet right?

26:19 And now in math and coding became very good. It's very hard to predict what will be the next breakthrough. I mean, chat GPD itself is only like four years back. so simply very fast! You're right it was a pleasure. i can hear a kid in the back. he needs his father and Gonzalo it was.