What is thinking? Metaphysics meets modern AI and the illusion of reasoning

This episode is the intro to a special project by The AI Fundamentalists’ hosts and friends. We hope you're ready for a metaphysics mini‑series to explore what thinking and reasoning really mean and how those definitions should shape AI research. 

Join us for thought-provoking discussions as we tackle basic questions: What is metaphysics and its relevance to AI? What constitutes reality? What defines thinking? How do we understand time? And perhaps most importantly, should AI systems attempt to "think," or are we approaching the entire concept incorrectly?

Show notes:

• Why metaphysics matters for AI foundations
• Definitions of thinking from peers and what they imply
• Mixture‑of‑experts, ranking, and the illusion of reasoning
• Turing test limits versus deliberation and causality
• Towers of Hanoi, agentic workflows, and brittle stepwise reasoning
• Math, context, and multi‑component system failures
• Proposed plan for the series and areas to explore
• Invitation for resources, critiques, and future guests

We hope you enjoy this philosophical journey to examine the intersection of ancient philosophical questions and cutting-edge technology.

Do you have a question or a discussion topic for the AI Fundamentalists? Connect with them to comment on your favorite topics:

• LinkedIn - Episode summaries, shares of cited articles, and more.
• YouTube - Was it something that we said? Good. Share your favorite quotes.
• Visit our page - see past episodes and submit your feedback! It continues to inspire future episodes.

Chapters

• 0:03 Intro to the metaphysics mini‑series
• 0:33 Why metaphysics for AI
• 2:24 What is thinking? Voices from friends
• 3:43 “Reasoning” in LLMs
• 5:06 Turing test vs. true thinking
• 10:55 Agentic limits and stepwise reasoning
• 12:58 Math, context, and systemic failures
• 15:33 Defining the roadmap for the series

Transcript

SPEAKER_02: 0:03

The AI Fundamentalists, a podcast about the fundamentals of safe and resilient modeling systems behind the AI that impacts our lives and our businesses. Here are your hosts, Andrew Clark and Sid Mongole. Welcome to today's episode of the AI Fundamentalists, and we are live from AutoCamp in Cape Cod, and we're really excited to bring you a new series of episodes. I'm going to let Sid tell us all about it.

SPEAKER_04: 0:33

Welcome to our new mini-series on metaphysics. This is a spiritual follow-up to a previous episode we did on consciousness with Michael Herman. And we want to talk about metaphysics, which is the study of the most general features of reality, including existence, objects, and their properties, and the possibility and necessity of space and time, change causation, really like the fundamentals of existence, right? It's like physics about physics, is the idea of metaphysics. So what underpines reality and you know what can we first define before we define anything else on top of that?

SPEAKER_03: 1:09

I think where this is really critical as we're looking back and like the state of AI, and so as we've talked about with like the GPT-5 release and kind of where we are with just AI in general, and really taking this back of what are we trying to accomplish? What really is thinking? What is reasoning? How do these pieces fit together? What does the future of AI research look like? And are we kind of approaching the problem wrong of how we're even what are we what our goals are? So just really taking a full step back. Today is just really the introduction to the this this what the series will be, where we'll really dive into what is metaphysics, how does it apply to AI, and start kind of addressing these questions and and even getting to like even what space-time is and that relationship and change and like causation and those things all really come into how you think about AI, how AI works. And there's just really these fundamental underpinnings. And since the first episode on the on this of this podcast, we really talked about the first principles. So if we're really gonna go to really first principles, we have to go to metaphysics, but that will help us hopefully figure out how move forward with AI and best optimize AI. But I often think that we we shortcut too much and we kind of skip steps. So we're gonna really go back to the fundamentals here. And I'm very excited about this series, and um don't even show how many episodes we'll do around it yet.

SPEAKER_04: 2:24

Yeah, uh, so for this first episode, we want to walk through thinking and reasoning. And so to get us all started, to get everyone thinking about this, we asked a couple of our friends and colleagues the question: what is thinking? And let's go through some of the answers they gave us.

SPEAKER_03: 2:42

Analyzing information to make decisions.

SPEAKER_00: 2:47

A non-deterministic chemical process dictating some of an organism's action. It's possible thinking.

SPEAKER_04: 3:10

And finally, our most poetic answer was the inner working of the mind and heart as we process life in light of truth. And so I think this paints a really wide and large picture about what thinking and reasoning is, but it sounds like people have some agreement that it requires some amount of input, some amount of deliberate deliberation, and then some amount of output. Uh and it seems like this deliberation is a pretty consistent feature in how people are describing what thinking is. So let's talk about how we see thinking being talked about in the AI space. And specifically, you may have heard of LLM's reasoning. So, what does it mean for an LLM to reason? Uh I'm sure that if all of us have used Chat GPT or Gemini or Sonnet, you know, you'll ask it a question and it won't give you an answer immediately. You'll get like a little loading bar and it'll say, you know, the model is thinking. The model is reasoning right now. What does that mean? And what is that very practically? We have not seen these models gain a new magical ability to do thinking. What they are actually doing nowadays is what we call mixture of experts, or MOE, you might have seen it. In this paradigm, we have a handful of LMs debate a topic together, and by debate I mean literally one LM will give an output, one will take that as an input, and they'll talk to each other, vote on a best answer that they agree with, and then that answer is submitted to you, and the model has now thought. What do we think about this? Do we think that this is tantamount to thinking?

SPEAKER_03: 4:53

It's exactly why we're gonna have this series and um really de dig into the different definitions of what that is, and that's where going back to you know Aristotle, Plato, and really what some of this initial metaphysics was, I think is really critical. Of I think we've gotten really like conflating what the term thinking is, of like that's just a basic algorithm. What a lot of people have started thinking that these systems are you know thinking or reasoning is basically can pass a Turing test. Well the Turing test, he even called it the imitation game. So it's imitating uh the the whole general high level is if a person can will is having a conversation with like a computer and doesn't know if it's a if it's a human or a computer, it's gets mimicry. So you you you as the person participating in the test don't know if it if that output is from a human. You're kind of kind of conversing. But that can be done with decision trees. There's many ways you can just have a natural language conversation, but the mimicking human behavior, as we've established LLMs, it's it's the probabil probabilistically you know predicting what the next word should be like to sound human, that's doesn't mean it's thinking. That's maybe it is analyzing information and making a prediction, but like these reasoning systems of voting, like just voting on like the highest weight that we talked about in the previous series, you know, utility theory of just optimizing for like the highest weight and things, that does not seem like what thinking is, which maybe thinking is a continuum. And that's what we're gonna really dive into some of the original um literature and of philosophy and things and kind of see what what that is, because we're in the AI space. There's either the people that are super pro LLM saying, oh, they're thinking, they're what is thinking, like that is what thinking is, and we can we can say thinking, and then there's some of us who are saying, well, that's not thinking, or even that some of our coworkers are it's a uniquely human attribute or or things like that. So uh I I personally, and which all probably know my bias on this, do not think any of these are thinking, but honestly, going back into what a philosophers think of maybe it's a continuum, but like really defining what that is, but in either case, dystriangulating between like the Turing test is should not be the definition of what is thinking or not. But these are all like the deep questions that we we kind of like gloss over often. So um very excited to be digging into this series.

SPEAKER_04: 6:58

And I think we can all feel that there's this sense that these AI companies have a lot of benefit to gain from rebranding what is essentially sampling a couple answers, picking a nice one, and then presenting that one to you as reasoning and thinking. This fits very nicely with our sci-fi perception of what thinking is and what reasoning is, and so all the more reason to go ahead and use that naming scheme. Although, you know, if we look at the answers we got from our from our colleagues and how we personally define thinking, none of us are describing thinking as, well, I have a bunch of flashcards in front of me and I pick the best one, right? It's a deliberative, intuitive, perceptive process with with knowledge from the past to come to a clearer answer. Uh so you know, we would say that this would be illusory, right? We we would even call this an illusion of thinking. So let's go ahead and even if we accept the premise that these AI systems are thinking, let's go ahead and test the ability to think. Uh there's a very famous recent paper from Apple called The Illusion of Thinking. Uh, this one showed that AI models are very, very bad at handling problems that even just require a little bit of thinking, right? We have one example from most people's early coding classes, which is the Towers of Hanoi problem, which is a game that kids play, where you need to move blocks from one side to another following some rules. It does great for very simple cases, which have probably been seen on the internet, but as you ask it to do more and more and more, which would require deliberative thinking and following an algorithm, it seems to fall apart. This would not mesh with a model that we'd expect to be thinking, since even a child can understand the algorithm and then redo it.

SPEAKER_03: 8:38

And how the recursive programming is even set up, like you do it with, you know, five to five sets of disks or whatever, you can expand that. It's just a recursive programming, it'll take longer to run, but it's it can do it deterministically and figure it out, versus that doesn't make any sense that the system that is reasoning can do about five or six and then fall off a cliff. So it's very interesting, and then how they even set up that experiment, I think we've talked about it before in here is the multi-step, and like they're using other deterministic systems essentially to figure out what those different steps are. But it is very interesting of the premise of like if if a system is really thinking, it was also given the full prompts. Here's even the algorithm, here's the pseudocode, just implement this. So if it was this thinking and and outside of just route pattern matching, you would how the experiment was set up, you would think that the systems would be able to actually figure out what the answer is. And what that paper was showing was even with these reasoning systems with the the rankers and things that Sid was mentioning, the performance has not improved very much.

unknown: 9:35

Right.

SPEAKER_04: 9:35

I mean, we still see models today like failing the old test of like how many R's are in the word strawberry. It's only when you sample a ton of answers where some of them are still wrong and some of them are right that you're able to land on correct answers, um, showing that you know, even the fundamental models that are these experts are still rather flawed. Uh we can also look back to our episode that we just did with Adi, where we talked about therapy settings, where you find that a thinking model you would expect to understand context. This person is speaking to me asking for therapeutic advice on a chat client. Maybe I shouldn't give them direct answers. Maybe they don't want to solve their problem today. They want to have a conversation and work through their feelings. That's done with these LMs. No, they don't know context, they don't respond to that, and they don't think about how to respond to the context they exist in. They just give you, wow, that's a great question. You're so insightful. Let me tell you how to fix your problem.

SPEAKER_03: 10:29

I think where the test where it really comes into effect, I think, is when we start talking about agentic. Specifically, like the benchmarks that we're seeing of like, if you're really thinking and reasoning, and this is where we really want to, we want to not but dive into exactly how, because we really need to study this of what's what should our definition be of thinking. But what's really interesting with agentic is the fact that it's the multi-step, like we've discussed. But the performance is even with the Tower of Hanoi, it's a basic problem, recursive, like there's that's one area. But the agentic gets even more interesting because if you're actually thinking or reasoning through something, you should be able to take one step, figure out the best outcome, think about the next one. But what we're seeing is the fact that these systems aren't doing that through the route, pattern matching, the types of like branches you might have, it's very hard to be pattern matching into that. So these even extended reasoning, quote unquote systems aren't doing that, and it kind of shows that versus like a human would know solve the first problem, then you tackle the next problem, and that's why the performances seem seeming to be very different. We're starting to attach that. So, like something around like the the stepwise sequence of understanding, like I think there's something there to unpack for reasoning. And it's I think reasoning and thinking definitely much more than like the analyzing information for an outcome, which we we've heard as something, and that's where if you're going based on the Turin test, which is imitation, or you're going back, like I think that it's very interesting. I think as a as a culture, anyway, in the United States, we we're a little loose sometimes on how we define things or or or how we're using words. So I think thinking has gotten diluted a little bit.

SPEAKER_04: 11:58

Absolutely, and that and you you know this extends then to mathematical reasoning. We find that LMs are still not very good at very complex math. Yes, we've seen them try and do some proofs, and you know, there's been some debunking around capabilities there, but in these agentic use cases where we have a bunch of mathematical models and a bunch of formulas and they all need to work together, these models just cannot do it. They can handle maybe one problem at a time, but once you start having to have these complex systems with intermoving pieces, uh we find that they are generally not capable of doing these tasks, which is again reflective of models which are not reasoning or not thinking.

SPEAKER_03: 12:33

This is where like the there's a lot of components that we'll in our next episode really unpack what is metaphysics, what are the components about, and kind of outline the plan we'll go to together. And we'll definitely how we'll summarize this. So we're trying to like pose the problems here, if you will, in this episode. We'll then like how lay out kind of like how we're gonna examine them in depth of the different components, and then we'll circle back and actually try at the end of this series to really well here's our opinion of what thinking and reasoning is, maybe this should be used for AI research going forward, and also kind of answer the question is should we even be worrying about thinking or research or sorry, thinking or reasoning for AI? My currently uh current opinion, which may change over the course of the series, is I don't think that matters. I think we're all bent around the axle of like this general intelligence and things like that, where it's just like, well, if we're trying to make human lives better, more productive, and all the things that like why people are using LLMs or like why are you wanting to use agentic coding and all these things, we want to be more productive, we want to increase worker productivity, all these kinds of things. I don't think that matters. So, should we just completely take away this burden of the word thinking and just call it it for what it is and go from there? But I I think some of the initial questions we'll probably be talking about is like really this thinking is obviously, but I think we should leave that to the end after we've examined the components. But as Sid mentioned in the definition of uh metaphysics, it's really just like the what is reality, existence, objects, time, like it's it's a little bit more fundamental than that, and like that's where space-time I think we're gonna have a really exciting episode, bring back one of our previous guests to talk about space-time as an example. Space-time that's Einstein's theory of relativity as an example, of time is actually even relative, and a big component of you know anyone's life on earth is like time is always just like this continuous thing, but time is actually can be bent and things like that if you really get into the physics of it. So it's even that helps to understand with ties into what's reality, which ties into what's thinking. So I think really taking that step back and how these pieces fit together, and then it really goes into like causal relationships and modeling, and there's a lot of exciting areas there. So AI is such an exciting research area that it just seems we're kind of stunted our growth a little bit by picking a horse too early in the race.

SPEAKER_04: 14:37

Absolutely. And we're so excited to have you guys join us on this, you know, like a little bit weird, but I think it's gonna be a very relevant journey. I think we're gonna cover a lot of ground about you know the nature of causation and the universe, and with these strong definitions going forward, we'll have better wordings and better definitions that we can use to then understand AI more clearly.

SPEAKER_03: 14:57

Uh and I think a key part of this one, I'd love for audience participation as well, like we're very much trying in this episode, like, here's some of the problems we're gonna be solving over this, but want to be very much boiling the ocean and try not, we all have biases and preconceived notions, but try not, like, really take a step back. Of you obviously know if you listen to this podcast, our opinions on these matters, but maybe thinking isn't near as complex as we're thinking it is. Maybe it is thinking imitation. So, like, we want to really take that, like, let's really go back to first principles and build up from there. And if there's like, hey, make sure you guys incorporate this book or whatever, very interested in that kind of thought as well. And we'll be also looking for some really great guests around this topic and maybe even people that aren't in AI at all, but have maybe some expertise in some of these areas.

SPEAKER_02: 15:45

For sure. Well, Sid and Andrew, we can't wait for this series to start. Um, I'm really excited for all the topics that we're gonna introduce and really test ourselves as well as our audience. Um, for listeners, for if you have questions about this episode or any of the episodes you've been listening to so far, please contact us at theaifundamentalists at monetar.ai. Until next time.