Panel Discussion on the relationship between engineering and science in CBMM and the field
Date Posted:
September 30, 2020
Date Recorded:
September 29, 2020
CBMM Speaker(s):
James DiCarlo ,
Tomaso Poggio ,
Joshua Tenenbaum All Captioned Videos CBMM Special Seminars
Description:
CBMM hosted a panel discussion with Profs. Jim DiCarlo, Tomaso A Poggio, and Joshua Tenenbaum, who discussed and debate the relationship between engineering and science in CBMM and the field: We all believe that if we want to understand how our brain computes intelligence, we need a synergistic combination of the science of brains and the engineering of machines. We all agree that science and engineering are both equally important and should be equally deep and rigorous. Beyond these shared beliefs — which are the soul of CBMM — there are of course many open questions where each one of us may hold different opinions that would be fun to discuss. Is studying brains a top priority for AI? Do engineers need neuroscience? Current models for visual object categorization and synthetic text generation are thriving without new input from neuroscience, for example. What aspects of neuroscience are likely to improve AI? We have had difficulty developing neural network models of symbolic intelligence, intuitive physics, and intuitive psychology, for example. Are prospects better on the science side (real neurons and networks in experiments and models) or engineering (abstract formulations)? Will theoretical understanding of deep learning translate to a theoretical understanding of human intelligence?
GABRIEL: The theme today is the intersection between the science of brain function and the engineering of machines. How can we learn from brains to build better machines? How can we capitalize on the exciting progress in machine learning to better understand brain function?
So I think the order is going to be Tommy and then Josh and then Jim, with an initial introduction of about five minutes each. And then we'll turn it over for discussion between the three of them, and also questions and comments from the audience. Tommy.
TOMASO POGGIO: Thanks, Gabriel. So this idea came up because CBMM was started as a center focused on the science and the engineering of intelligence. That's something we came up-- the original founder, especially Josh and me. And now, it's seven years or so that have passed. So it's kind of interesting, I think, to have a discussion about where we stand.
And just to set the rules of the game, we're not trying to define science or engineering. What we mean with science and engineering-- this is just a definition of words, so it's not very useful to try to change that. The definition we adopted is, science is natural science, so defined by the object of study. So the object of study of science apropos intelligence is human brain, human intelligence, maybe animal intelligence.
And the objects of study of engineering are computing machines. So these are just definitions, correspond to the standard definition according to which, for instance, chemistry, and physics, and neuroscience are in the School of Science, and computer science is in the School of Engineering. It's just a definition of the object of study, of interest. It does not have anything to do with rigor or depth.
We all agree that science and engineering are both equally important. They must be rigorous. They must be deep.
Depending on your taste, or whether you are on the party of Aristotlists, on the party of Plato, you can, according to these definitions, put mathematics in the School of Science or the School of Engineering. So I don't buy it. But both science and engineering need mathematics, obviously, and the rigor of modern science.
So that's something we should all agree. The other thing that is important for CBMM, and I hope everybody at CBMM agrees, is that we believe that if we want to understand human intelligence ultimately, which means how our brain computes intelligence, then we need to combine the science of the brain and the engineering of machines. We need everything to go forward.
And of course, we need to do experiments on the brain, human brain, human behavior. So the idea of this meeting, of this discussion, was to go beyond this and address related questions, which are much less clear than just the definition I gave you, things like whether we need to understand human intelligence in order to build intelligent machines. Or maybe vice versa-- maybe we need to build first intelligent machines in order to understand human intelligence.
We don't know. We can discuss about it. Back when we started CBMM, Josh and others in 2012, we could, at the time, claim the neuroscience of visual cortex had led to models of visual recognition, like [? Konnectro, ?] HMAX, and so on, that were state of the art for computer vision. They were new inspiration for computer vision, very different from the computer vision algorithms.
But in the meantime, things have changed. And the pendulum has, perhaps, swing back. Now, it seems that models that come from engineering are actually good models of the brain, as it turned out.
And so is that going to continue this way, with neuroscience just testing engineering models? Or we can expect another phase in which neuroscience will suggest to engineer a rather different type of model from the ones that are in vogue today? So there are many such questions.
I want just to stop here, not to take too much time. Hope people bring many of these interesting questions up. In the meantime, I'll pass the microphone, the screen, to Josh.
JOSHUA TENENBAUM: OK, Thanks. So yeah, I'll also try to speak very quickly, just to give other people-- mostly what I like to hear is other people's perspectives here. Tommy talked about the pendulum swinging back and forth between, I guess, two poles, both of which are ones that I think we're all in CBMM very interested in, and committed to, and working on.
One is the idea that the science of intelligence, how it arises in the mind and brain, can inform engineering and can guide engineering of more human-like kinds of AI systems. And the other is that the engineering tools that are being developed can inform and guide our science, in the spirit that Tommy has long worked with and worked out with David Marr and that has inspired many of us. We think in CBMM about reverse-- so science as this reverse engineering.
We're trying to come to understandings of how the brain and the mind work in the same terms that engineers would use to build that system. So that tradition of bidirectional interchange between those fields is really, I think, what we're all about. But it does seem to be that sometimes, the most promising directions go one direction or the other direction, as Tommy said.
The perspective I guess I would just add, which comes from my experience in my lab, where we primarily work on the mind level, on the cognitive science level-- although increasingly, we are really interested in, and CBMM has really opened this up for me in a really significant way, making contact with the brain level-- is that actually, it seems that the pendulum might be swinging in different directions at those different levels in a way that is really interesting, and positive, and synergistic.
So as Tommy referred to, there's been great progress-- I think Jim will talk more about this; his lab has been at the forefront of leading this-- in applying recent developments from, in particular, deep convolutional neural nets to understand the venture stream. Nancy's lab has been using these in all sorts of really interesting ways as well, and many others here.
So that's a place where it seems like the engineering is, in some ways, kind of guiding the science now in terms of the studying the neural basis of vision. But on the cognitive level, I find actually in my lab these days, most of what we seem to do is kind of cognitively guided AI. Honestly, I wish there were more people in our lab who were really working on the cognitive science, the study of human behavior, and what it tells us about the mind.
Where the action seems to be right now is using cognitive science to guide AI to make more human-like forms of AI. I have many people coming to me, coming to our lab, whether they've been working in neural networks, or robotics, or computer vision, or natural language processing, and asking us, well, what have you learned or what has your field learned about how computations underlie cognition that we can use to make neural networks or make AI smarter? Because we know we've achieved a lot. And yet we still mostly have systems that solve pattern recognition problems and use mechanisms of function approximation.
And there's something missing, maybe something quite big missing. And how can computational cognitive science guide that? So I see us being-- in different projects in our group, we are both using, I'd say let's specifically focused on neurally based or neurally inspired engineering tools like neural networks where both on the cognitive level, using non-neural network ideas like ideas from probabilistic inference and symbolic representations and so on, to try to guide and improve, together with neural networks, the AI side.
But then I'm also involved in projects looking at vision, but also language in the brain, where we're using neural networks, today's best engineering tools, to try to understand how the brain works. And I think that's really interesting, that it has the promise-- in a sense, the engineering, reverse engineering process might be a little bit out of phase at the mind level and at the brain level, where it might be that if in neuroscience, science has the most to gain from engineering, but in cognitive science, the science has a lot to contribute to engineering, then by swinging back and forth at those two levels, that might be a way that we can actually bridge the cognitive and neural levels. Because it still is the great outstanding question for us in our field is, how does the mind work in the brain? Or how does the brain give rise to the mind?
And I think again, most if not all of us would agree that if we only work at the level of models framed in terms of words and intuitive concepts, we're not going to be able to do that. Because the language and the intuitions and the classical vocabularies of cognitive science and neuroscience are too far apart. But if we can translate our insights about cognition into engineering terms, and we can translate engineering terms into real biological neuroscience insights, then that can be a way to use the mathematics and the formal tools of engineering to make a bridge between the higher and lower levels of explanation or the more mind and brain levels of explanation. I don't mean to imply any particular vertical ordering, but if you guys know what I mean-- to link up those levels of analysis. And I think it may be our best shot for doing that.
I'll just point to one concrete research direction, which maybe people might want to talk about later, which is work that I've been very involved with in a very indirect or secondary capacity. The main work is done an Ev Fedorenko's lab and by Martin Schrimpf, who's one of Jim's students. But he's working on models of language in the brain together with Ev, and Nancy Kanwisher, and me, and a number of others.
And basically what Martin has done, together with Ev, is to show that to take a wide range of state-of-the-art artificial neural networks for language, especially these transformer models like BERT and GPT, if people are familiar with this, that these are the artificial neural networks that have been most responsible for transformative changes in natural language processing from the last year or two. And test these together with earlier generations of neural networks as accounts of language processing from the word up to the sentence level in experiments that Ev and colleagues have done for a while, and really shown-- great strides, basically, have come recently.
I won't go into the details of this work, although if you're interested, you could read Martin's paper on BioArkiv. But have really shown that especially these transformer-based models, especially, actually, the GPT-2 model, which is the one from Open AI, which has been, I think, getting the most attention, especially in sort of natural language generation, these models of predictive processing-- basically models that are trained to predict what word is going to come next given the previous context-- are basically by far the best computational models of language processing as measured in either fMRI or ECOG data. And I think that's very exciting.
I think that it's by no means a complete account of how language works in the brain. But kind of together with work that people like Roger Levy, especially, have done, and earlier work from Ted Gibson here, and many others, not just at MIT, studying behaviorally, showing that basically sentence processing can be, and online measures of sentence processing like in reading time, can be really well predicted by predictive language models, just like engram models or probabilistic grammar models. And now we have the best predictive language models that have ever been built with these neural transformer-based models. The picture that emerges when you connect behavior, computation, and brain there is really exciting. It suggests a unifying paradigm coming into play, like an account of language in the brain that might start to rival how we understand vision in the brain.
At the same time, that picture also leaves out a huge amount, which is basically, we know that those models, in some sense, they predict language but they don't really understand language. They don't explain how language grounds in perception. They can't turn into an embodied agent. And they say a lot of-- I mean, they do amazing things and yet it's very clear that they don't really have common sense. So they make all sorts of errors. I'm not going to go into those here.
So in other work that we do, we talk to people in natural language processing. And we're talking about how can we take insights from other aspects of cognition, commonsense reasoning, language grounded in perception, where we have non-neural network models, and take those and use those to make better engineering language models? And so we see these as just different parts of different steps in this process of the pendulum swinging back and forth, trying to learn from what's unfolded over now a couple of decades in the study of the visual system, and seeing, can we carry that same process of engineering and reverse engineering, maybe even now faster because we've learned a little bit how to do it, but into other cognitive domains such as language? So I'll just leave it at that, as one example of where I see these things going, and turn it over to Jim.
JIM DICARLO: Thanks Josh and Tommy. I just wanted to say upfront, I don't think Tommy and Josh and I disagree on things broadly. Probably more in common than in dispute, but I think for this discussion, it'd be fun to find the points of disagreement. But I mostly want to hear from other people on how that they're thinking. This is just a broader discussion.
I just want to say about Josh's comments-- I don't deserve any credit for the cool things he mentioned that Martin Schrimpf is doing in my lab. I'm just lucky enough to have him in my lab. But thank you, Josh, for mentioning Martin's work.
So now let me just give you guys three minutes on my overview of kind of how I think about science and engineering. Many of you heard this before, but for those who haven't, just to lay it out there. Let's take our goal as a community-- maybe not everyone in this room or this call wants to do this. The goal is to understand the mechanisms of natural intelligence.
That's not everybody's goal in the world. Let's just state, if we just take that as a goal. We can discuss whether that's a good goal, but I'm going to just take that as a given goal.
What does that mean, "mechanisms of intelligence?" That means there's questions about the mind that are explained in the biophysics of neurons and their connections. That's what it would mean, at least to me, to have a mechanistic model of intelligence, intelligence being an aspect of the mind.
So if you accept all that so far, how are we going to actually do that if we get real about it? Now we have assumptions-- do we think that the accurate assumptions are going to be more complex than the two main types of tools we've used in science for building our hypotheses? My assumption is yes-- in other words, it is going to be more complex than word models that we typically use. Word models might be things like "the brain is a prediction engine or things of that sort."
It's going to be more complex then elegant mathematical equations, simple mathematical equations. So we're going to have complex models to explain natural intelligence. I hope most of you agree with that, but let's assume. I'm just going to take that as an assumption-- that we have complex models as hypotheses on how you go from neurons to intelligence.
So that's usually going to have to be described in some language that we can transfer among humans. And right now, the best language is language of code, or maybe hardware, or some combination thereof. So we're going to have a hypothesis space that's complicated.
And how we're going to proceed as a species to build hypotheses in that space where you need a process? We need to build hypotheses, and then we have to test them against data. So far, hopefully, I can find the weaknesses in my assumptions, but so far, I don't know where they are.
So then when you get to that point, the only question you have is, who's going to build these things? And what data are we going to test them against? Those become the operational questions.
And I don't care if we call those people "scientists" that build them or "engineers" or "companies," but somebody has to build them. Because they're complicated and we need them. Otherwise, we don't actually have anything to test against data.
So I'm agnostic about what we call that, whether that's "engineering," "science," I don't care. But we have to build things. And that must be done. And the hypotheses must be built and they must be tested against data. And that's really just science, to me, in understanding natural intelligence at the mechanistic level.
And so that's all I wanted to say big picture. And I just want to add for Tommy's question about do we need to understand human intelligence to build intelligent machines? To me, that's sort of a non-question. How could we ever say we understood human intelligence if we couldn't build it?
So it always is going to have to be that we're going to have to build intelligent systems before we really understand natural intelligence. I'll just say that's a natural corollary of what I just described there. And I'm going to end it at that and leave it as points of discussion.
If people want to attack my logic, I would love to respond to that. But I want to let this be a discussion. So I think that's just an opening position for me. So I'm going to end there. Thank you, guys.
GABRIEL: OK so thank you very much, everyone. I want to emphasize that we want to hear from all of you. So don't be shy. I don't see any hands here.
So maybe one question-- I'll start, but I really want to see-- now as I was about to start, I see Nick [? Breu ?] raised his hand. So Nick, go ahead.
AUDIENCE: Yeah, great. Thank you. So it's true that the three positions are very similar. So I want to sort of make a hypothesis and get your reactions to it. Maybe it's a couple of hypotheses.
So first hypothesis is that it is true that in intelligence, science and engineering, they've talked to each other but they haven't done a great job of really sort of leveraging each other's strengths. So that's hypothesis one, which I guess we could argue about. But the more interesting hypothesis to me is that I think the reason that the science and engineering have not really engaged each other is because they think about the world in fundamentally different ways that are important, that engineering, to a large extent, is the act of putting things together.
There was a thing from Jerry in the chat about an engineer creates that which never was. And to create something, you basically have to compose things together. So you have to figure out how things go together in a new, and stable, and useful ways. Science tends to be a bit more reductionist in just trying to figure out what the thing is, and not thinking so much about the interfaces.
Josh, you made a really nice point about sort of figuring out the connection between the different layers of representation, but I don't know that the science of natural intelligence has really focused hard at like what are the compositional interfaces that let us take pieces of the architecture and put them together? I can't take a piece of GPT-2 and plug it into a new architecture and expect it to work in a predictable way without having to retrain. So in that sense, it's not composable.
So I guess my hypothesis is that level of compositionality has been missing from our knowledge or understanding of how the brain actually works. And how do we get it? What are the experiments that we can do to figure out what the interfaces are that give us more compositionality of the pieces of the brain?
JOSHUA TENENBAUM: Well, I guess I can try to respond to some of those things. I mean, I agree with you in terms of on the science side, but I guess maybe that's partly the kind of work that I've been trying to do and we're especially excited to do. So some of this is work that we have done using what people these days are calling like a "neurosymbolic interface," so together with, initially, with Jiajun Wu who was a student in our group, and part of the CBMM and is now a faculty member at Stanford, but also with Jiayuan Mao, who's a relatively new PhD student, and a number of others, Chuang Gan and some people from the MIT IBM Lab. Antonio Toralba has been part of these things.
We've built various kinds of neurosymbolic models that basically try to do some very limited kinds of grounded language. And they combine basically something much simpler than GPT, but like a stacked LSTM sort of language parser, that passes a question or a statement into a symbolic semantic parse, and then basically a neural imaging interpreter that segments and derenders an image into a symbolic scene graph, and then a reasoning engine, which applies the symbolic sentence parse onto the symbolic scene parse to answer questions or talk about scenes. And that was just one example. We've done static and dynamic stimuli.
These are examples of things where we're trying to build systems that put the pieces together guided by some ideas about cognitive architecture, which people have studied. People like Ray Jackendoff have studied these ideas for many years, understanding the interface between language and perception, but not actually built models of these things that can run on images. And so we're trying to basically be inspired by some intuitions from cognitive science to build these systems.
But I agree. I think that a lot of, especially today's neural networks, they're really good when they're on data that looks like what they were trained on. But to have composability and generalizability, strong generalization outside of a training set, is very tough. Those systems that I was just talking about don't do that. But here's where I think some of the probabilistic programming-based systems, that Vikash Mansinghka's group has been working on, and we're working with him on using these in various kind of common sense reasoning and building machine common-sense projects.
I think there, you have both composability and strong generalization. There are other engineering challenges and scientific challenges that those things face, but we think by combining these, basically by exploring in an engineering sense, guided by scientific questions of exactly the kinds that you're asking, the neurosymbolic and the probabilistic programming-- and those can even come together into a probabilistic neurosymbolic toolkit-- I see this as a way forward that's going to be valuable for both the science and engineering. But it's big challenges. And you're exactly right to point to the challenges that really, neither the scientists nor the engineers, I think, have addressed that well.
TOMASO POGGIO: Long ago David Marr made famous this metaphor of levels of understanding, understanding a complex system like a computer and the brain at different levels. You can understand the computer-- can say, "I understand the computer because I know how to use PowerPoint," without knowing anything about logical units in the computers, or nothing at all about how transistors work. And you can understand how transistors work and know nothing about the software.
And these are all valuable ways of understanding a system. With David, we stressed the fact that these levels are kind of separate, modular. That's related to Nick's point.
But if you go back-- and this was really for political/scientific reasons at the time. It was because we thought that neuroscientists did not understand the computational algorithmic level and it was important to stress the existence of that. But if you look at a study of this, of what David Marr wrote, the history of that is a paper we wrote together about levels. And that, in turn, came from an earlier paper I wrote about the fly, where levels of understanding, instead of the computational level, there was a behavioral level. And David correctly replaced that with a level of essentially characterizing a problem and its possible solutions to the computational level.
But the other thing that got lost in translation was the fact that the work in the fly, we stressed with Werner Reichardt the fact that in the brain, unlike a computer, these levels of understanding are quite tied together. You cannot really understand how circuits work without knowing details of transmitters, and synapses, and so on. And I think in modern times, people too much at heart the computational level.
It's not enough for having a theory of the brain. It's important to have a theory at different levels and to support models that can be tested and falsified at different levels, not only at some higher level. Whereas if you have a model using neural network, you shouldn't really ask how can you run, implement, an RLU?
Does that exist in the brain? How could it possibly be? Otherwise, this model is not biologically plausible not even falsifiable. And so with other things, like of course, gradient descent, and back propagation, and so on.
So I think if you are examining an engineering system, because it was made up by engineers and teams of engineers, you would find much more separation between the levels of understanding than you would find in a brain. Evolution did not have the luxury of doing that. And we have to take this into account when we develop biologically plausible models of the brain.
JIM DICARLO: I was just going to agree completely Tommy's last point, that engineers had to-- we had to find levels of abstraction and compositionality just to manage the complexity. But there is a hope that evolution had to have also work with choices of gene replications that would then be composable. So it just might not be in the language we're used to talking about.
So there's still some hope. It's not as if it's completely non-compositional, but I think it wouldn't be in Marr levels, as Tommy said. It's going to be in some other form of how do I replicate a few genes, and build another level of cortical loops, another cortexes, cortexes, cortex.
These are the natural things we see that might be that composed of pieces that evolution is working with. And it's the engineer, in this sense, engineering by trial and error, so to speak. And that's our only hope there if we really want to see the brain version, at least to me. Anyway, thank you.
AUDIENCE: I just wish we knew the objective function that was being optimized there. That would be wonderful to learn.
JIM DICARLO: Survive and reproduce.
AUDIENCE: Survival.
AUDIENCE: I think it's actually essential for understanding to be able to break problems into pieces, break systems up into parts, and understand how those parts fit together. And so the scientific understanding is going to be essentially what it looks like an engineering understanding, too. That's always been true for very complex systems.
The real difference between science and engineering has nothing to do with the tools used. Every engineer investigates the materials he works with. Every scientist builds equipment to do his experiments.
The honest answer is that it's really a question about what the goals are. But the mechanisms of thinking that are required for doing both are the same. And they're basically decomposition of a system into parts so you understand how they're put together, to have the behavior that's observed.
GABRIEL: Thank you. We're going to move on the next question by Mahshid.
AUDIENCE: Hi. I hope you can hear me well. Can you hear me?
GABRIEL: Yes.
AUDIENCE: OK, great. So thank you all for this discussion panel. I just wanted to your opinion on a top down approach to achieving intelligence, approaches like knowledge representation and reasoning. Would you comment more on that part instead of a bottom up approach?
JOSHUA TENENBAUM: I guess I could comment on that, at least of the official panelists. I'm not exactly sure but you have in mind, but I guess very broadly, that is a lot of what we do in our group. So we come from the cognitive science side, and a lot of what we do is we try to develop formal frameworks for knowledge representation and reasoning.
That might be probabilistic inference. It might be some kind of symbolic or probabilistic programs, which combine many of the powerful features of abstraction that you get from symbolic language and flexibility, ability to deal with uncertainty that you get from probability. And there's always uncertainty when you're learning from sparse, data or few examples, or in the ambiguity that confronts natural science or natural intelligence in the world.
So I think of that broadly as a knowledge representation and reasoning thesis that has been very successful in modeling many aspects of cognition at the computational level. I think there's issues of modularity that really come up in human knowledge. It's part of how cognitive scientists have understood how humans achieve flexible reasoning and broad generalization. And that's the toolkit by which we do that.
But then there are really big questions of how does that work in the brain. What are the neural mechanisms, either at the level of single neurons, or networks of neurons, or within single neurons? They're great mysteries about how these formalisms or computational tools for knowledge representation reasoning that have been successful at the cognitive level and explain and predict a lot of behavior, and really explain it in terms of underlying general principles we can make sense of.
A lot of what we're looking for is really there in the cognitive level, but how that works in the brain is a great mystery. And part of why I'm really interested in studying language, and language in the brain, is because it's a way into that. I also think there are aspects of commonsense scene understanding-- this is something that Jim and I and others, like especially with Dan Yamins and people in Nancy's lab, too, are interested in.
I think we can also get at some basic aspects of knowledge representation just in trying to take our models of vision in the brain, and go beyond what Jim has studied with so-called "core" object recognition, where you're just recognizing or categorizing a single object, to trying to understand a whole scene, the relations like these objects are on the table. These ones are balanced or not balanced. This object is contained inside that one.
So understanding multi objects scenes and their relations-- that's also a very basic kind of, I think, symbolic knowledge representation. And whether it's through trying to understand the neural basis of sentence meaning and understanding, or the neural basis of compositional scene structure, we think those are ways that we can start to understand these things. The brain is broadly top down, in that we have models that work to a certain degree at the cognitive level and fit behavior.
And then we're, in a reductionist way, trying to say now, how might these pieces be implemented in neurons? Maybe we'll posit some kind of things that look sort of like neural networks but aren't very much like anything we currently understand in the brain. And then that will pose a further challenge. Maybe there's new things we have to discover in biology.
Maybe we need to adjust those engineering mechanisms. For example, the transformer models which have been so successful in natural language processing, they're really different from how we understand neurons in the brain. Is that a fundamental thing? Is that just a compactness for getting certain things to work in GPU memory? We don't really know, but that's broadly a top-down approach driven in the way that I think you're suggesting.
AUDIENCE: Actually, general understanding will always involve both top down and bottom up understandings, which connect together at some point. And then--
JOSHUA TENENBAUM: Yeah, I absolutely agree.
AUDIENCE: Just a follow up question, if you don't mind. Do you have any favorite one, favorite method that you have used, probably, in knowledge representation using-- like we have semantic networks, we have inference graphs, we have Bayesian networks, we have concept graphs, hyper graphs. Is there any of them--
[INTERPOSING VOICES]
JOSHUA TENENBAUM: People who know me-- well, we've used various things. The graph representations have a lot of value. And they look a little bit more like neurons than some other symbolic approaches for neural networks. But I am a big fan of probabilistic programs and probabilistic programming.
I won't go into it here, but I think this is a toolkit that subsumes and generalizes a lot of other classic knowledge representation ideas, but especially really allows us to combine the power of abstraction that you don't just get and grasp, but like higher order. You need higher order logic or full recursive programming languages, like I'm very inspired by Jerry's work in using LISP and Scheme to try to think about the structure of intelligence and some of the first probabilistic programming languages that we built in our group, Vikash Mansinghka and Noah Goodman, who's now at Stanford built.
They took basically the ideas of knowledge representation of reasoning that Jerry and colleagues had worked out, and embedded that in a framework for probabilistic inference in a language called Church. And that was just the beginning of what's become a really fertile program developing these tools on the engineering side. And we're trying to apply them on the science side as well.
TOMASO POGGIO: Josh, I think you'd agree that that level of model, of that language is like, say, thermodynamics in physics. So it's at best a phenomenological description.
JOSHUA TENENBAUM: Yeah, yeah, or it might be like the level-- it might be like the level of algorithms and data structures in computer science.
TOMASO POGGIO: Maybe, but what I'm saying, you have to make a connection to the brain.
JOSHUA TENENBAUM: Exactly. That's what-- so that's why I'm saying to do that, I'm looking to apply these models in the places where I see right now we can make the best connection. So that includes scene understanding, vision going beyond--
[INTERPOSING VOICES]
TOMASO POGGIO: I also mean developing models, theories, of how neurons could compute those kind of--
JOSHUA TENENBAUM: Absolutely.
TOMASO POGGIO: We all agree.
JOSHUA TENENBAUM: Yeah, but again, I think we all agree, but we also all know that we don't know where to look for those ideas. So I think, again, everybody who hasn't read Jerry's book on the structure and interpretation of computer programs should read that as a classic example of crossing levels on the engineering side, and using and trying to cross between the sort of algorithm data structure and circuit levels. And that's one way to go.
Another way to go is to look at these neurosymbolic models which are like graph neural networks. These are things that are coming from the modern neural network toolkit that is trying to give you a language for implementing these kinds of things. Those are just two of several possibilities.
TOMASO POGGIO: Also you should look at the history of physics. Thermodynamics, we're speaking about quantities like entropy and heat without any idea of what they were. It's only later that mechanical statistics interpret Brownian motion and so on, to give a mechanistic interpretation and much more power in terms of what to do with them.
And electricity, the same story. People used, built engines, generators, batteries, without really understanding what electricity was, having some kind of fluid analogy. It was when Maxwell came around, and then there was a real understanding of electromagnetism. And you know Maxwell reverse engineered electricity to use this term of Jim I don't like, because that's science. The scientist never uses reverse engineering, but anyway--
JOSHUA TENENBAUM: I think--
[INTERPOSING VOICES]
TOMASO POGGIO: --engineering electricity, and then out of it came radio, telegraph, computers, the internet, everything.
JOSHUA TENENBAUM: Yeah, no, I mean, I think here, we'll probably be a lot of agreement. I think the top down route unfolds over time. And we often understand things at a certain more macro level before we understand how those things are implemented. But the macro level gives important guidance for the underlying micro theory.
TOMASO POGGIO: Actually, it's interesting that "thermodynamics" came first.
JOSHUA TENENBAUM: Yeah, exactly. So Chomsky used to make this point in his Philosophy class that I remember going to as a grad student here, that in physics, usually that's how things go. You start with phenomena that everybody can appreciate, like everybody.
Like oh, there's a moon out there and there's some planets, and they seem to be moving. How are they moving? Or I drop this apple and it falls. Why does it drop?
You start with phenomena like that and then you come up with mathematical descriptions that unify things at that level. And then you start looking for reducing things down and searching. Sometimes it takes a few years, sometimes it takes a few centuries, to come up with the micro level mechanisms. And I think the cognitive and neurosciences seem to also follow that, can also follow that trajectory.
TOMASO POGGIO: Agreed. Yes.
GABRIEL: Unless Jim wants to say anything, we're going to move on to a few more questions that we have from the audience. We have Sam next.
AUDIENCE: Actually, the thing that I wanted to ask fits very nicely in this dialogue. I think that many of us would agree in some sense with Tommy, that we should look to the brain for our insight into how to build intelligent machines. But the place where I kind of struggle, and I think a lot of people struggle, is what exactly about the brain matters and what doesn't matter?
I think that if you look at where engineers start to scratch their heads, when they try to read the neuroscience textbooks and derive something useful from that, is that there's just tons of details that seem irrelevant or even possibly detrimental to building an effective machine. And I think that's true not just for the biology, but also for the psychology. Like if you open up a psychology textbook, there's all sorts of biases, or apparent biases that people exhibit.
And I think many engineers, while they would like to be able to emulate the intelligence that of the people do, I think they would be perplexed if you asked them to implement all the weird stuff that people do in addition. And so how do you know what matters? And I think that is not just a philosophical question, because I think that-- I guess I perceive a kind of widget model of neuroscience-inspired AI right now, where somebody reads about something that neurons do, like consolidation, or some particular kind of spike timing-dependent plasticity rule, and then they just sort of slot that into their model and then see if it works.
And sometimes maybe it does, sometimes it doesn't. But that seems like a very inefficient way to make progress, if you just by trial and error try everything that biology throws at you. So I'd be curious to hear what you guys think about that.
JIM DICARLO: I need to chime in here. Sam, I completely agree with that point. I call that "neurobranding." It's all over the place, like you have an idea-- there's always something you can point to in the brain that says, well, the brain does that, so I should do this in my model.
But it's the combination of all the things in the brain measured as behavior and neural activity that is all the constraints that have to be brought to bear. And that's kind of where this Brain-Score platform comes from for us, that it's not one thing. That's just inspiration.
But then there is the point that you also mentioned, which is there's myriad details. So how do you know which things to pay attention to? And brain size doesn't have that answer.
So the only practical answer to give is if you choose a top-down problem-- Josh mentioned vision; that's what we do-- you start with that. Then you build models of that roughly in the style of neurons. And then you look in detail, and say, well, so far is it matching?
And that helps you sort out what details matter and which don't. But it's the top-down approach that lets you sort out what matters and for what reason. But I think it's a mistake to think neuroscience textbooks are going to tell AI how to build something. I'm certainly not in here staying-- I hope people don't take my position as being, let's look to the brain and that will be how we figure out how to build AI. I think of it more of the opposite, of building engineered systems that try to do AI as hypotheses for how the brain works.
AUDIENCE: Jim, can I pick up on that for a second? I think Brain-Score is a really interesting example, because how do you decide what goes into Brain-Score? Why aren't we taking the tertiary structure of ion channels into account when we compute Brain-Score? There's some heuristic.
JIM DICARLO: But the practical answer is you can't build models at that level of complexity and get them to run right now. So somebody's got a comment in the chat right now like, we can't build the Sun or things like that. There's certain things that we just can't build. So that's one answer to that.
But then the rest of it is, we don't know, so it's a political answer, which is a bunch of neuroscientists have measured these things. And since they bothered to spend animals and money on it, well, you should put them in because maybe there's some value there, until we later learn there isn't. But I agree with you. We don't know the right answer, so our bias is put everything in for now. But that's a hard task.
AUDIENCE: That's what I'm asking. If you could, would you? Would you put everything in? Would that make sense?
JIM DICARLO: You'd like-- everything that-- you could just choose the level of your measurement and say I put in things that apply at that level of measurement. You mentioned ion channels. We're not measuring ion channels. Then you don't put it into model.
[INTERPOSING VOICES]
AUDIENCE: But if you were measuring ion channels, would you put them into Brain-Score? That's what I'm asking.
JIM DICARLO: Well, that's the art more than science. This is Josh's comment about you go from big to small. You don't try to bridge all those levels all at once.
And I think that that's too big of an ask. But that's a practical answer. I'm just going from mind phenomena to neural spikes is one level, but it's not the whole answer.
And that's kind of where your question is going. I mean, you just have to break it off a piece at a time. You don't solve all of physics from one day, either.
That was what Tommy and Josh said, too. So you're just making practical choices along the way is how I think of that. But they're guided by the tools you have at the time, not by a truth.
GABRIEL: Next we have Mike West.
AUDIENCE: So Jim formulated our goal as understanding natural intelligence or understanding intelligent behavior, and Josh proposed trying to translate cognitive science descriptions into engineering terms. And people have been talking about how we bridge macro level phenomena with the micro level explanations, and Josh said how we start with the most universally agreed-upon phenomena in whatever field it is. And so there's different ways I could ask this question, but it seems to me that the most universally agreed-upon phenomena in terms of the intelligence of our brain is the distinction between our conscious and our unconscious brain states and conscious or unconscious processing.
And so one question is, do we think consciousness is relevant to understanding intelligence? Do you all think consciousness is relevant to understanding intelligence? And if so, how can we translate that conscious/unconscious distinction into engineering terms? Thanks.
TOMASO POGGIO: Well, consciousness is still a mystery.
AUDIENCE: Right, but it's like the statistical mechanics-level description that we would hope to fill in with micro level of physics, but it seems that we leave that out of all of our theorizing or a lot of our theorizing about our model making.
TOMASO POGGIO: Here is a question to everybody-- I'm curious. You all know the Turing Test. Suppose you are called to say whether a certain thing in the room you cannot see has human-level intelligence or not. You can speak, interact, ask questions, and so on. So to me, this is still the best definition, not very satisfactory, but the best definition of human intelligence.
Now, do you think that if I make a test for consciousness, a Turning test for consciousness, would that be the same as the one for intelligence or not? So the question is, now you can interact with something or somebody and you have to say, is it conscious or not? Is it going to be a different Turing test with different answers from the one for intelligence? I'm curious what people think about.
AUDIENCE: I think it's a different test. I think an intelligence test is kind of an arbitrary test that's relative to some particular task, like interpreting a visual scene, whereas the distinction between whether a brain state is conscious or not seems like an actual objective fact.
TOMASO POGGIO: Yeah. Can we make a poll, Gabriel?
GABRIEL: So for people who are interested in this question, I think Tommy expressed his opinion during the summer course and you can watch the video. And then we have lectures from Christof Koch expressing a completely different opinion on this. So you're welcome to join us and watch those videos as well.
So the question is whether a Turing test for intelligence is the same as a Turing test for consciousness. That's the poll that Tommy wants to--
[INTERPOSING VOICES]
TOMASO POGGIO: --to be explicit, Christof thinks no. I think yes.
AUDIENCE: Wait, so say the question, then, please, that you're answering.
GABRIEL: The question is, if a Turing test for consciousness is the same thing as a Turing test for intelligence. In other words, can you have human intelligence without human consciousness, and human conscious without human intelligence? I think those two things are equivalent--
[INTERPOSING VOICES]
TOMASO POGGIO: I think human intelligence in a broad sense, not specific to some topics.
AUDIENCE: Tommy, mean you're losing this one. It's all no's in the window.
JIM DICARLO: Can I offer a form of this question? The engineer's version is, if you had a system that could pass Tommy's intelligence test, would it be easy to make it pass a consciousness test? I think the answer is clearly yes, but many people think no.
GABRIEL: OK, so here's the quick poll. So you should see the poll on your screen now. Thank you, Chris, for creating this. So you can just submit your quick-- I like these polls. Maybe we should have a few more questions that--
[INTERPOSING VOICES]
AUDIENCE: I actually think that the concept of consciousness is a funny one, and that in fact, it's a-- one can actually decide whether or not something is like me-- that is, I can talk to it and it acts and behaves in such a way that I could interpret as being-- I could feel like I was like that. But I don't have any idea whether or not I could-- consciousness is a subjective impression. And I don't see that there's any way to test for it at all.
GABRIEL: So as you all know, the way we're going to decide funding for the next 10 years in the field is by doing polls. So depending on how people-- no, I'm just kidding here. So it's 81% no, and 19% yes. I don't know what the end is for that. OK, Arturo, go ahead.
AUDIENCE: That's very interesting, and I don't know if my question is going to tie up to that, but hopefully, it does. So there's been a lot of discussion, especially at the beginning-- I wish I would have asked this sooner about science versus engineering. And I'm pretty sure almost all of us here in our training were trained somewhat funnily, either as a scientist or an engineer in undergrad, like we did something, for example biomedical engineering, mechanical engineering, maybe someone in pure psychology, somebody in pure cognitive science and neuroscience. And somehow, along the way, paths diverted and everyone kind of became a bit more interdisciplinary.
So do you guys think that in the future, like it's happening now already that there's a 6-9 Cognition Computation major, that there's going to be a new PhD/major that is something like a science of intelligence field? Or there's going to be something like mechanical engineering, for example. 56 years ago, everyone was studying turbines and electric engineers were studying transformers, like the real transformers on the outside of the power grids.
But now, mechanical engineering has evolved. It's another thing. So now it's controls. Everyone learns circuits.
So how do you guys see the future in terms of education-wise, at the graduate level, even, for the intersection of understanding intelligence and maybe having more interdisciplinary psychologists and computer scientists? What are your thoughts in general? Or you think it's going to be separated? There's never going to be a new department that does kind of--
AUDIENCE: This is John Baras. I will quote the famous provost of MIT from several generations ago, that you can't separate the two. If you do, you make a big mistake. You can guess who that person was, but he was from MIT.
JOSHUA TENENBAUM: I mean, I think all of us in CBMM are committed to there being that kind of interest in my field that you're talking about. And we've been working on it. I guess I don't think that means the other enterprises are going to go away.
There are scientific questions about the brain that, whatever today's engineering toolkit is or tomorrow's one, are not necessarily especially a well-posed answer. And similar to those engineering challenges that do not necessarily want to be informed or have any good reason to be informed by the science of intelligence, but there's that intersection that is where we think the most exciting action is, and where I think many of us think the future of both fields ultimately has come from in the past. That's in the past.
That's where the future came from. In the future, that's also where the future will come from, is people who are trying to do exactly what you're saying, who were trained-- whatever it was, as a psychologist, or a neuroscientist, or an engineer-- but were motivated by questions that are at the intersection, and so tried to do work that met the challenges at the intersection. And that's where many, if not all, of the best ideas in the field have come from.
AUDIENCE: I think this section is inescapable for a very simple reason-- to understand the brain, you have to make measurements. To make proper measurements, you need the engineering. And then you can go and create science. So because of that, which is also what transpired in physics--
JOSHUA TENENBAUM: But--
[INTERPOSING VOICES]
JOSHUA TENENBAUM: --the engineering of intelligence, though.
AUDIENCE: You have to go back and forth, because just postulating a theory without validating is not the scientific approach.
GABRIEL: Sure, yeah, but maybe to be a little controversial, I mean, both neuroscience and cognitive science have always had-- methodologically, they've used engineering tools. But those aren't necessarily the same engineering tools as building amplifiers or something.
TOMASO POGGIO: Of course not.
JOSHUA TENENBAUM: --or doing-- they're not the same engineering tools needed to understand intelligence. And sometimes, the field has been held back. I see this both in neuroscience and in cognitive science, just as Sam said. I think both fields are guilty of this.
And it's a structural guilt, not an individual guilt, that people learn a certain engineering toolset just to get their work done. And then they kind of, maybe mistakenly, overgeneralize and think, that's what intelligence should look like.
AUDIENCE: Yeah, I want to make a comment, then, because I have booked another meeting. But I, if you allow me, I mean, Tommy knows what I think. But basically, what you need to do is to get some qualitative principles out of human intelligence and see to what extent you can come close to incorporate them in whatever you call, "artificial systems" or whatever.
My prediction in the end is that whatever we end up calling provably artificial intelligence, it might not necessarily be anthropomorphic at all. And there are many examples in science that this has happened before. So you learn things like the importance of learning. Fine.
But then you have to go beyond. You learn, and what do you do with it? You learn. You organize concepts. You compare knowledge. We haven't studied. We haven't addressed these questions.
How do you present evidence? How are you going to be able to compare? How you connect this to time? Where is attentional mechanism?
All of these things are qualitative properties of the brain that we understand pieces of them. We have to start putting them into something that we're building. And hopefully, we'll come to something that we call "artificial intelligence." My prediction is that it won't be anything near human intelligence.
And I'll close by saying for those of you who do not know it, go read the famous Theorem of Free Will. And don't laugh. That's a mathematical theorem from [INAUDIBLE]. And he connects measurements to biomechanics and brains. I enjoyed the discussion. And I have to go.
GABRIEL: Thank you.
AUDIENCE: Thank you.
GABRIEL: Thank you very much. Thank you. We have a question from Elias Blake. And I apologize to everyone if I'm mispronouncing your names.
AUDIENCE: No, it's perfect, pronounced my name perfectly. So I have a question about what the goal even is, and what it will tie back, though. So Jim named his goal, which I like, a mechanistic understanding of natural intelligence.
And we have this other goal, which is that perhaps the objective function that gives rise to intelligence is this surviving and reproducing. That's a good goal. Well, I have a question on surviving and reproducing, because I want intelligence that's smart enough not to destroy itself.
And at that rate, humans don't seem all that smart. Humans actually seem like we're running our society, in some ways, into the ground. And so you can ask, well, what would it take for intelligence to be smart enough to solve these collective action problems like climate change and other things?
But it ties back. And so I wonder if seeing these two different goals as perhaps the same goal can be helpful. Because in the same sense that humans have a collective action problem of this join-or-die mentality, neurons and cells have the very same situation of join or die.
And perhaps that's what gives rise to multicellular life is to see that cells can come together and form an intelligent organism, and that organism will survive better than the single-celled organisms would have. And so the same way that life comes out of multicellular cooperation, maybe the same very principle by which humans have to start cooperating, and maybe that's sort of the intelligence that happens everywhere from the most basic to the most complex of organisms. So the real question is, do you guys think that these two different-- unifying those two views is useful?
JIM DICARLO: Well, I would just say that I think of course some mathematics would apply at both levels. But I don't think-- we don't have an existence proof of a species that hasn't destroyed it-- won't destroy itself. We don't know that's true for us. But we have an instance proof that we don't have machines that do things that we do.
So it may not be achievable, but it's a nice goal that you're imagining. But again, we don't know if it's possible. Maybe theory could tell us that, but that's-- again, that's my first answer to this question.
But I want to use your question to just kind of give it to say-- somebody asked about future, brain and cognitive sciences and future of electrical engineering. Brain and cognitive sciences will not survive if it does not adopt the thing that someone is calling 6-9. It will become like alchemy.
That's a strong position to take, but I'm currently the head. I'm going to take that position. So that if we do not adopt that, we will not survive in the long run.
But electrical engineering and computer science will survive regardless of brain and cognitive sciences, so that literally there. Somebody said, oh, EECS. Josh said something like that. It doesn't need-- but science, if we don't put that engineering activity into our sciences, it will become like alchemy.
And that's what I'm afraid of, and that graduate students have to be trained in that. And that was back to the question of training. That may be too strong of an alarm to sound at this stage, but I honestly believe some version of that is what would happen.
AUDIENCE: Jim, I also think that's important. I raised that question. I think that's important in the future for a computer scientist, too, because now, you see in computer vision that everyone just throws a wacky, random thing to a deep net and it's the next-- and the objective is state of the art. It's not really forming a theory of learning or understanding a network architecture, whatever it is. So sure, it's propelled an entire field in other fields, but--
JOSHUA TENENBAUM: Yeah, but I think you should-- go ahead, Jim. Yeah.
JIM DICARLO: That goal is not to understand natural intelligence. So again, if the goal is to understand natural intelligence, then you're going to need these engineering things. So I don't disagree with you.
Of course, you can build stuff. And that kind of amplifies my point, that people will do that forever, and they will have departments around them. But the natural science of intelligence kind of is a more fragile thing.
I think that's where I'm kind of more resonant with Josh and Tommy, is how are we going to go forward in that regime? And that's kind of why we're into that goal. It's not the only goal in the world. It's just one that I thought this group was most oriented around.
JOSHUA TENENBAUM: Yeah, I mean Jim, I think BCS has always, even back before it was called BCS, stood for that kind of view. That's why David Marr was here, and Tommy came here, and people did robotics in earlier generations. And some of the-- even when it was still called the Department of Psychology, there were people using engineering tools, going back to Teuber's vision.
So I think we're all bought into that. But I agree with you that the quest to understand the brain, and the mind, and intelligence-- if it doesn't continue to keep renewing itself with more powerful engineering ideas, tools, and so on, it will run aground. And I think we've seen that in the past.
And I agree with you. I think we will see that. Other kinds of neuroscience, like to try to cure diseases, might not need those tools. They might be much benefited from them, but they might not need them in quite the same way.
But the quest to actually understand intelligence definitely needs them. And I think you see the same thing on the engineering side, because I think you can distinguish EECS from AI. in fact, AI was just carved out from EECS.
I mean, this might be a controversial view, but I think part of the reason why AI has had a series of springs and winters is because-- or where do you get the winters from? It's like at some point, some breakthrough idea comes in, and then you pursue it to its logical conclusion, and you scale it up as far as it can go. But since you made some big, great promise about human intelligence, and you don't have the whole story, then you disappoint expectations. And you temporarily die or wither.
I think CS and EE and EECS can do fine without the brain. But AI needs it just as much as the science of intelligence needs the engineering-- that is, to keep the really long-term vision of artificial intelligence as everybody has always wanted to understand it, for a long time from now I think it's going to need to keep looking to natural intelligence to guide, to set the problems, just to provide ideas about how to approach things. That's where the new ideas have come from, whether it was neural network people inspired by the brain, or Judea Pearl who is inspired by human causal inference. or so many people, I think.
I mean, look at Boole, for example, as in Boolean logic. He introduced many of the new ideas, effectively, into computer science before there even was such a thing. He was inspired by the mind.
That's why his book was called Investigation into the Laws of Thought. And it starts off reading like a cognitive science book. So for a long time, I think, in the past and in the future, that's where things have come from and will continue to come from.
AUDIENCE: And just as a quick last thing, yeah, I agree that you probably don't need the brain science for the engineering stuff. But I do think that there's something about the way of thinking that we're all taking for granted here-- that it's not that all engineers, for example, know how to run a controlled experiment or know what they're looking for when they design something. So I think that way of thinking, it's different from the other field. So yeah, that's maybe going back to--
[INTERPOSING VOICES]
AUDIENCE: I just wanted to add something. Diego Mendoza-Halliday here. Just to respond or to challenge the view of Jim or maybe others that we absolutely need engineering for science, and that we need to build something in order to understand it. Let me just bring a challenge here.
We might be able to do that in neuroscience. We have that the luxury that there is this engineering way to perhaps make something that looks like a brain. But what about the rest of sciences?
What about ecology? What about so many other sciences where that's-- weather science, where it's not even possible to build something? Will you tell all those scientists that they're doomed because they cannot build whatever they study?
I want to challenge that and say, yes, it's great to have the possibility to build something and to validate what we're studying. But I disagree that we scientists absolutely need, by principle, absolutely need engineering to understand whatever we are understanding. What do you want to say to-- I don't know if that's a valid point.
JIM DICARLO: That's a valid point. And that's a challenge. And again, I'm not trying to dismiss science as a whole. I'm just saying what good science does is build models that explain-- that's easiest-- predict-- that's next easiest-- and control.
And some science can't control, they can only explain and predict. And you mentioned some. But they better at least be able to predict or I think they're barely a science. They're a collection of phenomena.
And then we're lucky that we have a field that actually, we might have the possibility of building stuff that goes beyond explain and predict. In that sense, it is a more super science and more impactful for exactly the reasons you say. So I'm not trying to dismiss other sciences because they're not trying to do that, but I think all science has to aim to predict, at least. Otherwise, it's hard to-- it's a collection of phenomena to start. It's pre-paradigmatic in the Kuhn sense.
So that's how I would challenge back. It doesn't mean you have to build, but you have to aim to predict. And in our field, the things we're predicting are complicated phenomena, so we need complicated models, which means you need engineering not just as a tool, which was mentioned earlier, but as a hypothesis class. The models themselves are hypotheses.
And our field has a hard time with accepting those because they're quite complicated. But that's kind of where we are. And that's the only position I'm trying to take at the moment.
TOMASO POGGIO: But Jim, this is generally true for all of sciences. I mean, there are computers. There are many other tools, very powerful tools.
And if you have a theory of geology or evolution, you can simulate your theory on a computer and check whether what comes out is consistent with the data. And the same with the brain. And something we should be careful, as a computational neuroscientist, working between the science and engineering of intelligence, is to avoid thinking that the models we are testing are actually the real thing. They're just simulating models of the brain, which may be true or wrong. And so--
JIM DICARLO: But Tommy, I agree with that point that it's not the only activity you want to do. But those models in the examples you gave-- first of all, the question was about geology. Yes, you could build a model of geology, but you might not be able to influence geology even with the model. That was where the question was coming from.
But your point of the models are not themselves sufficient I agree with. But the models also, then, put together all the phenomenon principles you have and tell you where your theories are lacking. And then that leads to new theory.
And in complex systems, that's the way you have to go. And you need that in your tool box as well. So I don't think we disagree. I'm not saying the models are the only thing you need to do, and I think that's the point you're making.
GABRIEL: I want to quickly read a question from the audience, my friend and colleague, Tomer Ullman, who's too shy to raise his hand. He writes something-- he writes, "The main focus of the Turing test was behavioral. So perhaps a follow-up question would be, should a test for human intelligence need to satisfy only behavioral tests or do they also need to take the algorithm into account as well?" And Tomer, if you are still here, maybe you can expand on this. But the question, I think, for the three panelists is whether a Turing test should only concern behavior or should also match algorithms.
AUDIENCE: Yeah, it's a classic question and I'm sure that the panelists have heard this before, but it's maybe worth bringing up in the context of all the participants that we have here, which is, again, the Turing test wasn't a test of intelligence. It was supposed to replace the question, "Can machines think?" Turing said, "I don't know how to think about that question. I'm going to replace it with a different question, the Turing test."
But it's been taken as a test for intelligence. And one of the main focuses of it was behavioral. If you can do this behavior, you are said to pass the test.
And a lot of people-- philosophers, cognitive scientists, artificial intelligence people-- said, what that is fundamentally missing is the focus on an algorithm. And people like Ned Block made this very forceful argument that you could have a giant lookup table the size of the galaxy. And if that's a giant lookup table that caches all human's conversations, it would pass the Turing test.
But obviously, Block said-- this is appealing to his intuition. I don't know if we run this on Mechanical Turk, but Block said, obviously, we wouldn't consider that as intelligent. This is an appeal to an intuitive theory of intelligence. We wouldn't assume we wouldn't consider that giant lookup table intelligent. Therefore, algorithms are also important for our theory of intelligence, not just the purely behavioral test.
And then a lot of people argued with it. And one of the main counter arguments was, well, you can't build a galaxy-sized brain to catch all possible conversations. That lookup table would be insane. You can't do that. There's all sorts of arguments on this.
But I think that one of the reasons that this has come back in full force, the vengeance of the Blockhead since the '70s and '70s, is that GPT-3 is the Blockhead. I don't think GPT-3 is actually a model of human intelligence. And maybe there is, in the mind, things like lookup tables. Maybe there is, in the mind, things that are like caching.
And then I'm sure there are and I'm sure that they're useful, and I'm sure that they're useful for intelligence in the sense of getting some action done quickly. But in the same way the human intelligence can do 5 times 5 by a particular lookup table, you know that's 25 without multiplying it. But you can do 63 times 127 by enacting the multiplication algorithm that isn't a lookup table. And you can have both of these.
I think GPT-3 is much closer to a Blockhead, to a lookup table, to any of these things. Now, I don't know that for sure. That's one of the problems we have with this.
But I think our zero hypothesis should be for any one of these machine learning algorithms that if we see it's doing the behavior that people can do, is keep in mind that any behavior can be done by multiple algorithms. And our zero hypothesis for all these things is that whatever it is that they're doing is not human-like intelligence, because every time that we try to tweak these models, push them, try to do generalization-- including all this stuff for ImageNet, not just for GPT-3, and there's a bunch of people in the audience who have done these things-- they all broke.
They all don't do generalization. They all don't do vision. They all don't do language comprehension.
So I think it's a plausible hypothesis that all this amazing progress that we've been making is, to a certain degree, building a Blockhead. Now, people can retort and say, well, in the mind, you also have a Blockhead. And I agree, we do to some degree.
But I'm not sure that we've come closer with these algorithms to what we would call human intelligence. So I don't know-- I've been throwing a lot of gasoline on some fire right now, but maybe going back to is a behavioral test enough? Because then maybe GPT-3 is on the right track.
JOSHUA TENENBAUM: But by an "algorithmic test," Tomer, what do you mean? What does that mean actually? Because I think some of us in cognitive science, a lot of what we try to do is come up with basically better behavioral tests to get at algorithms which are otherwise invisible.
Or neuroscientists like Jim might say, well, no, the way you study the algorithms is you have to study the neurons. And that makes sense, but only gives you a certain window on algorithms. And I think those are basically two incomplete ways we have to study the algorithms. So do you have one of those in mind, or the other, or some other kind of thing when you say an "algorithmic level?"
AUDIENCE: Who knows what I have in mind? That's the argument. Sorry, that was facetious.
[INTERPOSING VOICES]
AUDIENCE: I feel like I'm taking up a lot of oxygen in the room by my--
JOSHUA TENENBAUM: No, it's great.
[INTERPOSING VOICES]
JOSHUA TENENBAUM: No, no, no.
TOMASO POGGIO: Tomer, can I ask in the meantime--
[INTERPOSING VOICES]
AUDIENCE: There's a famous exchange, I think, where Turing asked Wittgenstein, what's your point? And Wittgenstein said, I have no point.
TOMASO POGGIO: Chris, I wanted to ask Chris whether he can put up a poll asking the audience when they think that Alexa will be as good as the human assistant that you can hire. How many years will it take?
GABRIEL: So just to be-- so maybe we can put a couple of numbers in there-- so one to five years, 5 to 10, 10 to 50, 50 to 100, maybe.
TOMASO POGGIO: 2030, 2050. Chris, can you do that?
GABRIEL: So Chris, if you can help us put up a poll about when Alexa will be as good as a human.
TOMASO POGGIO: Will be at the level of a human assistant.
AUDIENCE: When you say level of human assistant, you mean when it passes the Turing test? You mean--
TOMASO POGGIO: In that particular sense, yes. I will hire Alexa instead of a person. That's my version of the Turing test.
GABRIEL: So while Chris is working on this, Jim and Tommy, you want to say anything about whether a Turing test would also include algorithms, loosely defined?
TOMASO POGGIO: I think I interrupted Tomer who can continue now. But I think this question I'm asking is that for--
AUDIENCE: I merely wanted to hear from other people on the panel and just on the panel. I mean, there's many, many clever humans here who have lots of interesting thoughts. And I wanted to hear from them.
JIM DICARLO: My answer to Tomer's question-- I mean, it's just an operational set point. You've got to list a set of questions. It's a set of tests.
It's never going to be complete. The bigger that set of tests goes, the more constrained the algorithm is going to be to pass them all. In the infinite limit of a bunch of behavioral tests, you're going to probably be pretty close to a human-like algorithm, would be my position on that.
You might say, well, you basically built a Blockhead. And if that Blockhead fits on my phone, well, success. You know that's going to take over the world in the sense that Tommy described.
AUDIENCE: That's a very reasonable answer from an engineering perspective, that says look-- and I've heard this from people who say, look, I don't care if you would call it intelligent or not. If I can talk in a Star Trek-like fashion to my phone, that's what I need. And it's worth $1 trillion and that's enough.
And that's fine. But I think this gets at possibly an intuitive theory of intelligence, or I don't know if it's a scientific theory of intelligence. I think that a lot of people would say, look, if you built a Blockhead, and I know how it works, and what you've basically built is a giant lookup table.
And you've cached all possible human conversations. And you do that by just referencing the lookup table. That's great. It's worth $1 trillion. I'm amazed that you can do that, but I would not call that intelligent.
And a lot of people have said, I would not call that intelligent. You might retort with saying, what else do you want? I would call that intelligent. But I think a lot of people wouldn't agree with that. And if you don't agree with that, that's at least highlighting the importance of it also matters how you get to the answer.
AUDIENCE: Tomer, I think there's a-- it's even worse than you say. Consider the fact that every behavior, including the time behavior, can be represented as a machine that takes inputs and the state, does a lookup function, and produces a new state and new outputs. So if you make that function, you can make a good enough function approximation by in fact, for example, storing all the possible behaviors that have ever existed, which I believe is almost possible with the current network-- I'm talking about the internet-- then you could probably make somebody that has all the behaviors of a human, at least that you wouldn't be able distinguish very much because you'd be close to that. And in some finite amount of time, I don't know how long, maybe 20 years, I don't know, but the bottom line is here, would that be something you would think of as being like a human? I say if it's me, not at all.
AUDIENCE: Again.
GABRIEL: So you should see a poll on your screen to vote on Tommy's question. So while people keep voting, we have a question from Kaur.
AUDIENCE: Oh OK. Hi. I think my question kind of goes this way-- if you move away from in silicone models, imagine we can look at all the neurons and can manipulate all the neurons of any species, any animal that we can think of, or any living creature. So let's take C. elegans or something.
So is that a good endpoint? So let's say we have this technique now where at any point of time, we can record activity of all neurons of a species, or we can manipulate the activity of all neurons of a species for any kind of stimuli you want. Will that be a good endpoint to an endeavor of science, or engineering, for that matter? Because I feel like there is still something more we want to do--
[INTERPOSING VOICES]
JOSHUA TENENBAUM: Sounds more like a good starting point than a good endpoint.
AUDIENCE: I understand, yeah, exactly. But I think that would be the endpoint, to some extent, of the way the conversation was going. We build a system completely replicating the system that we want to understand, but then if we still want to abstract away from the real system, then I feel like that's kind of not where I at least thought initially where we were going. Then it feels like there's something more to "understanding," more than just building the system that predicts, and something that we can control. It's like trying to build some sort of algorithm--
JOSHUA TENENBAUM: So are there general principles of explanation other-- yeah, I mean, I think here, Tommy and I, if you want to talk about the panelists, Tommy and I would say unequivocally yes. And I'd love to hear Jim's answer. I think-- well, I don't know, because Jim, of all of us, at least, he's been the one most arguing that the big model is the understanding. But Jim, you tell us what you think.
GABRIEL: So Jim, you have to go now, so maybe answer this question and then a few parting thoughts before you leave.
JIM DICARLO: My parting thought is this has been fun. But I would just-- my quick answer is Josh, at least building it would be powerful to do a lot of things, but I hope--
JOSHUA TENENBAUM: Yeah, absolutely, right. Is it necessary? Or maybe I should--
[INTERPOSING VOICES]
JIM DICARLO: --that there's principles underneath. And if they're there, they'll be short cuts, and they'll make things more efficient. And that would be really nice if it was true.
But we have no guarantees on that is the only thing I would add. And I would also add, related to that, you gave up brain disorders earlier as, oh, we don't need this models for that. I don't think you should give that up.
JOSHUA TENENBAUM: I didn't say that, or I didn't mean. Sorry if I was--
[INTERPOSING VOICES]
I was just saying--
JIM DICARLO: But it's the same idea.
JOSHUA TENENBAUM: Some people who study that may not-- yeah, yeah, no, I mean, it's-- I agree. I think--
GABRIEL: Yeah. OK, so--
JIM DICARLO: Sorry, I gotta run. Thank you, guys.
GABRIEL: Thank you. Thank you very much, Jim. I know that you have to go. So thank you very much. That was great.
For those of you who wanted to know when Alexa will be at the level of Tommy's assistant or anywhere assistants, the answer is very clearly and unequivocally, 10 to 25 years. That's the mode of the distribution, at least. So maybe we're getting close--
JOSHUA TENENBAUM: It's also an interestingly broad mode.
GABRIEL: It is pretty close to uniform distribution. That's very true. So I want to ask at this point, maybe also to Josh and Tommy, to have some parting thoughts on this topic.
JOSHUA TENENBAUM: Sure. I mean, I guess I can say a couple of things. I mean, just to finish on what Jim and I were both saying, so Jim said there's no guarantee that there'll be any principles. Yeah, I mean, I guess no guarantees in life, certainly, or in science.
But I think most of us are committed to the idea that we won't be satisfied, at least unless our science leads to general principles, a unifying theory in the way that science has always looked for, in addition to the kinds of engineering models that we're talking about. But it could turn out to be-- could not work, but I think that's what we're aiming for. And I think aiming for that has always served us well, whether it's to make more efficient models, as Jim is saying, or more efficient model building, or a lot of other things, or more flexible, generalizable toolkit for modeling, just as abstract knowledge helps human intelligence, not just in science, achieve flexibility and generality.
I guess I would also just say one other thing that's just reflecting on a comment that Paul Smolensky made at a workshop connecting AI and cognitive science that a number of us were at. I know Tomer was there, and maybe some others here, which was very telling, and also speaks to the difference between science and engineering, but where there is sometimes attention, but often, a productive one. This was a workshop that was specifically-- well, a lot of the interest was on cognitive development and can we build models of intuitive physics or language that learn the way human babies and children do? Or can we use AI to reverse engineer how human children learn common sense and language?
And as often happens when AI people and neuroscience and cognition people get together and debate that, one of the things that comes up is that people on the AI side especially, as well as some people in neuroscience or cognitive science, are very drawn to an approach which is like learn everything from scratch. The thing that makes the model the best models are the ones which you build in the least amount of structure or the least amount of knowledge, and then they learn the most out. And that's what you should be striving for.
But many people in cognitive science, as well as neuroscience, have a different goal. And they say, well, no, what we should be trying to build in is what actually is built in in biology. And our genome builds in an incredible amount of structure about the structure and function of the brain, the learning algorithms, as well as, in some ways, concepts and conceptual representation. And that's a lot of what the study of very early infant cognition has suggested.
So there was a tension between people who were basically trying to build in as little as possible and the less you built in and the more you got out, that was just assumed to be better, versus people who were trying to figure out, build in the key pieces that seem to be built into human babies. And in a moment of some tension between people there, Paul Smolensky-- who, those of you who don't know him, he was a physicist originally, and then a computational neuroscientist, and linguist, and he mostly works in linguistics right now, but using neural networks. He's one of the people who's most advanced creative ideas of neural networks and language, but also very much guided by what we know about human language.
Anyway, Paul said to one of the machine learning speakers, he said, "Are you trying to do something cool or are you trying to do something real?" And this was him, I think, encapsulating the difference between science and engineering. Now, it was it was a little disparaging, the way he put it, but as someone who's on the science side but very interested in engineering and very appreciative of engineering, I see what he was trying to say-- that it does seem like, and going back to what Jerry said also, in terms of engineering, in science, you're trying to understand something that is.
In engineering, you're trying to build something that has never been. And that's amazing, and awesome, and impressive, but what are the things we're trying to build? Well, we might be motivated by societal applications and making the world better. And it would be great for AI to be more motivated by that, when people try to say what should we build that has never been before.
But often these days, AI engineering has just been motivated by some sense of what will be cool. And given how powerful learning systems have been, the cool thing is to show how much you can get from the data without building stuff in. Whereas I think the scientist has a really healthy perspective on that, and says, OK, well, maybe that's cool, but I want to know what's real. And I want to use engineering tools to better understand what's real.
And I think that's not only the right scientific goal, but also one that will help to advance, fundamentally advance AI, because we look to human intelligence and we still see something. That is, by whatever you mean by intelligence, and here, I'd go back to that definition that Tommy really usefully introduced when we started the intelligence initiative-- look to the Latin. Intelligence means "to understand," [LATIN]. So by any notion of intelligence, just what does it mean for a system to understand the world?
We have really only one good example of that, which is the human mind, and the human brain as it implements the human mind. And the science that we do in CBMM and in all of our work, and the scientific discussions we're having here, are one of the best, most salient examples of human intelligence at work trying to understand the world, including itself in the world. So if AI is to really have a future, and an inspiring future that is also where it came from in the early days of the field, it is going to have to look to what is real, not just what is today's momentary sense of what is cool. It's going to have to look to what is real in intelligence and be guided by that. And I think all of us can gain a lot from that perspective, whichever side we're on, as we work at this joint enterprise together.
GABRIEL: Thank you very much, Josh. That was very nice. That was great. Tommy, final thoughts.
TOMASO POGGIO: Just very briefly, a pitched for CBMM-- CBMM was born with the belief that the engineering of intelligence must be complemented, not replaced but complemented, by a scientific investigation of the only example that we have of natural intelligence, which is Homo sapiens. And I think it's only if we understand better the biology of our own intelligence that we'll be in a situation to understand the promises and pitfalls of AI when it will be better than it is today. And in short, I think there are probably two or three order of magnitude in terms of money and people that goes into the engineering of intelligence compared with the science of intelligence today.
And I think that this effort, which is great, on the engineering of intelligence really calls for an equally grand and synergistic of effort on the science of human intelligence. So that's CBMM. And I think there is more work to do beyond the first eight years.
GABRIEL: Very good. So thank you very much, Tommy. That was great. I want to thank both Josh and Tommy and Jim, who's had to leave now, and also all the people who participated through your questions, and all the comments in the chat, which are also great. I'm sorry if we didn't get to your questions. Thank you very much for participating.
TOMASO POGGIO: Well, thank you, Gabriel.
JOSHUA TENENBAUM: Yeah, thanks, Gabriel. And thanks, everybody, for participating.
TOMASO POGGIO: Yeah, thank you all.
have an interactive transcript feature enabled, which appears below the video when playing. Viewers can search for keywords in the video or click on any word in the transcript to jump to that point in the video. When searching, a dark bar with white vertical lines appears below the video frame. Each white line is an occurrence of the searched term and can be clicked on to jump to that spot in the video.
