DAVID FERRUCCI INTERVIEW
THE THREAD SEASON TWO
David Ferrucci, Computer Scientist
April 25, 2024
Interviewed by: Kevin Maney
Total Running Time: 33 min and 57 seconds
START TC: 00:00:00:00
ON SCREEN TEXT:
Life Stories Presents
DAVID FERRUCCI:
You think about data driven techniques, you think about language and you think about logic and inference. How do we build AI systems that combine these things? Because we love the incredible language abilities of large language models, but we need precise, reliable, accurate problem solving to solve tough enterprise problems. How do we do that? And we have an answer to that. So I think our vision or EC or elemental cognition vision for how to architect AI systems, you know, ended up being right with a vengeance, as you see now what's happening in the AI space.
ON SCREEN TEXT:
The Thread
David Ferrucci
Computer Scientist
00:00:54:00
INTERVIEWER:
David, just as a starter. Just your name and who you are.
DAVID FERRUCCI:
Dave Ferrucci, CEO and founder of Elemental Cognition and AI company.
INTERVIEWER:
So let's just go back and start with the house where you grew up. Describe the house.
00:01:07:00
DAVID FERRUCCI:
I grew up in the Bronx, at an area called Pelham Parkway. And, those are, you know, small as a single family home on a small plot of land in a sort of a mostly Italian and Jewish neighborhood. I think when a lot of people think of the Bronx, they think very urban, but this was a little bit more suburban, if you will.
00:01:30:00
INTERVIEWER:
What were your parents like?
00:01:31:00
DAVID FERRUCCI:
My dad immigrated from Italy. My mother was born here, and they had a kind of a rough life. Especially because, you know, my mom had lost two young children. And so, struggling through that, you know, for married couples, that was really, really difficult. Eventually my parents got divorced, but they were still sort of very close and very supportive of each other and especially of me and my sister Diana. And so that was kind of the family unit, you know, before the divorce. But even after the divorce, it was kind of a very supportive family.
00:02:08:00
INTERVIEWER:
What did your father do?
00:02:09:00
DAVID FERRUCCI:
So my father worked in construction, and he was very focused on, you have to get educated, you know, education is everything. And it was like, you don't want to do what we, you know, you want to become a doctor, you want to become a lawyer. And there was so much respect for those professions at the time still is. But it was really the pinnacle, right, is to have someone working in, you know, parent working in construction and saying like, this isn't a life for you, you want to go out there and become a doctor or a lawyer. So and in my case, the focus was very much on becoming a doctor.
00:02:47:00
INTERVIEWER:
What was your mother's role in all this? What was she like?
00:02:49:00
DAVID FERRUCCI:
So my mother, my mother was, is also incredibly willful. Both my parents incredibly independent minded. You know, nothing is taken for granted. Everything's questioned. There's always like this search for the truth, search for what's really going on. And of course, they had different perspectives. And I played, I think, a big role, you know, in their lives as well. The role that I played was to get each to understand... To see the world through the other person's eyes. To get them each to understand each other. And that was hard because I was always in battle. They got breaks, but there was no break for me. Because when I was with my mom, I was always representing my dad's perspective. And when I was with my dad representing my mom's perspective. So I was always empathizing.
INTERVIEWER:
That's a big weight for a young guy.
DAVID FERRUCCI:
Yeah.
INTERVIEWER:
To carry.
00:04:05:00
DAVID FERRUCCI:
Well, I never, I never, shied away from taking on challenges. I would feel compelled, really.
00:04:14:00
INTERVIEWER:
As a kid in school, what kind of a kid were you? Were you a studier?
00:04:19:00
DAVID FERRUCCI:
As a little kid I apparently, I was very rambunctious. I mean, this story that my parents told me was that they bought me a Tonka truck. And because it couldn't be broken, because the advertisement was, you can't break this. And I, again, I don't remember this, but my parents told me this story. And my father came home that evening that they had, you know, they bought me a truck, and I was standing in the driveway, put the truck in two pieces and said, "I did it, I broke it." I guess it was a challenge. So, yeah, as a young kid, I was, but I think as, you know, the burden of, you know, you got to go to school and you got to do well and you got to become a doctor, and this is what, you know--- And so a lot of it was very much focused on it was very much focused on schoolwork. One summer, this was actually in high school, you know, you finish school. And so one of the things I liked to do and it would–
00:05:15:00
DAVID FERRUCCI:
I'd get out of school and it was summertime. I'd, you know, blast Alice Cooper's, "School's out for summer. School's out forever." And so meanwhile my dad's in his study and he's going, "What are you doing?" I said, "It's summer." "So why aren't you doing school work?" I said, "because we don't have school. You don't have to do school work." He said, "You should be studying and preparing for the next year." There was like there was no pause, really, that was allowed. And he sees this ad at a local college. Actually, it was Iona College, in New Rochelle, it was called and it was billed as an advanced math class. It was actually a computer programming class on a PDP 11, learning basic programming language. And so he said, "You should do this. So this way this keeps your mind active, you learn something new..." Meanwhile, it's interesting because he very much wanted me to become a medical doctor.
00:06:18:00
DAVID FERRUCCI:
No one really knew what computer science was. I mean, your average people didn't really know what computer science was. Certainly not what artificial intelligence was. And so he was like, go out there and just it's math and you'll learn math and you'll do whatever. So I went to the class and it changed the course of my whole life. I was just blown away. Not because of the programing language or the class itself, but because what just struck me was that I can give this machine these instructions and it would carry out those instructions tirelessly, forever and ever. And so now the matter just became the issue for me became, well, what can I describe? Can I describe how we think? Because if I describe how we think, then I can get the machine to do thinking for me. And wouldn't that be amazing?
00:07:17:00
DAVID FERRUCCI:
Because then I would have this partner that would help me think and do work for me and allow me to be more creative. And, you know, it just and, you know, and I thought to myself, because I spent so much time doing and worrying about school, knowledge was king. Knowing stuff was so critical to doing anything and exploring ideas. And if I can just capture that thinking. And I was obsessed, you know, with that. And I didn't know anything about what it meant from a career perspective or I didn't suddenly that day say, "Oh, I want to become a computer programmer." I was just obsessed with that notion. And I liked to program because I liked-- because it was like a series of puzzles and brain twisters. And how could I get the computer to do this? And how could I get the computer to do that? And, you know, and so, that kind of journey started even though I was off on this path to go to college, you know, as a major in biology.
00:08:19:00
DAVID FERRUCCI:
The biology program had an Apple 2 computer. They weren't doing much with it at the time. And I ended up developing software for the biology department. Ecology software, statistical packages, graphing software, software for the Physiology department. The physiology course, the ecology course. I did analog to digital interfacing. My friend had an Etch A Sketch, and I programmed the Etch A Sketch and then and he was using it and drew something--- I just couldn't stop doing it. The desire to just understand how these machines work, how you can program them, and what the nature of describing algorithms was like. You know, what were the limits of it? What were the possibilities of it? I still wasn't thinking about it from a career perspective. I was on my path to be a medical doctor.
00:09:12:00
INTERVIEWER:
How did you get from there to becoming part of IBM? Joining IBM?
00:09:16:00
DAVID FERRUCCI:
It happened when I was in college. I had worked on all this computer stuff for the school. I loved it, but I was still on the path to become a doctor, and I had to take the MCATs, you know, the entrance exams for medical school and, you know, you take a MCAT preparation class. And the instructor says, Okay, please open your MCATs to page whatever. Take the chemistry test for the next 45 minutes or whatever. So I open up and I look at it, and I did fine in chemistry, but I had no problems with chemistry. I didn't have any prejudice against chemistry, but all of a sudden it hit me, I don't want to do this. I don't want to be a medical doctor. And all my pre-med friends were in the room, you know, and I just closed the book and I got up, and they looked at me and they go, what are you doing? I went to the instructor. I said, I didn't say, hey, I'm leaving the class or whatever. I said, I don't want to be a doctor, I said to the guy.
00:10:18:00
DAVID FERRUCCI:
He looks at me and he and I give him the booklet and he says, "Well, you know, you still have to pay the $500." I said, yeah, that's the least of my troubles right now. Because I knew my trouble was I was going to have to talk to my dad about that. I ended up going to RPI for computer science. It wasn't known for artificial intelligence, but at that point, I didn't have an undergraduate computer science degree. I had, you know, great grades. I did some of my own programing. RPI gave me a shot, accepted me there, and I started doing regular graduate studies in that. And I was interested in artificial intelligence and doing things, and projects related to that. I wanted to do something that related to language and computations with language and intelligence. There was nothing like that at the time.
00:11:18:00
DAVID FERRUCCI:
But there was this class called, I think it was language structures. And it was actually taught in the literature department. It wasn't taught in the computer science department or in the engineering department. And it was about, you know, the logical structure of language, the primitives that allow us to construct language from ideas, to organize ideas in the form of language and what the fundamentals of that kind of word. I found it fascinating, and I said, I want to build a computer program. I saw anything I liked and I thought, can I build up a computer program that does this? And I was like, you know, I wanted to kind of build a computer program that does semantic, what was called semantic networks, to represent knowledge in terms of a semantic network, connections between ideas and words. And can I get it to answer questions? And this became my master's thesis.
00:12:18:00
DAVID FERRUCCI:
It's called Cosmos. And RPI had an industry day or something. And this group from IBM research from the T.J. Watson Research Center came up and they saw me giving a pitch on Cosmos, and they were starting and had started an AI project at IBM. And they said, would you like to come for an interview for a job? And I went there and the T.J. Watson Research Center was just beautiful, majestic really. Beautiful, modern, yet kind of classic architecture mix. As you entered, there was a desk and it was all stone. And I went there, anyway, I did an interview and I got offered a job, and so even though I didn't finish my Ph.D., I was like, this is just too good to be true. And I started working there. And it was an AI project. It was in conjunction with the group at Carnegie Mellon University, so they were partnering with them.
00:13:17:00
DAVID FERRUCCI:
I didn't have as big of a role as I would have liked. I was only a master's student as opposed to a PhD, and they mostly just hired PhDs. I wanted to get my PhD anyway, so I'm going to go back and get my PhD. Where's my father at this point is like, oh my gosh, you actually got a job in this stuff at this great company at IBM, and you're going to quit? I quit and I went back to RPI and I got my PhD. While I was getting my PhD, I was looking for a really good project to exercise my thesis on, and looking around for that inside of RPI and even outside of RPI. And I actually get a call from somebody at IBM research, who I had never met before, wasn't part of when I first one went there. And said, you know, you came up in the list of resumes and kind of what you're doing at RPI that we review. Are you interested in this kind of project? And I said, sure. And I got a job at IBM research.
00:14:26:00
INTERVIEWER:
So you're back at IBM. You're working on, my understanding, question and answering technology.
00:14:32:00
DAVID FERRUCCI:
Yeah.
INTERVIEWER:
And there's this push to have a big kind of event.
00:14:37:00
DAVID FERRUCCI:
Yeah. So I go back to IBM. The IBM execs are thinking about what's the next grand challenge that IBM could take on. And at around that time, Ken Jennings was winning on jeopardy! He had a 72 game winning streak. And people started thinking, hey, wow, can we create a machine that is that good at taking these jeopardy questions? Which were, you know, punny and tricky and, you know, and answer these questions as precisely and correctly. And can it even know what it knows? Because, you know, do you buzz in? You have to buzz in. You know, if you think you know, you don't buzz if you don't think you know, otherwise you lose points. So there are lots of interesting challenges. And it was popular. An opportunity to kind of be on television for IBM. And so they shopped this around, you know across... They shopped it around across IBM, especially IBM research for 2004 2005.
00:15:42:00
DAVID FERRUCCI:
And all the researchers were saying no. This is impossible. This is crazy. You'll look ridiculous. The vast and put across the company was no. When it got around to me for the first-- I didn't... For the first couple years it was shopped around, I was very busy doing something else. I didn't really understand what the hope and expectation was. And then when I had a little bit more time to focus on it came around again. This was the end of 2006. And I said, like, this is a perfect fit for me, and what I've trained for, and what I built this team that does open domain question answering for. This is really a perfect fit. It looks impossible, but let me think about it. And so I think about it, I did some personal experiments and I came back and I convinced everybody that while very hard, there's a shot. I think it is possible.
00:16:44:00
DAVID FERRUCCI:
My managers at the time and my mentor, Art Sokolow, was very supportive of this. We went to Paul Horn and pitched it, and Paul Horn was like, "Okay, let's do it." Like, this is the kind of thing IBM does, which was fabulous. And we're all, you know, and I get my team committed to this. I have some really challenging conversations with the team, because even some of the people on my technical team were like, this is too hard. We're going to embarrass ourselves. And I really, you know, I had a call to action. I was like, you know, we're working in this space. Natural language question answering. You know, we're chugging away at it. We've been a number of government projects. We knew where all the other universities stood in this sort of a much easier version of the problem. They're only doing like on average, like 35% or consistently in that range accuracy. To win a jeopardy, you had to do 75% and better, and you had to know what you knew.
00:17:43:00
DAVID FERRUCCI:
You had to predict, you know, or have an accurate confidence. In other words, if I think I know this answer, in fact, you do. If you don't, you don't, right. You had... All the state of the art systems were nowhere near good at this. So there were some very serious challenges. And I said, you know, I said to the researchers that I and I said, guys, if... Let's say we don't take this project and you keep chugging along and, you know, I won't say you publish, I don't know, three or 4 or 5 papers a year on advancing this whole thing based on your years of experience doing this. And if I come back to you four years from now, five years from now, will you know whether or not you can do the jeopardy problem after all those papers and accolades? And they looked at me and he said, we still wouldn't know. I said, come on. We're researchers. Even if we fail, at least we'll learn why we failed. Because we'll have enough investment to take a really honest crack at this thing.
00:18:46:00
DAVID FERRUCCI:
And that convinced them. And so, you know, they were on board. We kicked this project off. And, you know, the Watson thing kind of gets started. There was still a lot of internal debate that went on, you know, lots of people at IBM were like, this is crazy. Some people would laugh. I'd walk down the hallway, they'd laugh at me. They would say, you're hilarious. Why would you ever do this? All you're going to do is, like, risk your career, you know. For me, because I had been working in question answering for a long time, it was very clear what we were going to do. You know, system was going to get questions. We were going to decide whether or not to answer it and then answer it. It was--- we wanted it to be about question answering. We didn't want to be about robotics or anthropomorphism. So some of the stipulations I had is I really don't want a human face. I don't want a human hand involved.
00:19:43:00
DAVID FERRUCCI:
I, you know, I don't. I didn't even want voice recognition. I said voice recognition wasn't really up to snuff at the time, and it would just introduce error. But then they wondered, you know, in actual competition, is it going to be competitive? Is it going to be, like, unbelievably competitive where it was like no contest or was it going to be terrible? And so there was this whole concern of whether it would be an interesting game one way or the other. It was nerve racking, quite honestly. I mean, so much was at stake. It was a very special day. They closed down the entire site. All the IBM executives were invited. My team was invited, senior people did jeopardy and Sony Pictures was invited. You know, my biggest fear was that we had worked so hard and we knew we built a system that can win against champions. It wouldn't win every game, but it could win against champions. And if and and we proved that out in all these practice games, and in all these stats that we did.
00:20:41:00
DAVID FERRUCCI:
But the game that we're going to air, it was there was a chance, you know, 25 to 30% chance we would lose it. And if we lost it people would think, no, we didn't achieve the goal, but we basically did achieve the goal, right. So, we didn't achieve the goal to win every single game, but we can beat the best. So it was kind of interesting, you know, how this whole thing was going to be perceived. And then we went on to play the game. We didn't know we would win it because jeopardy numbers can be very high, but it could flip on a moment because of a double jeopardy or a Final Jeopardy, and the whole game could flip. So it wasn't actually until like the last, I don't know, ten questions on the final board where there was a Daily Double and Watson and Ken Jennings were competing for it, and Watson got it and then got the answer right. And then that clinched the game. And then we knew that we we would win.
00:21:36:00
ALEX TREBEK:
Now we come to Watson, we're looking for Bram Stoker and we find who is Bram Stoker. And the wager? Hello, 17,973…
00:21:52:00
DAVID FERRUCCI:
And then, you know, it was started to be hyped. Watson was hyped a lot. It was being talked about as something more than it was. It was the most advanced open domain question answering system at the time. It could have helped a lot in improving search and in knowledge management and things like that. A lot of what LLMs, for example, can do today. But that was the beginning of that. It wasn't an LLM per se, but it was the type of technology that can get more value out of unstructured information, out of language. Lots of opportunities for that.
00:22:31:00
INTERVIEWER:
So, David, you accomplished the amazing thing. You build Watson, wins on jeopardy! And then you become an IBM fellow.
00:22:37:00
DAVID FERRUCCI:
So I become an IBM fellow during the centennial. It was an amazing time. Incredible. You know, I was full of pride. We got so much, so much publicity. There was the PBS special, the smartest machine in the world. Smartest machine on Earth. Steve Baker wrote a book, Final Jeopardy. So there were so many amazing things. I was traveling all over the place around the world. Giving pitches about Watson, during the IBM centennial, where you learn about what an amazing company IBM was. What an amazing company it is. The amount of influence that it had on the industry was remarkable. And to be part of the whole IBM Centennial. So it was a remarkable-- it couldn't have been more beautiful and rewarding and fulfilling for me. And, I felt such a connection, you know, to the company.
00:23:32:00
DAVID FERRUCCI:
And, you know, I remember the night I got home from the win. And because my my wife Elizabeth, my kids weren't allowed to go, right, they weren't in the group. And they... And this kind of will start to answer your question. So I got home that night and, you know, my wife Elizabeth was like, so, so, so, so. I was like, "It's over. I'm waiting for the next thing now." What's next? So I don't revel a lot in my successes. I love to reflect on my journey, but I don't revel. I'm ready for the next challenge very quickly. I just wanted to pursue this independent idea I had about how AI should... It should--- It's funny because of what happened with large language models, but like AI should... We should create intelligences that separate language, logic, and data.
00:24:39:00
DAVID FERRUCCI:
We kind of have to understand the difference between language, logic and knowledge. These things get conflated in the LLMs. But when we architect AI systems, and this is before large language models, but still I've been working in natural language processing for a very long time. Let's learn how to communicate with language systems. Let's learn how to formalize knowledge for performing exacting, precise inferencing that could solve problems with general purpose problem solvers, which we know how to do. Let's be able to trace knowledge back to its original sources, the facts and the underlying data. Because when we think about human cognition and when we do things well, not just when we do things with language, and we introduce our biases, but when we do things well, we follow a much more structured approach.
00:25:39:00
DAVID FERRUCCI:
We look at information and we look at what people say. We look at where the information's coming from. We assess it, we weigh it. We extract the knowledge. We connect it to the data. We do scientific analysis, and then we do mathematically oriented reasoning on that to make sure that our conclusions are right. This is what we should be doing. This is the sort of intelligence that complements human cognition, which could be full of biases. This is why we follow methods like this, because we know that our language and our cognitive processes can be biased and distorted. And, you know, we talk about hallucinations and we really need to have a more structured approach. We need to use all those things. And so the idea was to kind of bring these things together. And so we started Elemental Cognition, working on that premise, and developed an AI architecture capable of using language for what it's good for, but using mathematical, logical, reasoning and inference for what it's good for: the precision, the accuracy, the traceability, and then using again large language models to smooth the communication with humans and to help extract the knowledge from all this content.
00:26:55:00
DAVID FERRUCCI:
And so it's a more holistic architecture. And it's now hot. I think we will see AI continue to advance, I think we will not suffer a winter, we will see composite architectures that use LLMs and lots of other techniques to solve problems we definitely could not solve before. So I think this is, you know, this is really significant and more than evolutionary, I think this is revolutionary, what you're seeing now. This is the age of artificial intelligence is really is really beginning now. There are fears I have. We as a society have to focus on what we believe is right for us as people. How do we formulate? Commit to, adhere to, the values that we care about. Excellence, integrity, honesty, commitment, compassion, right.
00:27:56:00
DAVID FERRUCCI:
And AI is going to be a tool, so there will be things that are be harder to, you know, easier to do as a result of AI, you know, that's fine. Learn how to use it, but use it in ways that help humanity, don't hurt humanity. And that's, you know, the way I position AI. When people say, but what if it does, you know, something that I already know how to do? I said, there's always places to go. There's always better ideas. There's always thinking about what matters to humanity. To learn how to use it to help humanity. It's really that simple.
00:28:36:00
INTERVIEWER:
Did you have heroes that you really looked up to? People that really were a big influence on you?
00:28:41:00
DAVID FERRUCCI:
Picture this, right, in this small house in the Bronx, a little patch of land. I think it was 100 by 80 or something like that. And I'm in my basement, no windows, and I have my Apple two computer, and I'm just programing all the time, and imagining my future. And I would flip through the magazines we had in the house, and in the Time magazine--- This is now 1980 or 1981. In Time magazine is an IBM ad that talked about IBM fellows. And it was about freedom, which starts, you know, with the headline, "Dreamers, Heretics, Gadflies, Mavericks and Geniuses. The story goes that Henry Ford once hired an efficiency expert to evaluate his company. After a few weeks, the expert made his report. It was highly favorable, except for one thing. "It's the man down the hall," said the expert. Every time I go by his office, he's just sitting there with his feet on his desk."
00:29:50:00
DAVID FERRUCCI:
""He's wasting your money." The man replied. Mr. Ford once had an idea that saved us millions of dollars. At the time, I believe his feet were planted right where they are now. At IBM we have 46 people like that and we don't worry about where they put their feet. They are IBM fellows. They earn the title by having ideas that made a difference. Their job is to have more ideas like that, but under a very special condition. It's called freedom. For a time of at least five years. An IBM fellow is free to pursue any advance project of value to IBM, even if chances for success may seem remote. As a result, some of the great innovations of our time have come from IBM fellows. We may not always understand what they're doing, much less how they do it, but we know this, the best way to inspire an IBM fellow is to get out of the way." Freedom to create. The time to pursue a difficult idea. And, you know, if we're going to struggle, or take our time to do anything. At least the way I perceived it is why do the ordinary? Why do the perfunctory? If you're going to take time to get up in the morning, do the extraordinary.
00:32:03:00
INTERVIEWER:
Why does it have such a powerful impact that you're emotional about talking about it? What's what is it about that freedom that's just so powerful to you?
00:32:14:00
DAVID FERRUCCI:
The persistence to work against the odds in my life produced extraordinary value. You know. Why are you bothering trying to get your parents to understand each other? You'll never get there. The Watson stuff. Why are you doing this? It's impossible. You're going to ruin your career. You can't, you know--- You know all these things about, it's going to take too long. It's going to be too hard. It's not worth it. There's too much risk. So I think I valued independence. I valued freedom. I valued the internal strength to persist.
END TC: 00:33:57:00
