What It’s Doing Back There

On the afternoon Kemi Jona and I first talk, the Wall Street Journal has a story about kids becoming cognitively dependent on AI. When I ask what is lighting him up right now, he points to that story and the worry behind it: that a generation leaning on chatbots will stop learning to think. “Thirty-plus years ago, I was drawn to my career because of AI and the promise it held for, in particular, helping people learn,” he says. “It’s just such a kick that here we are at the back end of my career, and the stuff that I cared about at the very beginning of my career is now front and center in everybody’s conversation.”

He finds that funny, and he finds it useful, because he has a vantage point almost nobody else in the conversation has. Everyone is talking to a computer now. Jona, Vice Provost for Online Education and Digital Innovation at the University of Virginia, built one that talked back in 1987, when he was a senior at the University of Wisconsin, and he remembers what happened to the magic.

The assignment in his AI class was to write a program modeled on ELIZA, the therapist chatbot from the earliest days of the field. He had wanted to do exactly that since he was a kid. “That was the closest to HAL I ever got,” he says. “I’m talking to the computer and it’s saying smart things back to me. This is what I signed up for.” Then he finished the code, and the feeling left. “As soon as you write the program to do that thing, it doesn’t feel intelligent anymore because you wrote the code,” he says. Show it to a friend, he says, and the friend would be impressed. He could not be. “People ascribe intelligence very readily to a system that might not actually be that smart,” he says.

That is the lens he has carried for nearly forty years, and the want that put him in that class is older still. Somewhere between eighth and tenth grade in Milwaukee, he saw 2001: A Space Odyssey. The film never uses the words artificial intelligence, he points out. It didn’t need to. “The computer was HAL, and it was smart,” he says. “How would anybody ever make a computer that could do that, and talk intelligently to the astronauts?”

The first piece of an answer arrived around 1982, by his reckoning, when his high school put about twenty Apple IIs in a lab: forty-eight kilobytes of memory, “not meg and not gig, but K,” and a language called BASIC. Nobody had a computer at home, so you reserved the room or stayed after school, and Jona stayed as much as he possibly could, teaching himself from code printed in magazines. What hooked him was the sense of agency. “I could write a program and it will do what I tell it to do,” he says. His parents had no frame for it. “At least he’s not out doing drugs,” he says, summing up their view.

Wanting more of that feeling is most of his high school story. Senior year he and his friends Pete and Dave talked the principal into an independent study: scheduling software for the gym teachers, who had to match every student’s ranked sports against the open slots. He made it up as he went, and the gym teachers used it anyway; every few weeks a printout went up on the locker-room wall telling every kid where to report. “It was useful,” he says, and you can still hear what that meant. When winter break came around, it meant losing access to the computer lab for three weeks, a huge setback for getting the program ready and tested for spring semester. The three of them went back to the principal to explain the problem, and he agreed to let them take one of the machines home, equipment Jona puts at around $4,000 by his guess. “Imagine that,” he says. They bought two-liter bottles of Coke and Chips Ahoy and pulled all-nighters on a computer they had sworn to return in one piece. “That was the fuel,” he says. “Fuel for the passion.”

The feeling was never enough on its own, though. At Wisconsin he knew his major on day one, and the university had almost no AI to give him: one graduate course he petitioned his way into, and a class called Machine Learning, the department’s first, the year he left. So he double majored in cognitive psychology. “I wanted the perspective on how humans learn and think,” he says, “so I could create my own AI major.” Senior year he applied to graduate school at the handful of places doing serious AI research — MIT, Stanford, Berkeley, Carnegie Mellon, Yale, UMass Amherst — and taped the replies to the door of his apartment, the rejection letters on one side, the acceptances on the other. “There was all nos and two yeses,” he says. He bought a Yale sweatshirt at his on-campus interview because he was sure he wouldn’t get accepted and would never be back in New Haven. He still has it.

Yale is where the want changed direction. He worked with Roger Schank, whose programs could read a story about a man who goes to a restaurant, leaves a big tip, and walks out, and answer a question about whether he enjoyed his meal. “Everyone knows you don’t leave a big tip if you have a bad meal,” Jona says. “You don’t need the text of the story to spell it out for you.” The work was about everything a person brings to a story that a machine has to be given, and it turned his attention from how a machine could be smart to how a person is. After a year Schank moved his whole lab, “20 of us, 23 of us or something,” to Northwestern, and Jona finished there with a dissertation that turned the HAL question around. Instead of a computer that is smart by itself, he built a tutor meant to be “as smart or approximately as smart as a human teacher,” using what he knew about minds to help people learn. “I spent a good five years thinking about nothing but what makes for a good tutor,” he says.

Everything since has held that line. Learning-by-doing training for GE and the EPA on CD-ROM, before the web could carry video. eCornell. Carnegie Mellon’s West Coast campus. Research faculty at Northwestern, taking the work into middle and high schools. “Let’s use technology for good,” he says of the through line. “And in my particular case, for educational good.” Today he leads the team working to make the University of Virginia reachable for people who cannot come to Charlottesville.

Which is why the worry about kids and AI, the story he brought up the day we met, lands on him differently than it lands on the rest of us. He is not anxious about the machine. He knows what it is doing back there. What worries him is what the research says it does to the person at the keyboard, and the research is the ELIZA lesson at national scale. A student who uses AI gets a better grade on the paper, then does worse on a test taken without it. Tested again six months later, the kids who used AI remember less. “It’s a very sneaky sugar high effect,” he says. Cigarettes were like that once. “You don’t appreciate the long-term negative impacts when you first take that puff.”

The part he will not let go of is who gets hurt. The same tool that helps strong students get stronger makes weaker students weaker, because they use it to get around the work they are struggling with. “No educator that I know of, and no parent that I know of, likes the idea of a strong get stronger, weak get weaker intervention,” he says. He thinks the country needs something like a National Hurricane Center for AI, the agency that gets the supplies in before the storm. “Once it hits, it’s too late,” he says. “We don’t yet really have a feel for how bad it is.”

I ask where the optimism is, if the kid with the Coke bottle still has any. Jona finds it in the reckoning. A five-paragraph essay is no longer proof that anyone learned anything, and every teacher now has to ask the question he has been asking since graduate school. “How do I know that you learned something?” he says. He remembers his own answer from high school, an English teacher who loved Dickens. “I thought I was a computer math nerd, and this guy’s making me love literature,” he says. His fear is a K-shaped education, where the haves get that teacher and everyone else gets the AI version. His hope is the opposite. “It’s going to be painful, it’s going to take a lot of work,” he says, “but hopefully it’s going to get education at K-12 and higher ed back to the meaningful part of it.” He has been talking to the machine for forty years. He is still the one who knows what it is doing back there.