By Ian Gormely

Jeff Clune doesn’t talk like your average machine learning researcher. 

Where many AI scientists will happily chat about model optimization or robustness, talking to Clune about his work quickly veers towards the evolution of human thinking and culture. 

“I find evolutionary biology endlessly fascinating,” he says. “How does the relatively unintelligent algorithm of Darwinian evolution produce all the amazing marvels that you see in the natural world? Evolution has been innovating endlessly for 3.5 billion years and is still producing surprises. Could we recreate such an open-ended process of invention and discovery inside of a computer?” 

That’s not to say that Clune, an associate professor at the University of British Columbia and one of Vector’s newest Faculty Members, lacks the “canonical” bonafides of his colleagues. Far from it. But he’s embraced this broader way of looking at problems as part of his goal of creating “open-ended” algorithms that continuously learn. 

Of course, reproducing the “magic” of evolution with computers will require extremely powerful, creative, novel AI algorithms. “I’m open to any AI method that might help us do that,” he says. “I try to marry inspirations and ideas from open-ended processes that I see in the real world, such as human culture and evolution, with the best AI methods that might help reproduce that magic inside of a computer.” Often those methods involved deep reinforcement learning, or occasionally evolutionary algorithms.

Even with such lofty goals, the Detroit native — he obtained his bachelor’s degree at the University of Michigan and his Master’s and PhD at Michigan State — has made significant progress. 

In a 2015 paper in Nature, published while working as an assistant professor at the University of Wyoming, he and his colleagues Antoine Cully, Danesh Tarapore, and Jean-Baptiste Mouret trained a robot to walk in a huge variety of different ways. “Once a robot became damaged, it could look into the set of walking behaviors it already knew and, using some fancy machine learning, quickly identify which behavior it should adopt now that its body is damaged.” In a separate Nature paper, Clune and collaborators Adrien Ecoffet, Joost Huizinga, Joel Lehman, and Ken Stanley introduced Go-Explore, an algorithm they developed while working at Uber AI Labs (a lab Clune helped create). Go-Explore helps agents explore and solve complex environments that are challenging because they provide very little feedback about whether the agent is doing the right thing, making it difficult for it to learn. “It’s kind of like playing warmer colder, but where the teacher is silent until you get within a foot of the object.” They challenged the agent to fan out and explore the world and learn as many diverse, yet high-quality, behaviors as possible. 

Taken together, this research is part of a new AI subfield called “quality diversity algorithms,” which Clune and his colleagues are pioneering. Unlike most machine learning systems, which offer the best single solution for a given problem, quality diversity algorithms try to return to you a diverse set of high quality solutions. “After years of people failing to make much progress on some of these hard exploration challenges, this new branch of methods enabled us to finally break through,” he says. 

Quality diversity algorithms are part of a larger aim to create open-ended AI systems, where the goal is a system that truly innovates forever. As an important step in that direction, Clune and  collaborators Rui Wang, Lehman, and Stanley recently introduced the POET system, which creates its own challenges and then solves them in an “open-ended stream of learning and innovation.” For Clune, open-ended algorithms are part of an even more ambitious goal: AI-generating algorithms, or AI that endlessly improves itself. The potential is to create an algorithm that starts simple and keeps getting smarter, all the way until it has produced human-level, or even super-human-level, AI. “I’m interested in AI-generating algorithms because that gets at this issue of open-endedness, which is fascinating, but also because I think they might be the fastest path to producing general intelligence like humans possess.”

Clune will be researching these ideas in his new lab at the University of British Columbia and with his collaborators at the Vector Institute. For more on his work, you can follow him at @jeffclune on Twitter or visit his website http://jeffclune.com.