A Brief History of Machine Intelligence

Part 2 of "How We Got Here," Chapter 1 of the The Machine Intelligence Primer

MI has long captured our imaginations through science fiction, but it has also existed as an area of serious academic research for more than 60 years. 

In the 1950s, advances in computer science were generating great excitement about the potential for machines to replicate or even surpass human capability at a wide range of tasks. Academics began testing the notion that, instead of providing computers explicit, step-by-step instructions for performing tasks (as is the case with computer programming), machines could be trained to perform tasks on their own.

In 1956, a small group of American researchers proposed an 8-week study of MI at Dartmouth University. The resulting “Summer Research Project on AI” is widely viewed as the birth of formal MI research. The leaders of the project set as their goal to give machines the ability to “use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves.”¹ Modern MI experts are still working to address each of these challenges. But those early researchers’ efforts established the most promising technique we have for achieving MI today: giving machines the ability to learn from data without explicit programming, or “machine learning.”

Machine learning is a technique for achieving MI which involves training algorithms to parse, learn from, abstract, and act on data.² To accomplish this, we feed algorithms, which are sets of rules used to help computers perform problem-solving operations, large volumes of data from which to learn. Generally, the more data a machine learning algorithm is provided the more accurate it becomes at making inferences. 

Just as there is no one way to write a poem or compose a piece of music, there are many approaches to creating machine learning algorithms. Researchers have taken inspiration from a variety of academic fields to develop distinctive styles and structures for algorithms. The “symbolist” approach, for example,takes inspiration from logic and philosophy. It involves codifying human knowledge as sets of rules and programming these rules into computers. Other machine learning approaches draw inspiration from fields as diverse as neuroscience, psychology, evolutionary biology, and statistics.

Over time, various groups of researchers have grown to focus on or favor specific machine learning methods, coalescing into groups MI expert Peter Domingos dubbed the “five tribes of machine learning.”³ (Fig. 1.2) While they may at times disagree about how best to approach machine learning, the term “tribe” isn’t meant to call to mind Survivor. Machine learning experts often do communicate and collaborate across tribal lines. Instead, the “tribal” umbrella is a helpful way to think about the spirit of discovery and competition among machine learning experts.

Since the 1950s, the tribes of machine learning have been working to bring MI out of the lab and into real-world applications. While MI often fell outside the realm of public consciousness during this time, it did achieve incredible feats. Many will remember; for example, Gary Kasparov’s crushing defeat at the hands of IBM’s “Deep Blue” computer in 1997. MI systems also beat the world champion at backgammon in 1979 and drove a car more than 1,000 kilometers on a Paris highway in 1994.^{4, 5}

Considering these successes, you may wonder why MI did not take off prior before the past five years.

A primary hindrance was that, until recently, most approaches to MI systems were incredibly labor-intensive to create and to produce but exhibited only very limited intelligence. This problem is most clearly represented in “expert systems,” which were built primarily in the 1970s and 1980s. Researchers created expert systems by carefully documentinged as data the knowledge of human experts—like accountants and doctors—and representing it as logic-based rules using symbolist machine learning. Once programmed with these rules, computers could answer questions and perform simple tasks from narrow problem domains with great accuracy and scale. A doctor expert system input with the symptoms “runny nose”  and “grogginess” could; for example, indicate a diagnosis of “cold.”

The problem with expert systems was that they needed to be painstakingly reprogrammed each time the rules upon which they relied changed. If the medical community, for example, discovered a new raft of symptoms indicative of a disease, the machine would have no way of knowing without being explicitly reprogrammed with new rules. For this reason, expert systems, and other early forms of MI, were called “brittle”—a term MI researchers use to convey a system is easily broken because it does not evolve to reflect changing circumstances. By the 1980s, most expert systems had been abandoned, and the commercial sector had grown wary of claims about “artificial intelligence” solving business and research problems.⁶

Why couldn’t MI experts simply find a way to make systems that were less brittle? The biggest challenge was the lack of raw materials needed to make machine learning work—particularly the dearth of computer-readable data. Remember: Creating a doctor expert system in the 1980s would have required carefully capturing all a doctor’s thoughts, ideas, and inclinations as data, and then repeating this process with dozens of additional doctors for posterity. A second, and related challenge, was the lack of computing power to process data for machine learning in problem spaces where computer-readable data was readily available. Running machine learning experiments in the lab often required days as computers pored through massive amounts of data. To understand better why a lack of data and computing power held MI back until recently, it’s helpful to take a look at how machines learn.

Below, you can download a quick guide to how machines learn—supervised versus unsupervised learning—and watch the video explainer: