An Introduction to Deep Learning

Part 3 of "How We Got Here," Chapter 1 of The Machine Intelligence Primer

Deep learning is a machine learning technique that relies on algorithms called Deep Neural Networks (DNN). DNNs are a form of Artificial Neural Network (ANN)—a connectionist algorithm loosely inspired by the circuitry of the human brain [revisit Fig. 1.2 in the previous section]. 

ANNs, first invented in the 1950s [Fig. 1.6], comprise artificial neurons, which are connected by numbers called “weights” and arranged into series of layers. This structure resembles the arrangement of biological neurons in our brains. Data are passed through each neuron in a layer and assigned a weight before being passed on to the next. As data are passed through more neurons, the algorithm makes increasingly complex abstractions. At the output layer, the ANN produces a final inference about the data it ingested (e.g., “there is a cat in this photo,” or “employee pay is correlated with attrition”). 

A simple way to think about ANNs is to use the metaphor of vision. When we perceive an object, say, a slice of cake, light waves are bouncing off it, hitting our retinas, and traveling along the optic nerves of our brains in the form of electrical impulses. The individual impulses that travel to our brains are nonsensical in isolation, but our brain pulls them together and forms an internal representation of the cake. Similarly, ANNs take in many singular data points that are nonsensical in isolation and reconstitute them into a meaningful pattern [Fig. 1.7]. 

ANNs can have anywhere from dozens to thousands of layers of neurons. Advances in digital data availability and computing power in the past 5 years have made it possible to make ANNs perform better by massively increasing the number of layers in them. This is where the “deep” in DNN and deep learning comes from, and it’s what makes deep learning so special and exciting.¹ DNNS have proven to be remarkably accurate and efficient at wide-ranging tasks—particularly those concerned with categorization, such as image identification and speech recognition. 

If you’d like to visualize how a neural network works, we recommend you visit Google’s TensorFlow playground, which allows you to see how neural networks are trained and provides more background on how these amazing algorithms work.