With the emergence of AlphaGo, driverless cars, and intelligent translation, artificial intelligence , a term that has existed for more than 60 years, has become a hot word again as if overnight. Also widely mentioned in the technology and business circles are machine learning , deep learning , neural networks ... But the fact is that in such a noisy and enthusiastic atmosphere, most people still have only a limited understanding of this field.

If anyone is qualified to talk about the "AI revolution" currently underway, it has to be Terry Sejnowski .

When intelligent translation, driverless cars, AlphaGo, and Microsoft XiaoIce were still considered distant visions, Shenovsky had already laid the foundation for the field of deep learning.

Professor Terry Sejnowski.
Image: Salk Institute

Shenovsky was one of a small group of researchers who challenged the mainstream approach to building artificial intelligence in the 1980s. They believed that AI implementation methods inspired by brain biology, called neural networks, connectionism, and parallel distributed processing, would eventually solve the problems that plagued logic-based AI research, and proposed using mathematical models that could learn skills from data. It was this small group of researchers who proved that a completely new approach based on brain-like computing was feasible, laying the foundation for the development of "deep learning."

Taking advantage of the publication of the book "Deep Learning: The Core Driving Force of the Intelligent Age" , the American technology media "The Verge" interviewed Terence Shenovsky and discussed with him what is the difference between artificial intelligence , neural networks , deep learning , and machine learning ? Why has deep learning suddenly become ubiquitous? What can it do? What can't it do? The following is the full interview:

Deep Learning: The Core Driving Force of the Intelligent Era,
CITIC Publishing Group, 2019.2

Full interview

Q: First, I want to ask about definitions. People use the terms artificial intelligence , neural networks , deep learning , and machine learning almost interchangeably . But these are different things. Can you explain?

A: Artificial intelligence can be traced back to 1956 in the United States, when engineers decided to write a computer program that attempted to emulate intelligence.

Within artificial intelligence, a new field has grown up called machine learning. Instead of writing a step-by-step program to do something—the traditional approach in artificial intelligence—you collect a lot of data about the thing you're trying to understand. For example, imagine you're trying to recognize objects, so you collect a lot of images of them. Then, through machine learning, an automated process that can dissect various features, you can determine that one object is a car and another is a stapler.

Machine learning is a very large field that dates back much further. Initially, it was called “pattern recognition .” Later, the algorithms became more extensive and mathematically complex.

In machine learning there are neural networks inspired by the brain, and then there is deep learning. Deep learning algorithms have a specific architecture with many layers of networks through which data flows.

Basically, deep learning is a part of machine learning, and machine learning is a part of artificial intelligence.

Q: Is there anything that “deep learning” can do that other programs can’t?

A: Writing programs is very labor intensive. In the past, computers were so slow and memory was so expensive that people wrote programs using logic, which is how computers work. They manipulated information through basic machine language. Computers were too slow and computing was too expensive.

But now, computing power is getting cheaper and labor is getting more expensive. And computing power is getting so cheap that, slowly, it will become more efficient to let computers learn than to let humans write programs. At that time, deep learning will start to solve problems that no one has written programs for before, such as in fields such as computer vision and translation.

Machine learning is computationally intensive, but you can write one program and by giving it different data sets, you can solve different problems. And you don't need to be a domain expert. So for anything where there is a lot of data, there are a lot of applications.

Q: Deep learning seems to be everywhere now. How did it become so dominant?

A: I can pinpoint this specific moment in history: December 2012 at the NIPS conference, which is the largest AI conference. There, computer scientist Geoff Hinton and two of his graduate students showed that you could use a very large dataset called ImageNet, with 10,000 categories and 10 million images, and use deep learning to reduce classification errors by 20 percent.

Typically, on this dataset, the error decreases by less than 1% in a year. In a year, 20 years of research were spanned.

That really opened the floodgates.

Q: Deep learning is inspired by the brain. How do these fields of computer science and neuroscience work together?

A: Deep learning is inspired by neuroscience. The most successful deep learning network is the convolutional neural network (CNN) developed by Yann LeCun.

If you look at the architecture of a CNN, it's not just a lot of cells, they're connected in a way that basically mirrors the brain. The best studied part of the brain is in the visual system, and in basic research work on the visual cortex, it's shown that there are simple and complex cells there. If you look at the CNN architecture, there are equivalents of simple cells and complex cells, and this comes directly from our understanding of the visual system.

Yann didn't blindly try to copy the cortex. He tried many different variations, but he ultimately converged to the same methods that nature converges to. This is an important observation. The convergence of nature and artificial intelligence has a lot to teach us, and there is much more to explore.

Q: How much of our understanding of computer science depends on our understanding of the brain?

A: Most of our current AI is based on what we knew about the brain in the 60s. We know much more now, and more of that knowledge is baked into the architecture.

AlphaGo, the program that beat the Go champion, includes models not only of the cortex, but also of a part of the brain called the "basal ganglia," which is important for making a series of decisions to achieve a goal. There is an algorithm called temporal difference, developed by Richard Sutton in the 80s, that, when combined with deep learning, is capable of very complex play that humans have never seen before.

As we understand the brain's architecture, and as we begin to understand how to integrate them into artificial systems, it will provide more and more capabilities beyond what we have now.

Q: Will artificial intelligence also affect neuroscience?

A: They work in parallel. There have been huge advances in innovative neurotechnology, from recording one neuron at a time to recording thousands of neurons simultaneously, and involving many parts of the brain at the same time, which has opened up a whole new world.

I say there is a convergence between AI and human intelligence. As we learn more and more about how the brain works, that will be reflected in AI. But at the same time, they've actually created a whole theory of learning that can be used to understand the brain, allowing us to analyze thousands of neurons and how their activity is generated. So there's this feedback loop between neuroscience and AI, which I think is even more exciting and important.

Q: Your book discusses many different applications of deep learning, from self-driving cars to financial trading. Which specific areas do you find most interesting?

A: One application that I was completely blown away by was generative adversarial networks, or GANs. With traditional neural networks, you give an input and you get an output. GANs are able to do things without input — they produce output.

Yeah, I've heard this in the context of these stories about networks creating fake videos. They really do generate new things that look real, right?

In a sense, they generate internal activity. It turns out that's the way the brain works. You can look somewhere and see something, and then you can close your eyes and you can start to imagine something that's not there. You have a visual imagination, and when it's quiet around you, you have an idea when your alarm goes off. That's because your brain is generative. Now, this new type of network can generate new patterns that have never existed. So you can give it, for example, hundreds of images of cars, and it creates an internal structure that can generate new images of cars that have never existed, and they look exactly like cars.

Q: On the flip side, what ideas do you think might be overhyped?

A: No one can predict or imagine what impact the introduction of this new technology will have on the way things are organized in the future. Of course there is hype. We have not yet solved the really hard problems. We do not yet have general intelligence, and some people say that robots will replace us soon, but in fact robots are far behind artificial intelligence, because replicating the body is found to be more complicated than replicating the brain.

Let's look at one technological advancement: the laser. It was invented about 50 years ago and it took up a whole room. It took 50 years of technology commercialization to go from taking up a whole room to the laser pointer I'm using right now when I give this talk. It had to be small enough that you could buy it for five bucks. The same thing is going to happen with hyped technologies like self-driving cars. It's not expected to be ubiquitous next year or in the next 10 years. It may take 50 years, but the point is that there will be incremental advances along the way that make it more and more flexible, safer, and more compatible with the way we organize our transportation networks. What's wrong with hype is that people have the wrong timescales. They expect too many things to happen too quickly, when things only happen in due time.

Welfare time

In order to help readers better understand the book "Deep Learning: The Core Driving Force of the Intelligent Era", Ashin specially invited Mr. Wei Qing, CTO of Microsoft (China), to authoritatively read "Deep Learning" and share with everyone the new business opportunities of artificial intelligence in 2019 and how individuals should prepare for the AI ​​era.

The live broadcast is free for a limited time, so hurry up and you’ll miss it! The live broadcast time is 8pm on January 24th. Scan the QR code below to participate in the live broadcast.

Identify the poster and click "My Invitation Card" in the upper right corner to generate a special poster and invite friends to study together. The top three listeners on the invitation list will receive a signed copy of "Deep Learning: The Core Driving Force of the Intelligent Age" by Terence Shenovsky, worth RMB 88.

This article is compiled from the book "Deep Learning: The Core Driving Force of the Intelligent Age" and the interview with Terrence Sejnowski by the technology websites "The Verge" and "TechRepublic". The original address is as follows:
https://www.theverge.com/2018/10/16/17985168/deep-learning-revolution-terrence-sejnowski-artificial-intelligence-technology

https://www.techrepublic.com/article/the-deep-learning-revolution-how-understanding-the-brain-will-let-us-supercharge-ai/