How ordinary programmers turn to AI

At present, artificial intelligence has become an increasingly popular direction. How can ordinary programmers turn to artificial intelligence? This is a question on Zhihu. This article is an archived version of my answer to this question. Compared with the original answer ( https://www.zhihu.com/question/51039416/answer/138535202 ), some content has been added.

First, the purpose

The purpose of this article is to provide a simple, smooth, and easy-to-implement learning method to help "ordinary" programmers step into the field of AI. Here, I define ordinary programmers as those who have undergraduate knowledge, are usually busy at work, and have limited access to data. Therefore, this article is more like an introductory tutorial on AI "from the scratch".

2. Introduction to AI

AI, or artificial intelligence, does not only include machine learning. Symbols and logic were once considered the key to the realization of artificial intelligence, but now statistical-based machine learning has taken the leading role. The recently popular deep learning is a sub-item of machine learning. At present, it can be said that learning AI is mainly about learning machine learning. However, artificial intelligence is not equivalent to machine learning, and this must be clearly understood when entering this field. For an introduction to the development history of the AI ​​field, it is recommended to read "Introduction to Machine Learning" written by Teacher Zhou. The next question is: Is it easy to cross the door of AI? In fact, it is not easy to cross. Let's take machine learning as an example. In the learning process, you will face a large number of complex formulas, and in actual projects you will face a lack of data and difficult parameter adjustment. If it is just because you think this direction will be "hot" in the future, then these difficulties will make it easy for people to give up. Considering the characteristics of ordinary programmers, is there no way to learn such a difficult subject? The answer is no. Just develop a suitable learning method.

3. Learning Methods

Simply put, the setting of learning methods is to answer the following questions: What do I want to learn? How do I learn? How do I learn? These three questions can be summarized as: learning goals, learning policies and learning plans. The learning goal is relatively clear, which is to step into the field of AI. This goal is not big, so it is easier to achieve. "Too big a goal is to find enough reasons for you to give up it in the future." The learning policy can be summarized as "interest first, practice combined with learning." Simply put, it is to cultivate interest first, and then intersperse practice in learning, and improve in a spiral. This method has a good learning effect and is not easy to give up. After having a learning policy, you can make a learning plan, also known as a learning route. The following is an introduction to the learning route.

4. Learning route

The learning path I recommend is as follows:

Figure 1 AI field learning roadmap

This learning route is designed as follows: first understand this field, establish a comprehensive vision, cultivate sufficient interest, and then start to learn the basics of machine learning. Here, choose a course from shallow to deep to learn, and the course should have enough experiments to be able to practice. After the foundation is laid, you have a sufficient understanding of machine learning and can use machine learning to solve a practical problem. At this time, you can still treat machine learning methods as a black box. After accumulating practical experience, you can consider continuing to learn. At this time, there are two choices, deep learning or continuing machine learning. Deep learning is currently the hottest direction of machine learning. Some methods are different from traditional machine learning, so they can be studied separately. In addition to deep learning, machine learning also includes practical methods such as statistical learning and ensemble learning. If the conditions are sufficient, you can learn both at the same time. Some rules are common to both. After learning, you already have a strong knowledge reserve and can enter more difficult practical combat. At this time, there are two choices. Those in the industry can choose to look at open source projects and read the code for the purpose of modifying the code; those in the academic world can look at papers in specific fields and want to publish papers to solve problems. No matter which one you choose, you need solid knowledge and strong coding skills, so it is very easy to test and train your level. After this stage, you can say that you have stepped into the field of AI. "The master leads you to the door, but the practice is up to you." You have to walk the road by yourself.

The following is a detailed introduction to each stage:

0. Field understanding

Before learning any knowledge, the first step is to understand what this knowledge is? What can it do? Where is its value? If you don't understand these, then learning itself is a boat without direction, you don't know where to go, and it is very easy to sink. After understanding these issues, you can cultivate interest. Interest is the best guide. The motivation and endurance of learning can enable you to cope with the next several stages. For what machine learning is, what it can do, and its relationship with deep learning and artificial intelligence, you can read my blog Let's talk about machine learning:

1. Knowledge preparation

If you have been away from school for too long, or feel that your foundation is not solid, it is best to do some preparatory review in advance. "If you want to do your work well, you must first sharpen your tools." The following preparations are not much, but they are enough to cope with the later stages of study.

• Mathematics: Review the following basic knowledge. Linear algebra: matrix multiplication; Advanced mathematics: derivatives; Probability theory: conditional and posterior probability. Other knowledge can be supplemented as needed in the later learning process;

• English: Always have an online English dictionary, such as iCiBa, so you can easily read some English information web pages;

• FQ: You can use Google anytime and anywhere. This is a very important tool. It's not that the information you search on Baidu is useless, but in many cases, the information you search on Google is more useful than the dozens of pages of information you search on Baidu, especially when searching for English keywords. Saving time is very important to improve learning efficiency.

2. Machine Learning

The first course on machine learning is Andrew Ng's Machine Learning. This course has the following characteristics: moderate difficulty, and enough practical examples, which is very suitable for first-time learners. I do not recommend this course here. Why? The reasons are as follows:

• Time: cs229 was too early, and some knowledge has not kept up with the current development. The most popular neural network was only briefly mentioned. However, Cousera used two class hours to explain neural networks! And it was very detailed;

• Teaching: Ng's teaching in cs229 was a little immature, probably because he was facing online teaching. There were many questions that he didn't explain clearly, and the questions from the people below were actually very annoying, and you often don't care about their questions. This has been significantly improved on Coursera. You will find that Ng's teaching level has improved significantly. He will guide you step by step, be open-hearted, and teach from the simple to the complex. When you encounter a word or term you don't understand, he will also tell you not to worry. More importantly, the derivation and diagrams should not be too perfect, but very detailed and clear. This is really highly recommended;

• Subtitles: The quality of cs229's subtitles is much worse than that of Coursera. The Chinese subtitles on Coursera have been checked by many people, so the quality is guaranteed.

• Homework: There is no homework for cs229. Although you can do some, no one will read it. This is far inferior to the homework on Coursera, which has a weekly deadline and each homework is graded. More importantly, each homework has practical examples, allowing you to practice step by step, and you can see your own results, which gives you a sense of accomplishment!

3. Practice projects

After completing the basic courses, you will have a preliminary understanding of machine learning. There is no problem using them now. You can treat the machine learning algorithm as a black box, put in the data, and you will get results. In actual combat, you need to pay more attention to how to obtain data and how to adjust parameters. If you have time, it is best to do a simple practical project by yourself. Here you need to choose an application direction, whether it is image (computer vision), audio (speech recognition), or text (natural language processing). It is recommended to choose the image field here. There are many open source projects and it is relatively easy to get started. You can use OpenCV for development, which has implemented neural networks, SVM and other machine learning algorithms. After the project is completed, it can be open sourced to Github, and then continuously improved. After the practical project is completed, you can continue to study in depth. At this time, there are two choices, deep learning and continued machine learning;

4. Deep Learning

Deep learning: Deep learning is the hottest research direction at present. It has the following characteristics: knowledge updates quickly, is relatively fragmented, and there is no systematic book. Therefore, learning resources are also relatively scattered. Here are some resource introductions. The parts that are not recommended do not mean they are bad, but they are not suitable for this beginner stage:

• Recommended, UFLDL: A very good basic DL tutorial, also written by Andrew Ng. It has very detailed derivations and translations, and the translation quality is very high;

• Recommended, Deep learning (paper): This 2015 Nature paper was written by three experts in the field of deep learning. After reading the whole paper, you will feel like you have a bird's-eye view of the world. I highly recommend it. If you can only read one paper to understand deep learning, I recommend this one. This paper has a Chinese translation of the same name;

• Recommended: Neural networks and deep learning: The author of this book is very good at expressing profound truths in simple language. Although it is not translated, it is not difficult to read.

• Recommended: Recurrent Neural Networks: This course will explain what RNN is with a practical example. After finishing the course, you will have a clear understanding of how RNN works, which is something you won’t get even after reading a few related papers.

• Not recommended, Neural Networks for Machine Learning - University of Toronto | Coursera: This course is taught by the founder of deep learning. The biggest problem is that it is too difficult, and the old teacher's pronunciation is sometimes not very standard.

• Not recommended, Deep Learning (book): This book is also written by a deep learning expert, but it feels like it was written by the second author, his student. A lot of content is covered, but it doesn't seem to tell you much, just telling you that it comes from a paper. In this case, it might be more appropriate to read the paper directly.

• Not recommended, cs231n: Li Feifei's course is very famous and specializes in CNN. But the biggest problem with this course is that there are no subtitles. Although there are automatic translation subtitles from YouTube, it is worse than nothing.

5. Continue machine learning

Deep learning may not necessarily be the mainstream in the future, at least some experts think so. Traditional machine learning has the following characteristics: knowledge is systematized and there are relatively classic books. Among them, statistical learning (represented by SVM) and ensemble learning (represented by adaboost) are technologies that are widely used in practice. The following are related resources:

• Recommendation, Machine Learning (Zhou Zhihua): In the past, PRML was the first choice for classic machine learning textbooks, but now that Mr. Zhou's book has come out, it is no longer the case. First of all, I recommend reading Mr. Zhou's book. This book has a feature, that is, no matter how difficult the theory is, it can be expressed in simple and concise language. As Mr. Zhou said, "Your level is reflected in making difficult things easy, not making easy things difficult. It is too easy to make something difficult."

• Not recommended, Pattern Recognition And Machine Learning: Not recommended at this stage. PRML looks at many machine learning methods from a Bayesian perspective, which is also one of its major features. But for beginners, this perspective is actually unnecessary. Moreover, this book has no Chinese translation, so it is easy to give up if you read it hard at this stage;

6. Open Source Projects

When the knowledge reserve is sufficient, learning can be transferred to the practice stage again. At this time, practice can still be divided into two steps: learning classic open source projects or publishing high-quality papers. The purpose of learning open source projects should be to optimize as much as possible. Learning simply for reading code is often not very effective. Good open source projects can be searched in Github. Here we take deep learning as an example. There are many excellent open source libraries for deep learning, such as torch, theano, etc. Here are two of them:

• Recommended, DeepLearnToolbox: An earlier deep learning library, written in MATLAB language, more suitable for transitioning from the course you have just learned. Unfortunately, the author no longer maintains it;

• Recommended, tensorflow: Google's open source library, to date, has more than 40,000 stars, very popular, supports mobile devices;

7. Conference Papers

Better courses will recommend you some papers. Some famous technologies and methods are often born in some important conferences. Therefore, reading conference papers from previous years is a way to learn in depth. At this time, the content of some papers will drive you to learn parts of mathematics that you are not good at. Sometimes you will feel that your mathematical knowledge reserves are not enough, so you often need to learn some auxiliary courses. When you have read enough papers, at this stage, if you are a student, you can choose a topic and study and research for the purpose of publishing a paper. Generally speaking, a paper is the product of work. Sometimes an experimental-based paper often requires you to write code or be based on open source projects. Therefore, there is a correlation between the study of open source projects and the work of conference papers. Both can be studied at the same time. As for where to read papers, you can take a look at the CCF recommended rankings to find out which excellent conferences are in this field. Here are two famous top conferences in the field of image and machine learning:

• CVPR: Together with ICCV and ECCV, it is one of the three major conferences in the field of computer vision. Note that the homepage of the conference changes every year, so the search needs to include the year.

• Conference on Neural Information Processing Systems: NIPS for short. Many important works are published here, for example, an important paper on CNN is published here.

8. Freedom to learn

Free learning: At this point, you can say that you have entered this door. You can now study freely according to your interests. You can also study learning resources that were not recommended in the previous stage at will. Here are the comments:

• cs229: The lecture notes written by Ng are very good. The derivation of SVM is very clear. I want to learn SVM recommendation.

• Neural Networks for Machine Learning: The perspective of experts is different from that of ordinary people. Seeing how Hinton views neural networks often makes you feel that it is so. In fact, taking this course is equivalent to reading a paper, because there are papers for you to read in the reference materials of almost every class;

• CS231n: Convolutional Neural Networks for Visual Recognition: The latest knowledge, with detailed homework. There should be a team in China that has translated the subtitles, you can look for it;

• PRML: As a classic machine learning book, it is necessary to read it, and it will give you a different perspective on machine learning.

V. Conclusion

The purpose of this article is to help students who do not have a deep understanding of the field of AI but want to enter this field to step into this door. Here we only say step into this field because it is very difficult to specialize in this field, which requires years of accumulation and hard work. Before learning in the field, it is very important to fully understand your own characteristics and develop a suitable learning method. First of all, you must fully understand this field and cultivate interest. When learning, maintain a step-by-step learning policy, and do not rush to learn too difficult resources; combine learning with practice. Don't just read and watch, and you will feel a sense of accomplishment when you actually do it. When learning a resource, you must have a full purpose. It is not to read the code for learning open source projects, but to read it for writing open source projects; it is not to write papers for publishing papers, but to write papers for doing things. If a learning resource is too difficult for you, it does not necessarily mean that it is your problem, but it may be a problem with the speaker or writer of the learning resource. People who can make difficult problems simple are truly capable people. Therefore, you must learn high-quality resources instead of learning indiscriminately. Finally, remember to learn with interest. Learning takes a long time and the process is difficult, but only interest can be the best help for you to persevere and overcome difficulties.