Machine learning is so annoying! The five sins of ML have sparked a heated discussion among netizens

Yuyang from Aofei Temple
Quantum Bit Report | Public Account QbitAI

In today's artificial intelligence field, machine learning has become dominant and is leading the trend of the times.

But ascending the throne of technology does not mean that your position will be forever secure and everyone will support you.

But some people just don't buy it and want to use a loudspeaker to announce to the world: I hate machine learning!

It even has a logical basis and soul-searching questions for machine learning.

Once you raise your hand, many people will follow you. The doubts about machine learning have caused a stir and sparked a heated discussion.

Five Questions About Machine Learning

The question came from an artificial intelligence researcher (shou) who wished to remain anonymous.

Soul-searching question 1: This year’s technology will be outdated next year

Back then, people said that RNN was good, but then they turned around and embraced CNN.

If you don’t become a technology pioneer, the huge wave of machine learning will soon wash you to death on the beach.

Technological iteration is normal, but endless updates will make a day's focused learning go to waste.

Soul Question 2: Paywall

If you want to do machine learning, both data sets and computing power are indispensable.

With more powerful computing resources, you can train and test solutions faster.

Isn't it just a competition of who can spend more money?

It is true that in many industrial fields, capital investment is also very important, and money always brings faster and better progress, but in the field of machine learning, the problem goes far beyond that.

As long as you are willing to spend money, even if your dataset classifier is garbage, your model may learn faster and better than others - this is the problem.

Oh, and making datasets costs money too!

Isn't this annoying enough?

Soul Question 3: Code/Improving Machine Learning Code is Frustrating

Black box is a cliché, but did you know that it actually causes mental torture to programmers?

Solving problems with code gives people a sense of creativity, but the black-box nature of neural networks undermines all of this.

Is it happy to be an alchemist? No. I adjusted the parameters, and my classification accuracy improved, but why did it improve? No one knows!

The engineer gave an example where he spent weeks changing the shape and settings of the input data and adjusting the number of nodes in each layer, but to no avail.

One day, he changed the activation function on the dense layer from relu to selu, and something magical happened. Just by changing the letters, the accuracy of the network surpassed all attempts in the past few weeks!

It's a terrible feeling, as if you don't have control over your code. You make a change and know it will improve, but no one can tell exactly when, where, and to what extent the improvement will appear.

The longer I spend on this kind of thing, the more the feeling of disappointment will tighten around me. Is it me playing with machine learning, or is it machine learning playing with me?

Engineers are not ruthless parameter-adjusting machines.

Soul-searching question 4: Dependence on data sets

A neural network without a data set is an empty shell without a soul, but it is hard to say what kind of bias the data set will contain.

Taking exam review as an example, the non-machine learning approach is to build an understanding of the learning material from scratch, a solid understanding that is enough to solve any problems that may arise.

The machine learning method is to collect the test questions given by the professor in previous years, and then practice them frantically.

It is true that practicing questions may make your grades look better, but the problem is that after the exam, when you actually solve the problem, those who have truly mastered the knowledge are more likely to play a solid role.
What's worse is that in the logic of machine learning, if the answer is wrong, it must be because the question has never been asked before.

In actual application scenarios, neural networks will turn the inherent biases in the data set into their own characteristics, and when they encounter situations they have never seen before, they become unreliable teammates.

Soul Question 5: There is a lack of connection between people who suggest using machine learning to solve problems and real ML engineers

Arousing heated discussion

The five questions sparked heated discussions and resonated with many people:

I am very annoyed by people who write terrible articles and are smug about it. If they don't open source their code, I don't want to read their articles at all. In this field, many results are simply impossible to reproduce.
I also want to complain about the paywall. I saw in the Nvidia paper: Oh, look how goosey our network is, you only need 8 V100s to reproduce our work. It makes me want to hit someone.

Agreed. Making models fast is mostly a matter of experience, and courses and textbooks are not very helpful. However, there are some technical criteria that can be used to diagnose bias vs. variance issues in models.

Some people disagree:

I don't think these methods are changing that fast. Attention was proposed in 2013, but it's still very important for machine translation. The same example is LSTM, which has been around since the late 90s.
Even RNN, although it's used less than before (this is also controversial), ResNet still has a big conceptual influence on LSTM.

The entry barrier for machine learning is very low, and sometimes you can even get free GPU time from Google and AWS. Even in the field of computer science, ML is not the most expensive. What's more, ML is very open, and many of the latest research results can be easily obtained.

In fact, machine learning is still a very young field with huge potential, but this also means that there are many unknowns. With the advancement of basic research, this black box is likely to become more transparent in the future and exert more powerful superpowers.

What do you think?