[Audio recognizer based on GD32F350RB] 3. CMSIS-NN neural network configuration analysis

北方

[Audio recognizer based on GD32F350RB] 3. CMSIS-NN neural network configuration analysis [Copy link]

1. After completing the basic test, you can analyze the program and logic of this specific implementation in detail. As shown in the following attachment, main.c (7.89 KB, downloads: 2)

2018-9-21 09:44 上传

点击文件名下载附件

The library used in the middle is CMSIS-NN, as shown in the following screenshot 2. Main code analysis

1. // conv1 img_buffer2 -> img_buffer1 arm_convolve_HWC_q7_RGB(img_buffer2, CONV1_IM_DIM, CONV1_IM_CH, conv1_wt, CONV1_OUT_CH, CONV1_KER_DIM, CONV1_PADDING, CONV1_STRIDE, conv1_bias, CONV1_BIAS_LSHIFT, CONV1_OUT_RSHIFT, img_buffer1, CONV1_OUT_DIM, (q15_t *) col_buffer, NULL); arm_relu_q7(img_buffer1, CONV1_OUT_DIM * CONV1_OUT_DIM * CONV1_OUT_CH); // pool1 img_buffer1 -> img_buffer2 arm_maxpool_q7_HWC(img_buffer1, CONV1_OUT_DIM, CONV1_OUT_CH, POOL1_KER_DIM, POOL1_PADDING, POOL1_STRIDE, POOL1_OUT_DIM, NULL, img_buffer2); // conv2 img_b uffer2 -> img_buffer1 arm_convolve_HWC_q7_fast(img_buffer2, CONV2_IM_DIM, CONV2_IM_CH, conv2_wt, CONV2_OUT_CH, CONV2_KER_DIM, CONV2_PADDING, CONV2_STRIDE, conv2_bias, CONV2_BIAS_LSHIFT, CONV2_OUT_RSHIFT, img_buffer1, CONV2_OUT_DIM, (q15_t *) col_buffer, NULL); arm_relu_q7(img_buffer1, CONV2_OUT_DIM * CONV2_OUT_DIM * CONV2_OUT_CH); // pool2 img_buffer1 -> img_buffer2 arm_maxpool_q7_HWC(img_buffer1, CONV2_OUT_DIM, CONV2_ OUT_CH, POOL2_KER_DIM, POOL2_PADDING, POOL2_STRIDE, POOL2_OUT_DIM, col_buffer, img_buffer2); // conv3 img_buffer2 -> img_buffer1 arm_convolve_HWC_q7_fast(img_buffer2, CONV3_IM_DIM, CONV3_IM_CH, conv3_wt, CONV3_OUT_CH, CONV3_KER_DIM, CONV3_PADDING, CONV3_STRIDE, conv3_bias, CONV3_BIAS_LSHIFT, CONV3_OUT_RSHIFT, img_buffer1, CONV3_OUT_DIM, (q15_t *) col_buffer, NULL); arm_relu_q7(img_buffer1, CONV3_OUT_DIM * CONV3_OUT _DIM * CONV3_OUT_CH); // pool3 img_buffer-> img_buffer2 arm_maxpool_q7_HWC(img_buffer1, CONV3_OUT_DIM, CONV3_OUT_CH, POOL3_KER_DIM, POOL3_PADDING, POOL3_STRIDE, POOL3_OUT_DIM, col_buffer, img_buffer2); arm_fully_connected_q7_opt(img_buffer2, ip1_wt, IP1_DIM, IP1_OUT, IP1_BIAS_LSHIFT, IP1_OUT_RSHIFT, ip1_bias, output_data, (q15_t *) img_buffer1); arm_softmax_q7(output_data, 10, output_data);

复制代码

3. This program is a standard digital recognition model. It uses a handwritten digital image 32x32pixal to identify which data from 0 to 9 is. A picture is input. Starting from the leftmost, it is divided into 3 layers of convolution calculation. Each layer includes a convolution layer, a pooling layer, and finally a fully connected layer. Finally, it passes through the softmax classification function to directly output the parsed data results. This process directly corresponds to the above code. The first layer, convolution function arm_convolve_HWC_q7_RGB(); uses a special RGB conversion, which is more efficient on color graphics, activation function arm_relu_q7(); pooling layer function arm_maxpool_q7_HWC(); The second layer, the same as the first layer, but the convolution function uses the _fast mode, because the color conversion has been performed in the first layer, convolution function arm_convolve_HWC_q7_fast(); activation function arm_relu_q7(); pooling layer function arm_maxpool_q7_HWC(); The third layer, the same as the second layer, convolution function arm_convolve_HWC_q7_fast(); activation function arm_relu_q7(); pooling layer function arm_maxpool_q7_HWC(); The final output layer is the fully connected layer, arm_fully_connected_q7_opt(); softmax classification function arm_softmax_q7(output_data, 10, output_data); The output_data is a 1x10 array. When the bit is 1, it corresponds to the recognized number. According to the model, the recognition accuracy here is about 92%, which means that some recognition is inaccurate. 4. From an intuitive point of view, there is nothing very difficult to implement. However, the parameters and number of layers of each layer are actually defined and trained by yourself. This training process needs to be trained in a deep learning model. It can be implemented in python or C++. Finally, the trained model needs to be output, and the important parameters need to be saved in another program and converted to the input array parameters for easy use. The following definitions are required: #define CONV2_BIAS {55,50,34,43,-37,35,-21,10,35,-53,-76,7,14,-1,92,20} #define CONV3_WT{ ... ...} ... ...See the attachment for details, arm_nnexamples_cifar10_weights.h (99.29 KB, downloads: 7)

2018-9-21 10:06 上传

点击文件名下载附件

5. Which deep learning model to use depends on your familiarity. This example program uses UCB's caffe, which is very different from the later upgraded caffe2. If you need to install and use it, the required version and development environment are limited. The installation process was ultimately unsuccessful, so it cannot be shared. However, this model is an open model. In fact, it is also possible to train such weights using tensorflow and pytorch. The trained model can be used after exporting. So, let's briefly summarize the process of using edge computing:

• Use tensorflow to create a calculation graph, which includes the number of layers and parameter configuration of NN.
• Then, collect the training data set, input it into the graph, and train it.
• After the training results, export the weights and data of each layer.
• According to the functions and definitions of cmsis-nn, establish the number of layers and activation functions, construct the graph, and import the model data output by the training.

Finally, debug and run. If the development board has a camera, you can crop the camera screenshot in time and input it into the calculation graph to get the corresponding data.

littleshrimp

This is awesome

freebsder

The parameter import of cmsisnn is not open and is only available to commercial customers. Training on a single-chip microcomputer is impossible, and third-party parameters cannot be imported, so cmsisnn is just a gimmick.

nmg

Looking forward to the follow-up

北方

freebsder posted on 2018-9-22 10:57 The parameter import of cmsisnn is not open, and it is only provided to commercial customers. Training on a single-chip microcomputer is impossible, and third-party parameters cannot be imported, so cmsisnn...

This is true that it is not open, but the training model is not made by ARM, but you have to train it yourself. The trained data needs to be exported from Tensorflow or other AI engines, and then written into an array and directly written into the program. It is indeed very difficult, and ARM probably didn't do it either. I now understand that this example uses caffe, but caffe is no longer updated. You can only use Caffe2, but this thing binds data and layers together. To use this framework, you must backport the model trained in Caffe2 to Caffe. This is possible. It's not a gimmick, but a hard nut to crack.

freebsder

北發于2018-9-27 09:55 It is true that it is not open, but the training model is not made by ARM, but you have to train it yourself, and the trained data needs to be obtained from Tensorflow ...

I am talking about the parameters obtained from a third party. ARM does not provide any public information such as tools, formats, documents, etc. to non-commercial customers. Do you want to load it yourself? Well, I wish you all the best.

北方

freebsder posted on 2018-9-28 03:00 I am talking about the parameters obtained from a third party. ARM does not provide any public information such as tools, formats, and documents to non-commercial customers. If you want to install it yourself...

It is indeed quite difficult. I can't do it recently. But it should be enough if I spend some time. After the competition time, I will also make it out to give you some ideas.