100ASK_Allwinner V853-PRO development board supports human detection and face recognition

aleksib

100ASK_Allwinner V853-PRO development board supports human detection and face recognition [Copy link]

 

## 1. Introduction The V853 chip has a built-in NPU core with a maximum processing performance of 1 TOPS and a 128KB internal cache for high-speed data exchange. It supports API calls for OpenCL, OpenVX, android NN and ONNX, and also supports the import of a large number of commonly used deep learning models. This chapter provides an example to show how to use the NPU part of the V853 chip for human detection and face recognition. Resource package (including application, model and yuvplayer software package files): [source.zip](https://bbs.aw-ol.com/topic/3483/) ## 2. Sample program directory After downloading and unzipping the above resource package, put the sample program 100ASK_viplite-driver in any directory in Ubuntu. Assuming I put it in the /home/book/workspaces directory, the following is the directory structure of the human recognition and face recognition applications. ``` book@100ask:~/workspaces/100ASK_viplite-driver$ tree . ├── 100ASK_Algo │ ├── det_demo │ │ ├── 100ASK_det_demo │ │ ├── aw_errors.c │ │ ├── aw_errors.h │ │ ├── body_384x216.yuv │ │ ├── body_640x360.yuv │ │ ├── demo.cc │ │ ├── face_480x270.nv12 │ │ ├── list.c │ │ ├── list.h │ │ ├── Makefile │ │ ├── Readme.txt │ │ └── testcase.txt │ ├── fdet │ │ └── 100ASK_Facedet_480_288_nv12.nb │ ├── include │ │ ├── awnn_det.h │ │ ├── awnn.h │ │ ├ ── awnn_info.h │ │ ├── awnn_lib.h │ │ └── cross_utils.h │ ├── libawnn.a │ ├── libawnn.so │ │ ├── 100ASK_humandet_384_ 224_V0.nb │ │ ├── 100ASK_humandet_384_224_V1.nb │ │ ├── 100ASK_humandet_640_384_V0.nb │ │ └── 100ASK_humandet_640_384_V1.nb │ └── Readme.txt └── sdk_release ├── libVIPlite.a ├── libVIPlite.so ├── libVIPuser.a ├── libVIPuser.so └── sdk ├── vip_lite_common.h └── vip_lite.h 7 directories, 31 files ``` * The 100ASK_Algo file includes: 1.det_demo directory contains applications and test images 2.fdet directory contains face models 3.include directory contains application header files and 4.pdet directory contains humanoid models 5.sdk_release contains NPU related model processing files## 3. Main function demo.cc The following only explains main.cc in the demo.cc program in the application. For other codes, you can directly view other files in the application compressed package. ``` int main(int argc, char* argv[]) { if (argc != 2) { printf("%s testcase.txt\n", argv[0]); return 0; } int count = 0; /*Create a structure to get the data in parameter 1. The structure is the input nbg path and returns the awnn_info_t structure. Contains library name, network input image width and height, memory usage, threshold information, etc., where count is the number of images and network models processed*/ network_info_t **infos = create_infos_from_testcase(argv[1], count); printf("count=%d\n", count);if (count && infos) { awnn_info_t * nbinfo; unsigned int mem_size = 0; pthread_t thread[count] = {0}; //Get parameter 1 to get the number of corresponding models for (int i = 0; i < count; i++) { nbinfo = awnn_get_info(infos[i]->nbg); printf("info %s %s %u %u %u %f\n", nbinfo->name, nbinfo->md5, nbinfo->width, nbinfo->height, nbinfo->mem_size, nbinfo->thresh); infos[i]->nbinfo = nbinfo; mem_size += nbinfo->mem_size; } //Apply for the memory size required by the NPU model according to memory initialization printf("mem_size: %u\n", mem_size); awnn_init(mem_size); //Create thread processing model and pictures for (int i = 0; i < count; i++) { pthread_create(&thread[i], NULL, detect_thread, infos[i]); } for (int i = 0; i < count; i++) { if (thread[i] != 0) pthread_join(thread[i], NULL); } awnn_uninit(); } //Destroy structure memory destroy_infos(infos, count); return 0; } ``` You can see that the above process does not involve the model processing process. You can check the function description of the corresponding function in the 100ASK_viplite-driver/100ASK_Algo/include/ directory. ## 4. Compile the application Assume that I put the application folder 100ASK_viplite-driver in ~/workspaces/100ASK_viplite-driver, enter the application directory ``` book@100ask:~$ cd ~/workspaces/100ASK_viplite-driver/ book@100ask:~/workspaces/100ASK_viplite-driver$ cd 100ASK_Algo/det_demo/ ``` Assume that my Tina SDK is placed in /home/book/workspaces/tina-v853-open, add environment variables for the application. ``` book@100ask:~/workspaces/100ASK_viplite-driver/100ASK_Algo/det_demo$ export TINA_TOP=/home/book/workspaces/tina-v853-open book@100ask:~/workspaces/100ASK_viplite-driver/100ASK_Algo/det_demo$ export STAGING_DIR=~/workspaces/tina-v853-open/openwrt/openwrt/staging_dir/ ``` After adding, you can compile the application ``` book@100ask:~/workspaces/100ASK_viplite-driver/100ASK_Algo/det_demo$ make /home/book/workspaces/tina-v853-open/prebuilt/rootfsbuilt/arm/toolchain-sunxi-musl-gcc-830/toolchain/bin/arm-openwrt-linux-muslgnueabi-g++ demo.cc list.c aw_errors.c -I./ -I./../../100ASK_Algo/include -L./../../100ASK_Al go -L./../../sdk_release -l:libawnn.a -l:libVIPlite.a -l:libVIPuser.a -o 100ASK_det_demo book@100ask:~/workspaces/100ASK_viplite-driver/100ASK_Algo/det_demo$ ls 100ASK_det_demo aw_errors.h body_640x360.yuv face_480x270.nv12 list.h Readme.txt aw_errors.c body_384x216.yuv demo.cc list.c Makefile testcase.txt ``` After the compilation is complete, an application named 100ASK_det_demo can be generated in the current directory. ## 5. Modify testcase.txt The testcase.txt file contains the model path, the image to be detected, and the image size. 0),4(face) # thresh: postprocess thresh # loop: loop count # input: multi test case each in oneline[network] models/100ASK_humandet_640_384_V1.nb 1 0.25 1 body_640x360.yuv 640 360[network] models/100ASK_humandet_384_224_V1.nb 3 0.25 1 body_384x216.yuv 384 216[network] models/100ASK_Facedet_480_288_nv12.nb 4 0.6 1 face_480x270.nv12 480 270 ``` You can see that the file has explained how to use the file and the parameter definitions of the network nodes. The testcase.txt file can contain multiple processed models. Each model to be processed needs to have specific format requirements. Assuming that I am using the 100ASK_humandet_640_384_V1.nb model file to process the test1_640x360.yuv image file, you can modify the testcase.txt file to ``` [network] models/100ASK_humandet_640_384_V1.nb 1 0.25 1 test1_640x360.yuv 640 360 ``` models/100ASK_humandet_640_384_V1.nb is the storage path of the model; 1 is the category of the model; 0.25 is the post-processing threshold; 1 is the number of cycles; test1_640x360.yuv is the name of the test image; 640 360 is the size of the test image. Note: 100ASK_humandet_640_384_V1.nb is category 1 and 100ASK_humandet_384_224_V1.nb is category 3. If you still want to use the same model and add a test2_640x360.yuv test image, you can directly add the test2_640x360.yuv test image and image size under the model, for example: ``` [network] models/100ASK_humandet_640_384_V1.nb 1 0.25 1 test1_640x360.yuv 640 360 test2_640x360.yuv 640 360 ``` ## 6. Run the test program We have put images in the sample program for you to test. You can also use a TF card to copy the application, model, test image and other files in the virtual machine to the TF card. The file structure in the TF card should be as follows``` book@100ask:/media/book/B89C-7C55$ tree . ├── 100ASK_det_demo ├── body_384x216.yuv ├── body_640x360.yuv ├── face_480x270.nv12 ├── models │ ├── 100ASK_Facedet_480_288_nv12.nb │ ├── 100ASK_humandet_384_224_V1.nb │ └── 100ASK_humandet_640_384_V1.nb └── testcase.txt ``` You can see that the application, test images and testcase.txt file are placed in a folder in the root directory of the TF card. Create a models folder separately and place the model files in this folder. Power on the development board, plug in the 12V power cable and two Type-C data cables, and then insert the TF card into the development board! [Insert picture description here](https://img-blog.csdnimg.cn/df5d526dbb5b42eaa795d013c3f43b6a.jpeg#pic_center) After the development board is started, mount the TF card to the /mnt directory and enter mount /dev/mmcblk1p1 /mnt/ ``` root@TinaLinux:/# [ 9866.263682] sunxi-mmc sdc0: sdc set ios:clk 0Hz bm PP pm UP vdd 22 width 1 timing LEGACY(SDR12) dt B [ 9866.523742] sunxi-mmc sdc0: no vqmmc,Check if there is regulator [ 9866.553676] sunxi-mmc sdc0: sdc set ios:clk 400000Hz bm PP pm ON vdd 22 width 1 timing LEGACY(SDR12) dt B [ 9866.593686] sunxi-mmc sdc0: sdc set ios:clk 400000Hz bm PP pm ON vdd 22 width 1 timing LEGACY(SDR12) dt B [ 9866.606971] sunxi-mmc sdc0: sdc set ios:clk 400000Hz bm PP pm ON vdd 22 width 1 timing LEGACY(SDR12) dt B [ 9866.619855] sunxi-mmc sdc0: sdc set ios:clk 400000Hz bm PP pm ON vdd 22 width 1 timing LEGACY(SDR12) dt B [ 9866.633161] sunxi-mmc sdc0: sdc set ios:clk 400000Hz bm PP pm ON vdd 22 width 1 timing LEGACY(SDR12) dt B [ 9866.855649] sunxi-mmc sdc0: sdc set ios:clk 0Hz bm PP pm ON vdd 22 width 1 timing LEGACY(SDR12) dt B [ 9866.866027] sunxi-mmc sdc0: no vqmmc,Check if there is regulator [ 9866.893675] sunxi-mmc sdc0: sdc set ios:clk 400000Hz bm PP pm ON vdd 22 width 1 timing LEGACY(SDR12) dt B [ 9866.914333] mmc1: host does not support reading read-only switch, assuming write-enable [ 9866.924044] sunxi-mmc sdc0: sdc set ios:clk 400000Hz bm PP pm ON vdd 22 width 4 timing LEGACY(SDR12) dt B [ 9866.938320] sunxi-mmc sdc0: sdc set ios:clk 400000Hz bm PP pm ON vdd 22 width 4 timing UHS-DDR50 dt B [ 9866.948838] sunxi-mmc sdc0: sdc set ios:clk 50000000Hz bm PP pm ON vdd 22 width 4 timing UHS-DDR50 dt B [ 9866.959640] mmc1: new ultra high speed DDR50 SDHC card at address aaaa [ 9866.967694] mmcblk1: mmc1:aaaa SU08G 7.40 GiB [ 9866.990824] mmcblk1: p1 root@TinaLinux:/# mount /dev/mmcblk1p1 /mnt/ [ 3573.953864] FAT-fs (mmcblk1p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck. ``` 进入mnt目录下即可看到TF卡内的文件 ``` root@TinaLinux:/# cd /mnt/ root@TinaLinux:/mnt# ls 100ASK_det_demo face_480x270.nv12 System Volume Information models body_384x216.yuv testcase.txt body_640x360.yuv root@TinaLinux:/mnt# ls models/ 100ASK_Facedet_480_288_nv12.nb 100ASK_humandet_640_384_V1.nb 100ASK_humandet_384_224_V1.nb ``` 确保应用程序、模型文件夹中的模型文件、测试图片、testcase.txt都存在后，即可开始运行测试程序，输入 ./100ASK_det_demo testcase.txt ``` root@TinaLinux:/mnt# ./100ASK_det_demo testcase.txt count=3 info 05dc32ee903c08a3abd53ddd829a8dd5 640 384 10061394 0.000000 info 827b3a725d0fd7173338a56fd06c8db0 384 224 2829624 0.000000 info 3d42fcd47490a62d52ea021760[ 7944.592701] npu[665][665] vipcore, device init.. [ 7944.600170] set_vip_power_clk ON mem_size: 16530637 Version: AW[ 7944.608597] enter aw vip mem alloc size 16530637 NN_LIB_1.0.3 [0xb6f8f560]vip_in[ 7944.615697] aw_vip_mem_alloc vir 0xe1328000, phy 0x48800000 it[121], The version of Viplit[ 7944.623477] npu[665][665] gckvip_drv_init kernel logical phy address=0x48800000 virtual =0xe1328000 e is: 1.8.0-0-AW-2023-02-24 VIPLite driver version=0x00010800... malloc 384x224 nv12 buffer malloc 480x288 nv12 buffer malloc 640x384 nv12 buffer save file: post_body_384x216.yuv save file: post_face_480x270.nv12 save file: post_body_640x360.yuv [ 7946.373546] npu[665][665] gckvip_drv_exit, aw_vip_mem_free [ 7946.379816] aw_vip_mem_free vir 0xe1328000, phy 0x48800000 [ 7946.386027] aw_vip_mem_free dma_unmap_sg_atrs [ 7946.391003] aw_vip_mem_free ion_unmap_kernel [ 7946.396101] aw_vip_mem_free ion_free [ 7946.400118] aw_vip_mem_free ion_client_destroy [ 7946.410983] npu[665][665] vipcore, device un-init.. ``` After executing the test program, you can view the files in the current directory, enter ls ``` root@TinaLinux:/mnt# ls 100ASK_det_demo post_body_640x360.yuv.input_0.txt System Volume Information post_body_640x360.yuv.input_1.txt body_384x216.yuv post_body_640x360.yuv.output_0.txt body_640x360.yuv post_data face_480x270.nv12 post_face_480x270.nv12 models post_face_480x270.nv12.input_0.txt post_body_384x216.yuv post_face _480x270.nv12.input_1.txt post_body_384x216.yuv.input_0.txt post_face_480x270.nv12.output_0.txt post_body_384x216.yuv.input_1.txt post_face_480x270.nv12.output_1.txt post_body_38 4x216.yuv.output_0.txt post_face_480x270.nv12.output_2.txt post_body_384x216.yuv.output_1.txt testcase.txt post_body_640x360.yuv ``` Among the above files, the files with the post prefix are generated after executing the application. The post_data folder contains the execution results and execution performance of the model``` post_data/result_TID.txt: Output the detection results of each model, where TID is the detection thread ID number. post_data/performance_TID.txt: Output the detection performance results of each model, where TID is the detection thread ID number; it is recommended to run a single model and redirect stdout to /dev/null when performing performance evaluation. post_inputN: The frame result of each input, where inputN is the input image path. Add the post_ prefix directly in front of the path to save it. Therefore, do not add ./ to the input image path or use an absolute path. ``` post_body_384x216.yuv, post_body_640x360.yuv, post_face_480x270.nv12 are the images of the output frame results. Unmount the file system used by the TF card, enter ``` root@TinaLinux:/mnt# cd ../ root@TinaLinux:/# umount /mnt/ ``` ## 7. Use yuvplayer software to view the image. After putting the TF card into the card reader, insert it into the computer, and use yuvplayer software to open the image. The file is placed in the yuvplayer-2.5.zip in the source directory. It can be used after decompression. Set the image size and color space to view the image. On the computer, open the U disk folder. Note: If you have not used yuvplayer.exe before, the same icon as my .nv12 file may not appear, but it will not affect subsequent operations.  Select the image we need to view, for example, I want to view the body_640x360.yuv file, click the body_640x360.yuv file, right-click and select Open with.  Click More apps Use the scroll wheel to scroll down and click Find other apps on this computer Find the yuvplayer.exe file in the yuvplayer-2.5 folder extracted from yuvplayer-2.5.zip, select the file and click Open! [Insert image description here](https://img-blog.csdnimg.cn/fa743884d8fe4ce2a29e8da8b107ca9d.png#pic_center) After opening, the following screen will appear. This screen is caused by not setting the size and color space! [Insert image description here](https://img-blog.csdnimg.cn/e711ab0ec47b4751b4bde4f2b7e9ee69.png#pic_center) After clicking Size in the option bar, select Custom. This choice means choosing a custom size! [Insert image description here](https://img-blog.csdnimg.cn/636789ca1a034287a8bebcfc57c09a67.png#pic_center) Enter the image size in the custom size interface. The image size is 640 in width and 360 in height. After setting, click OK! [Insert image description here](https://img-blog.csdnimg.cn/5e85000375914b45bc325ae3b717e24b.png#pic_center) After confirmation, you can see the approximate image, but at this time you still need to modify the displayed image space.  After clicking Color in the options bar, click to select NV12 color space After the selection is completed, you can see the normal image The same is true for other images. It is worth noting that the size of each image may be different and needs to be set according to the size of the source image. Use yuvplayer to view the post_face_480x270.nv12 image file:  ## 7. Test with custom images Since the image formats we usually use are mostly JPG or PNG formats, and their image data formats are common color spaces such as RGB, but the NV12 video storage format is usually used in video encoding and decoding and video transmission applications. It stores the Y-axis chromaticity information and UV-axis sub-pixel information in the image in the same plane. This results in the NV12 format storing a Y value (representing chromaticity information) for each row of the image, and then storing a U and V value (representing sub-pixel information) every two rows. Therefore, the image data in NV12 format is usually half less than other formats (such as RGB). If the image you obtain for your application is NV12, you do not need to convert the format. If the image you want to test is not in NV12 format, but a JPG or PNG image in our common RGB format, you need to use a tool to convert it. The tool we use below is ffmpeg, and the following demonstrates how to use ffmpeg to convert the image storage format. To install ffmpeg, enter sudo apt-get install ffmpeg ``` book@100ask:~$ sudo apt-get install ffmpeg [sudo] password for book: Reading package lists... Done Building dependency tree Reading state information... Done The following package was automatically installed and is no longer required: libdumbnet1 Use 'sudo apt autoremove' to remove it. The following additional packages will be installed: i965-va-driver liba acs0 libass9 libavcodec57 libavdevice57 libavfilter6 libavformat57 libavresample3 libavutil55 libbdplus0 libbluray2 libbs2b0 libchromaprint1 libcrystalhd3 libdc1394-22 libflite1 libgme0 libgsm1 libllvm10 libmysofa0 libnorm 1 libopenal-data libopenal1 libopenjp2-7 libopenmpt0 libpgm-5.2-0 libpostproc54 librubberband2 libsdl2-2.0-0 libshine3 libsnappy1v5 libsndio6.1 libsoxr0 libssh-gcrypt-4 libswresample2 libswscale4 libva-drm2 libva-x11-2 libva2 libvdpau1 libx26 4-152 libxvidcore4 libzmq5 libzvbi-common libzvbi0 mesa-va-drivers mesa-vdpau-drivers va-driver-all vdpau-driver-all Suggested packages: ffmpeg-doc i965-va-driver-shaders libbluray-bdj firmware-crystalhd libportaudio2 sndio d libvdpau-va-gl1 nvidia-vdpau-driver nvidia-legacy-340xx-vdpau-driver The following NEW packages will be installed: ffmpeg i965-va-driver libaacs0 libass9 libavcodec57 libavdevice57 libavfilter6 libavformat57 libavresample3 libavutil55 libbdplus0 libbluray2 libbs2b0 libchromaprint1 libcrystalhd 3 libdc1394-22 libflite1 libgme0 libgsm1 libllvm10 libmysofa0 libnorm1 libopenal-data libopenal1 libopenjp2-7 libopenmpt0 libpgm-5.2-0 libpostproc54 librubberband2 libsdl2-2.0-0 libshine3 libsnappy1v5 libsndio6 .1 libsoxr0 libssh-gcrypt-4 libswresample2 libswscale4 libva-drm2 libva-x11-2 libva2 libvdpau1 libx264-152 libx265-146 libxvidcore4 libzmq5 libzvbi-common libzvbi0 mesa-va-drivers mesa-vdpau-drivers va-driver-all vdpau-driver-all 0 upgraded, 51 newly installed, 0 to remove and 242 not upgraded. Need to get 32.5 MB/47.9 MB of archives. After this operation, 222 MB of additional disk space will be used. Do you want to continue? [Y/n] y ``` At this time, you will be prompted that 222M of space needs to be used. At this time, enter y to download and install After the installation is complete, enter ffmpeg -h ``` book@100ask:~$ ffmpeg -h ffmpeg version 3.4.11-0ubuntu0.1 Copyright (c) 2000-2022 the FFmpeg developers built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04) ... ``` You can see the version number of ffmpeg and the applicable Ubuntu. You can watch the subsequent usage guide by yourself. I will not show all the output information here. Below we use the photos from the "RISC-V Embedded Development Offline Exchange Meeting", the size is 480×270, the image format is JPG  ``` book@100ask:~/workspaces/testImg$ ls test_480-270.jpg ``` 使用ffmpeg进行格式转换，在终端输入ffmpeg -i test_480-270.jpg -s 480x288 -pix_fmt nv12 100ask_face-480-270.yuv ``` book@100ask:~/workspaces/testImg$ ffmpeg -i test_480-270.jpg -s 480x270 -pix_fmt nv12 100ask_face-480-270.yuv ffmpeg version 3.4.11-0ubuntu0.1 Copyright (c) 2000-2022 the FFmpeg developers built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04) configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared libavutil 55. 78.100 / 55. 78.100 libavcodec 57.107.100 / 57.107.100 libavformat 57. 83.100 / 57. 83.100 libavdevice 57. 10.100 / 57. 10.100 libavfilter 6.107.100 / 6.107.100 libavresample 3. 7. 0 / 3. 7. 0 libswscale 4. 8.100 / 4. 8.100 libswresample 2. 9.100 / 2. 9.100 libpostproc 54. 7.100 / 54. 7.100 Input #0, image2, from 'test_480-270.jpg': Duration: 00:00:00.04, start: 0.000000, bitrate: 5699 kb/s Stream #0:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 480x270 [SAR 96:96 DAR 16:9], 25 tbr, 25 tbn, 25 tbc Stream mapping: Stream #0:0 -> #0:0 (mjpeg (native) -> rawvideo (native)) Press [q] to stop, [?] for help [swscaler @ 0x55787ff64b20] deprecated pixel format used, make sure you did set range correctly Output #0, rawvideo, to '100ask_face-480-270.yuv': Metadata: encoder : Lavf57.83.100 Stream #0:0: Video: rawvideo (NV12 / 0x3231564E), nv12, 480x270 [SAR 1:1 DAR 16:9], q=2-31, 38880 kb/s, 25 fps, 25 tbn, 25 tbc Metadata: encoder : Lavc57.107.100 rawvideo frame= 1 fps=0.0 q=-0.0 Lsize= 190kB time=00:00:00.04 bitrate=38880.0kbits/s speed=0.664x video:190kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000% ``` 执行完成没有出现报错，并可以输入ls查看转换后的图像文件 ``` book@100ask:~/workspaces/testImg$ ls test_480-270.jpg 100ask_face-480-270.yuv ``` 可以发现生成了一个名为100ask_face-480-270.yuv文件，该文件的图像尺寸为480×270，图像存储格式为NV12。 将其拷贝到开发板上，与应用程序100ASK_det_demo放在同一目录中，例如： ``` root@TinaLinux:/mnt# ls 100ASK_det_demo 100ask_face-480-270.yuv System Volume Information body_384x216.yuv body_640x360.yuv face_480x270.nv12 models post_100ask_face-480_288.yuv.input_0.txt post_100ask_face-480_288. yuv.input_1.txt post_100ask_face-480_288.yuv.output_0 .txt post_100ask_face-480_288.yuv.output_1.txt post_100ask_face-480_288.yuv.output_2.txt post_body_384x216.yuv post_body_384x216.yuv.input_0.txt post_body_384x216.yuv.input_1.txt post_body_384x216.yuv.output_0.txt post_body_384x216.yuv.output_1.txt post_body_640x360.yuv post_body_640x360.yuv.input_0.txt post_body_640x360.yuv.input _1.txt post_body_640x360.yuv.output_0.txt post_data post_face_480x270.nv12 post_face_480x270 .nv12.input_0.txt post_face_480x270.nv12.input_1.txt post_face_480x270.nv12.output_0.txt post_face_480x270.nv12.output_1.txt post_face_480x270.nv12.output_2.txt testcase.txt ``` Modify testcase.txt, add the input image of the face model, and enter ``` in the serial terminal root@TinaLinux:/mnt# vi testcase.txt ``` Modify the file content For: ``` # example: # [network] # network type thresh loop # input1 width height # input2 width height # input3 width height # # network: nbg path # type: 1(humanoidv1.0),3(humanoidv3.0 ),4(face) # thresh: postprocess thresh # loop: loop count # input: multi test case each in oneline[network] models/100ASK_humandet_640_384_V1.nb 1 0.25 1 body_640x360.yuv 640 360[network] models/100ASK_humandet_384_224_V1.nb 3 0.25 1 body_384x216.yuv 384 216[network] models/100ASK_Facedet_480_288_nv12.nb 4 0.6 1 face_480x270.nv12 480 270 100ask_face-480-270.yuv 480 270 ``` Run the 100ASK_det_demo application ``` root @TinaLinux:/mnt# ./100ASK_det_demo testcase.txt count=3 info 05dc32ee903c08a3abd53ddd829a8dd5 640 384 10061394 0.000000 info 827b3a725d0fd7173338a56fd06c8db0 384 224 2829624 0.000000 info 3d42fcd47490a62d52ea021760[212.538113] npu[465][465] , device init.. [ 212.545449] set_vip_power_clk ONmem_size: 16530637 Version: AW[ 212.553935] enter aw vip mem alloc size 16530637 NN_LIB_1.0.3 [0xb6fe0560]vip_init[121], The version of Viplite is: 1.8.0-0-AW-2023-02-24 [ 212.569158] aw_vip_mem_alloc vir 0xe1353000, phy 0x48800000 [ 212.575548] npu[465][465] gckvip_drv_init kernel logical phy address=0x48800000 virtual =0xe1353000 VIPLite driver version=0x00010800... malloc 480x288 nv12 buffer malloc 384x224 nv12 buffer malloc 640x384 nv12 buffer save file: post_body_384x216.yuv save file: post_face_480x270.nv12 save file: post_body_640x360.yuv save file: post_100ask_face-480-270.yuv [ 215.019076] npu[465][465] gckvip_drv_exit, aw_vip_mem_free [ 215.025231] aw_vip_mem_free vir 0xe1353000, phy 0 x48800000 [ 215.031525] aw_vip_mem_free dma_unmap_sg_atrs [ 215.036451] aw_vip_mem_free ion_unmap_kernel [ 215.041627] aw_vip_mem_free ion_free [215.045646] aw_vip_mem_free ion_client_destroy [ 215.051785] npu[465][465] vipcore, device un-init.. ``` Use yuvplayer software to view the image, configure the image size: width 480, height 270; configure the image color space to NV12  You can see that the model has detected most of the face models. Some of them may not be recognized due to the blurry image, but the accuracy of the model is still relatively high. Here we only demonstrate face recognition. For human figure recognition, you can also refer to this example to modify and add it yourself. With reference to this example, you can try to obtain custom images for face recognition or human figure detection. How to process the images obtained from the camera in real time in 100ASK_V853-PRO for face recognition and human figure detection can be described in the next section MPP.

Jacktang

Using the NPU part of the V853 chip for human detection and face recognition doesn't seem to be very easy to do.

freebsder

What frame rate can video detection achieve?