[The 3rd Xuantie Cup RISC-V Application Innovation Competition] LicheePi 4A Selection of posture recognition model based on Yolov5...

ccccccc@ · Published on 2023-10-25 17:10

[The 3rd Xuantie Cup RISC-V Application Innovation Competition] LicheePi 4A Selection of posture recognition model based on Yolov5... [Copy link]

<i class="pstatus"> This post was last edited by ccccccc@ on 2023-10-26 09:49 </i> The best ready-made model is selected this time: [ Image-based-Human-Fall-Detection ](https://github.com/uttej2001/Image-based-Human-Fall-Detection " Image-based-Human-Fall-Detection ") 
<ignore_js_op>

<div class="tip tip_4 aimg_tip" id="aimg_743736_menu" style="position: absolute; display: none" disautofocus="true">
<div class="xs0">
<p><strong>选区_031.png</strong> <em class="xg1">(160.2 KB, downloads: 0)</em></p>
<p>
<a href="javascript:;" class="loginf" >download attach</a>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_743736" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=743736&handlekey=savephoto_743736">save to album</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>
 The model can be simply verified on the computer: ```shell #First, you need to install a virtual environment, anconda or miniconda, please Baidu for details#Create a virtual environmentconda create -n sample#You can name it whatever you want, simple and easy to remembersource activate#Activate the virtual environmentconda activate sample#Enter the virtual environment#Enter the local yolov5 pathcd /home/chen/work/nfs/yolov5n/yolov5 #If not, download it#git clone https://github.com/ultralytics/yolov5.git #cd yolov5 pip3 install -r requirements.txt#Arrange the environment``` Save the following picture locally and name it `kite.jpg` 
<ignore_js_op>

<div class="tip tip_4 aimg_tip" id="aimg_743749_menu" style="position: absolute; display: none" disautofocus="true">
<div class="xs0">
<p><strong>kite.jpg</strong> <em class="xg1">(909.1 KB, downloads: 0)</em></p>
<p>
<a href="javascript:;" class="loginf" >download attach</a>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_743749" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=743749&handlekey=savephoto_743749">save to album</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>
 Perform the operation: ```shell python detect.py --source ./kite.jpg --weights ./best.pt --device cpu ``` The result is printed as follows: ```shell (sample) chen@ubuntu:~/work/nfs/yolov5n/yolov5$ python detect.py --source ./kite.jpg --weights ./best.pt --device cpu detect: weights=['./best.pt'], source=./kite.jpg, data=data/coco128.yaml, imgsz=[640, 640], conf_thres=0.25, iou_thres=0.45, max_det=1000, device=cpu, view_img=False, save_txt=False, save_csv=False, save_conf=False, save_crop=False, nosave=False, classes=None, agnostic_nms=False, augment=False, visualize=False, update=False, project=runs/detect, name=exp, exist_ok=False, line_thickness=3, hide_labels=Fal se, hide_conf=False, half=False, dnn=False, vid_stride=1 YOLOv5v7.0-227-ge4df1ec Python-3.11.5 torch-2.1.0+cu121 CPU Fusing layers... Model summary: 213 layers, 7018216 parameters, 0 gradients, 15.8 GFLOPs image 1/1 /home/chen /work/nfs/yolov5n/yolov5/kite.jpg: 448x640 3 walkings, 304.4ms Speed: 40.6ms pre-process, 304.4ms inference, 111.8ms NMS per image at shape (1, 3, 640, 640) Results saved to runs/detect/exp2 ``` 
<ignore_js_op>

<div class="tip tip_4 aimg_tip" id="aimg_744444_menu" style="position: absolute; display: none" disautofocus="true">
<div class="xs0">
<p><strong>选区_040.png</strong> <em class="xg1">(1.16 MB, downloads: 0)</em></p>
<p>
<a href="javascript:;" class="loginf" >download attach</a>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_744444" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=744444&handlekey=savephoto_744444">save to album</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>
 The model can run normally, and start the conversion from pt to onnx. Refer to hhb for conversion: [HHB User Manual](https://www.yuque.com/za4k4z/oxlbxl/xgm5bybl2v3stf0r#qbIEY "HHB User Manual") Because many python libraries cannot be installed when the machine is installing the environment, and git cannot be used in the docker image, I choose to download it directly on the computer, copy it to docker for operation, and then transfer it to the machine, or you can consider mounting the local folder when the image starts```shell #First enter the computer Docker imagesudo docker start hhb2.4#Activate the image#Download the posture recognition modelcd /home/chen/work/nfs/yolov5n/yolov5 git clone https://github.com/uttej2001/Image-based-Human-Fall-Detection fall #Copy the locally downloaded files to the image systemsudo docker cp /home/chen/work/nfs/yolov5n/yolov5/fall hhb2.4:/home/example/th1520_npu/yolov5n/yolov5/fall #Enter the imagesudo docker exec -it hhb2.4 /bin/bash cd /home/example/th1520_npu/yolov5n/yolov5 #Convert the pt file to onnx: python3 export.py --weights ./fall/best.pt --data ../fall/custom_data.yaml --include onnx --imgsz 384 640 ``` 
<ignore_js_op>

<div class="tip tip_4 aimg_tip" id="aimg_744398_menu" style="position: absolute; display: none" disautofocus="true">
<div class="xs0">
<p><strong>选区_038.png</strong> <em class="xg1">(84.08 KB, downloads: 0)</em></p>
<p>
<a href="javascript:;" class="loginf" >download attach</a>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_744398" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=744398&handlekey=savephoto_744398">save to album</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>
 ```shell #Write the following content to run.sh hhb -S --model-file ./fall/best.onnx --data-scale 0.017 --data-mean '124 117 104' --board c920 --input-name images --output-name '/model.24/m.0/Conv_output_0;/model.24/m.1/Conv_output_0;/model.24/m.2/Conv_output_0' --input-shape '2 3 384 640' --postprocess save_and_top5 --simulate-data ../kite.jpg [2023-10-25 07:45:48] (HHB LOG): Start import model. ```Print the following ```shell ./run.sh + hhb -S --model-file ./fall/best.onnx --data-scale 0.017 --data-mean '124 117 104' --board c920 --input-name images --output-name '/model.24/m.0/Conv_output_0;/model.24/m.1/Conv_output_0;/ model.24/m.2/Conv_output_0' --input-shape '2 3 384 640' --postprocess save_and_top5 --simulate-data ../kite.jpg [2023-10-25 07:45:48] (HHB LOG): Start import model. [2023-10-25 07:45:51] (HHB LOG): Model import completed! [2023-10-25 07:45:51] (HHB LOG): Start quantization. [2023 -10-25 07:45:51] (HHB LOG): Start optimization. [2023-10-25 07:45:51] (HHB LOG): Optimization completed! [2023-10-25 07:45:51] (HHB LOG) : Start conversion to csinn. [2023-10-25 07:45:52] (HHB LOG): Conversion completed! [2023-10-25 07:45:52] (HHB LOG): Start operator fusion. [2023- 10-25 07:45:52] (HHB LOG): Operator fusion completed! [2023-10-25 07:45:52] (HHB LOG): Start operator split. [2023-10-25 07:45:52 ] (HHB LOG): Operator split completed! [2023-10-25 07:45:52] (HHB LOG): Start layout convert. [2023-10-25 07:45:52] (HHB LOG): Layout convert completed! [2023-10-25 07: 45:52] (HHB LOG): Quantization completed! [2023-10-25 07:46:09] (HHB LOG): cd hhb_out; qemu-riscv64 -cpu c920 hhb_runtime ./hhb.bm kite.jpg.0. bin Run graph execution time: 199610.84375ms, FPS=0.01 === tensor info === shape: 2 3 384 640 data pointer: 0x11a92c0 === tensor info === shape: 2 24 48 80 data pointer: 0x3d9270 The max_value of output: nan The min_value of output: nan The mean_value of output: nan The std_value of output: nan ============ top5: ====== ===== -1: -169473953031737902729750710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 -1: -1694739530317379 02729750710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 === tensor info === shape: 2 24 24 40 data pointer: 0 x2156c0 The max_value of output: nan The min_value of output: nan The mean_value of output: nan The std_value of output: nan = =========== top5: =========== -1: -169473953031737902729750710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 -1: -1694739530317379027297 50710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 === tensor info === shape: 2 24 12 20 data pointer: 0x251880 The max_value of output: nan The min_value of output: nan The mean_value of output: nan The std_value of output: nan ============ top5: =========== -1: -169473953031737902729750710990328037376.000000 -1: -16947395303173790272975071099032 8037376.000000-1 : -169473953031737902729750710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 ``` The generated files are as follows: 
<ignore_js_op>

<div class="tip tip_4 aimg_tip" id="aimg_744438_menu" style="position: absolute; display: none" disautofocus="true">
<div class="xs0">
<p><strong>选区_039.png</strong> <em class="xg1">(110.83 KB, downloads: 0)</em></p>
<p>
<a href="javascript:;" class="loginf" >download attach</a>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_744438" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=744438&handlekey=savephoto_744438">save to album</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>
 Among them, hhb_runtime is the generated executable file, hhb.bm is the model weight used by the sample program, kite.jpg.0.bin is the Preprocessed input data. The specific meanings of other files can be found in the official file definitions. [HHB User Manual](https://www.yuque.com/za4k4z/oxlbxl/xgm5bybl2v3stf0r#qbIEY "HHB User Manual") Copy the file to the machine: ```shell scp -r ../yolov5 sipeed@192.168.0.104:/home/sipeed/Desktop/udisk/nfs/yolov5n/ ``` Use ssh to remotely log in to the machine for operation: ```shell ssh sipeed@192.168.0.104 cd ~/Desktop/udisk/nfs/yolov5n/yolov5 ./hhb_out/hhb_runtime ./hhb_out/hhb.bm ./hhb_out/kite.jpg.0.bin ``` Print as follows: ```shell (env) sipeed@lpi4a:~/Desktop/udisk/nfs/yolov5n/yolov5$ ./hhb_out/hhb_runtime ./hhb_out/hhb.bm ./hhb_out/kite.jpg.0.bin Run graph execution time: 1607.92810ms, FPS=0.62 === tensor info === shape: 2 3 384 640 data pointer: 0x11a96b0 === tensor info === shape: 2 24 48 80 data pointer: 0x3d9660 The max_value of output: nan The min_value of output: nan The std_value of output: nan ============ top5: = ========== -1: -169473953031737902729750710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 -1: -1694739530317379027297 50710990328037376.000000 === tensor info === shape: 2 24 24 40 data pointer: 0x215ab0 The max_value of output: nan The min_value of output: nan The mean_value of output: nan The std_value of output: nan ============ top5: =========== -1: -169473953031737902729750710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 -1: -1694739530317379027297 50710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 === tensor info === shape: 2 24 12 20 data pointer: 0x251c70 The max_value of output: nan The min_value of output: nan The mean_value of output: nan The std_value of output: nan ========== == top5: =========== -1: -169473953031737902729750710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 -1: -169473953031737902729750710990328037376.000000 ``` The data is abnormal. Try to use the general model deployment method: #First change the command```shell hhb -D --model-file ./fall/best.onnx --data-scale 0.017 --data-mean '124 117 104' --board c920 --input-name images --output-name : '/model.24/m.0/Conv_output_0;/model.24/m.1/Conv_output_0;/model.24/m.2/Conv_output_0' --input-shape '2 3 384 640' --postprocess save_and_top5 --simulate-data ../kite.jpg ``` Copy the generated `hhb_out` to the machine# for operation```shell ./hhb_out/hhb_runtime ./hhb_out/hhb.bm ../kite.jpg Run graph execution time: 1546.36108ms, FPS=0.65 === tensor info === shape: 2 3 384 640 data pointer: 0xed96a0 === tensor info === shape: 2 24 48 80 data pointer: 0x3d9660 The max_value of output: 2.798828 The min_value of output: -11.468750 The mean_value of output: -1.485657 The std_value of output: 12.376108 ============ top5: =========== 38644: 2.798828 38564: 2.746094 38645: 2 .6777 38565: 2.648438 38647: 2.617188 === tensor info === shape: 2 24 24 40 data pointer: 0x215b80 The max_value of output: 3.070312 The min_value of output: -11.781250 The mean_value of output: -1.402809 The std_value of output: 10.823801 ============ top5:=========== 21699: 3.070312 6339: 2.812500 21659: 2.767578 21619: 2.753906 21739: 2.707031 === tensor info === shape: 2 24 12 20 data pointer: 0x251d40 The max_value of output: 5.281250 The min_value of output: -10.593750 The mean_value of output: -1.239471 The std_value of output: 7.504400 ============ top5: =========== 1589: 5.281250 3509: 4.863281 1609: 4.378906 1590: 4.367188 1569: 4.281250 ``` The print seems to be OK. Return to the image for cross-compilation. ```shell cd /home/example/th1520_npu/yolov5n/yolov5 riscv64-unknown-linux-gnu-gcc ../ yolov5n.c -static -o fall_example hhb_out/io.c hhb_out/model.c -Wl,--gc-sections -O2 -g -mabi=lp64d -I hhb_out/ -L /usr/local/lib/python3.8 /dist-packages/hhb/install_nn2/c920/lib/ -lshl -static -L /usr/local/lib/python3.8/dist-packages/hhb/prebuilt/decode/install/lib/rv -L /usr/local/lib/python3.8/dist-packages/hhb/prebuilt/runtime/riscv_linux -lprebuilt_runtime -ljpeg -lpng -lz -lstdc++ -lm -I /usr/local/lib/python3.8/dist-packages/hhb/install_nn2/c920/include/ -mabi=lp64d -march=rv64gcv0p7_zfh_xtheadc ``` In this Generate `fall_example` in the path and copy it to the machine```shell scp -r ./fall_example sipeed@192.168.0.104:/home/sipeed/Desktop/udisk/nfs/yolov5n/yolov5 ``` Enter the machine side page```shell cd /home/sipeed/Desktop/udisk/nfs/yolov5n Modify `interface.py` line 72 `yolov5_example` to `./yolov5/fall_example`, `hhb_out` to Save the following image as `./yolov5/hhb_out` locally and name it kite.jpg ``` 
<ignore_js_op>

<div class="tip tip_4 aimg_tip" id="aimg_744472_menu" style="position: absolute; display: none" disautofocus="true">
<div class="xs0">
<p><strong>kite.jpg</strong> <em class="xg1">(132.53 KB, downloads: 0)</em></p>
<p>
<a href="javascript:;" class="loginf" >download attach</a>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_744472" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=744472&handlekey=savephoto_744472">save to album</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>
 ```shell #Run and print the following python3 inference.py ***** ***** preprocess image ********** ******* run yolov5 and postprocess ******* Run graph execution time: 1546.41077ms, FPS=0.65 detect num: 1 id: label score x1 y1 x2 y2 [0]: 0 0.443974 193.455765 81.969955 465.592712 354.700195 ********** draw bbox ********** [193.455765, 81.969955, 465.592712, 354.700195, 0.443974, 0] ``` The result is as follows: [attach]744474[/attach ] As can be seen from the comparison, due to the change of some accuracy in the conversion, some errors occurred in the result - the posture cannot be judged. However, the transplantation and conversion of the model has taken a relatively successful step./home/sipeed/Desktop/udisk/nfs/yolov5n/yolov5 ``` Enter the machine page```shell cd /home/sipeed/Desktop/udisk/nfs/yolov5n Modify line 72 of `interface.py`: `yolov5_example` is changed to `./yolov5/fall_example`, `hhb_out` is changed to `./yolov5/hhb_out` Save the following picture as a local and name it kite.jpg ``` 
<ignore_js_op>

<div class="tip tip_4 aimg_tip" id="aimg_744472_menu" style="position: absolute; display: none" disautofocus="true">
<div class="xs0">
<p><strong>kite.jpg</strong> <em class="xg1">(132.53 KB, downloads: 0)</em></p>
<p>
<a href="javascript:;" class="loginf" >download attach</a>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_744472" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=744472&handlekey=savephoto_744472">save to album</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>
 ```shell #Run and print as followspython3 inference.py ********** preprocess image ********** ******* run yolov5 and postprocess ******* Run graph execution time: 1546.41077ms, FPS=0.65 detect num: 1 id: label score x1 y1 x2 y2 [0]: 0 0.443974 193.455765 81.969955 465.592712 354.700195 ********** draw bbox ********** [193.455765, 81.969955, 465.592712, 354.700195, 0.443974, 0] ``` The result is as follows: 
<ignore_js_op>

<div class="tip tip_4 aimg_tip" id="aimg_744474_menu" style="position: absolute; display: none" disautofocus="true">
<div class="xs0">
<p><strong>kite_result.jpg</strong> <em class="xg1">(51.15 KB, downloads: 0)</em></p>
<p>
<a href="javascript:;" class="loginf" >download attach</a>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_744474" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=744474&handlekey=savephoto_744474">save to album</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>
 As can be seen from the comparison, due to the change of some precision in the conversion, some errors also occur in the result - the posture cannot be judged. However, the transplantation and conversion of the model has taken a relatively successful small step./home/sipeed/Desktop/udisk/nfs/yolov5n/yolov5 ``` Enter the machine page```shell cd /home/sipeed/Desktop/udisk/nfs/yolov5n Modify line 72 of `interface.py`: `yolov5_example` is changed to `./yolov5/fall_example`, `hhb_out` is changed to `./yolov5/hhb_out` Save the following picture as a local and name it kite.jpg ``` 
<ignore_js_op>

<div class="tip tip_4 aimg_tip" id="aimg_744472_menu" style="position: absolute; display: none" disautofocus="true">
<div class="xs0">
<p><strong>kite.jpg</strong> <em class="xg1">(132.53 KB, downloads: 0)</em></p>
<p>
<a href="javascript:;" class="loginf" >download attach</a>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_744472" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=744472&handlekey=savephoto_744472">save to album</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>
 ```shell #Run and print as followspython3 inference.py ********** preprocess image ********** ******* run yolov5 and postprocess ******* Run graph execution time: 1546.41077ms, FPS=0.65 detect num: 1 id: label score x1 y1 x2 y2 [0]: 0 0.443974 193.455765 81.969955 465.592712 354.700195 ********** draw bbox ********** [193.455765, 81.969955, 465.592712, 354.700195, 0.443974, 0] ``` The result is as follows: 
<ignore_js_op>

<div class="tip tip_4 aimg_tip" id="aimg_744474_menu" style="position: absolute; display: none" disautofocus="true">
<div class="xs0">
<p><strong>kite_result.jpg</strong> <em class="xg1">(51.15 KB, downloads: 0)</em></p>
<p>
<a href="javascript:;" class="loginf" >download attach</a>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_744474" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=744474&handlekey=savephoto_744474">save to album</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>
 As can be seen from the comparison, due to the change of some precision in the conversion, some errors also occur in the result - the posture cannot be judged. However, the transplantation and conversion of the model has taken a relatively successful small step.

ccccccc@ · Published on 2023-10-25 17:23

But I'm not sure where the problem is, if anyone knows please give me some advice!

wangerxian · Published on 2023-10-25 19:31

Can real-time video be identified?

nmg · Published on 2023-10-26 09:43

What if we change the same picture? Can we still only recognize the object but not the posture?

ccccccc@ · Published on 2023-10-26 09:46

nmg posted on 2023-10-26 09:43 What if we change to the same picture? Can it still only recognize the object but not the posture?

Even objects in the same picture have problems recognizing them

ccccccc@ · Published on 2023-10-26 09:47

wangerxian posted on 2023-10-25 19:31 Can real-time video be identified?

Um, image recognition still has problems

wangerxian · Published on 2023-10-26 15:11

ccccccc@ posted on 2023-10-26 09:47 Well, there are still problems with image recognition

Haha, take your time to find the problem, you can always solve it!

freebsder · Published on 2023-10-26 15:20

Give a thumbs up to the heroine!

[The 3rd Xuantie Cup RISC-V Application Innovation Competition] LicheePi 4A Selection of posture recognition model based on Yolov5... [Copy link]

Latest reply

Comments

Comments

Comments