[The 3rd Xuantie Cup RISC-V Application Innovation Competition] Intelligent cigarette cabinet management system based on Lichee Pi 4A

SeasonMay

[The 3rd Xuantie Cup RISC-V Application Innovation Competition] Intelligent cigarette cabinet management system based on Lichee Pi 4A [Copy link]

<i class="pstatus">This post was last edited by SeasonMay on 2023-12-6 13:08</i> ##<center>Intelligent cigarette cabinet management system based on Lichee Pi 4A</center>

### <center>Author: SeasonMay</center>

## 1. Problem Background With the development of the tobacco industry, cigarette cabinet management has become increasingly important. Traditional cigarette cabinet management methods usually require manual inspections and records, which are inefficient and difficult to monitor and manage in a timely manner. At the same time, cigarette boxes in the cigarette cabinet may be stolen or sales conditions cannot be accurately grasped, causing losses to the company and management difficulties. However, there are relatively few products on the market for this purpose, and the cost is high or it is not easy to operate. A relatively low-cost and effective solution is needed. ## 2. Work Introduction Therefore, the intelligent cigarette cabinet management system based on Lichee Pi 4A came into being. Using computer vision technology, it realizes the automatic management and monitoring of the cigarette cabinet by automatically detecting and identifying cigarette boxes in the cigarette cabinet, providing a more efficient, convenient and safe way to manage the cigarette cabinet. At the same time, it can perform intelligent post-processing such as voice broadcast reminders based on visual recognition results, and has great expansion space. ## 3. System Block Diagram
<ignore_js_op>

<div class="tip tip_4 aimg_tip" id="aimg_760196_menu" style="position: absolute; display: none" disautofocus="true">
<div class="xs0">
<p><strong>1.png</strong> <em class="xg1">(31.59 KB, downloads: 0)</em></p>
<p>
<a href="javascript:;" class="loginf" >download attach</a>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_760196" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=760196&handlekey=savephoto_760196">save to album</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>
 The above is the framework description of the entire system, in which the integrated functions mainly include cigarette box detection function, data statistics function and voice broadcast reminder function. The project center is controlled by LicheePi 4A, and other peripheral components used include a USB driver-free camera (such as Logitech C270) and SYN8266 speech synthesis broadcast module. ## 4. Functional Description of Each Part### 1. Cigarette Box Detection Considering that there are often multiple cigarette boxes of different types, different positions, and different light in cigarette containers, and there are a lot of interference in complex environments, it is necessary to accurately detect and classify from the containers. Therefore, the project mainly uses the One Stage target detection algorithm, which has a good combination of accuracy and speed, that is, the YOLOv5 algorithm. Attached is a YOLOv5 algorithm framework diagram: 
<ignore_js_op>

<div class="tip tip_4 aimg_tip" id="aimg_760197_menu" style="position: absolute; display: none" disautofocus="true">
<div class="xs0">
<p><strong>2.png</strong> <em class="xg1">(147.95 KB, downloads: 0)</em></p>
<p>
<a href="javascript:;" class="loginf" >download attach</a>

<a href="javascript:;" onclick="showWindow(this.id, this.getAttribute('url'), 'get', 0);" id="savephoto_760197" url="home.php?mod=spacecp&ac=album&op=saveforumphoto&aid=760197&handlekey=savephoto_760197">save to album</a>

</p>

</div>
<div class="tip_horn"></div>
</div>

</ignore_js_op>
 First, use the labelimg tool to label the dataset on a PC (more than 400 images were collected from the Internet, and preliminary statistics show that there are about 1k cigarette boxes), then throw it on the server to train the pt model, and then convert it to the onnx model. Then follow the official docker and document instructions to cross-compile and generate the corresponding hhb_out and .c files, and then deploy them to the Lizhi Pi development board to run successfully. When detecting and classifying targets, if there are fewer categories, you can directly label them by category when labeling the YOLO dataset, but if there are more categories, from personal experience, it may be more convenient to unify a category labeling YOLO dataset, and then add a classification model such as vgg16 for downstream processing. In addition, the official documentation also has tutorials on compiling and deploying common classification models such as resnet50, vgg16, etc., which is quite convenient. ### 2. When the total number of cigarette boxes is not too large, the results can be directly output. However, if the total number is large and statistical data and categories need to be separated by rows and columns, clustering methods such as DBSCAN can be used to cluster according to the number of rows and columns, so that the results of each cigarette box can be output independently by row and column. ### 3. The voice broadcast project uses the SYN8266 speech synthesis broadcast module, which can theoretically run with the serial port support of pyserial. I also reconstructed a python version based on the official C routine to facilitate the overall operation of the project. For specific code examples and effects, you can check my previous blog post (link at the end), which I feel is sufficient for now. ## 5. Video Demonstration The video demonstration effect is as follows. In order to make the video smaller, a waiting time is cut in the middle https://training.eeworld.com.cn/course/68275 ## 6. Project source code [SeasonMay/LicheePi-SmartCiga (github.com)](https://github.com/SeasonMay/LicheePi-SmartCiga) ## 7. Published blog posts 1. [The 3rd Xuantie Cup RISC-V Application Innovation Competition] LicheePi 4A+ Voice Module & Official Shell: [[The 3rd Xuantie Cup RISC-V Application Innovation Competition] LicheePi 4A+ Voice Module & Official Shell - Pingtouge Chip Open Community - Pingtouge (xrvm.cn)](https://www.xrvm.cn/community/post/detail?spm=a2cl5.27438731.0.0.31d4qTOYqTOYEp&id=4245707806944071680) 2.LicheePi 4A Belated Unboxing (Second Batch): [[Xuantie Cup Third RISC-V Application Innovation Competition] LicheePi 4A Belated unboxing (second batch) - Pingtouge chip open community - Pingtouge (xrvm.cn)](https://www.xrvm.cn/community/post/detail?spm=a2cl5.27438731.0.0.31d4qTOYqTOYEp&id=4237364030945169408) ## 8. Project summary In general, the functional requirements of the original creative plan have been basically met. It is a pity that the python ecosystem on RISC-V is not yet comprehensive, so the support of large models is not yet convenient to carry out. After this competition, I also experienced the development board of RISC-V architecture for the first time. It was very novel and I gained a lot. The project mainly integrates vision and hearing for comprehensive application. Among them, voice broadcast is more friendly to users without basic knowledge, such as merchants. You can know the situation just by following the voice prompts. The main innovation of this project lies in the application innovation of scenes and the comprehensive application of audio and video. Based on the official documents, HHB is used to accelerate the NPU, making full use of the performance of the board. At the same time, the peripheral expansibility of the board is actively explored, and the support of the voice module is also developed. Finally, I am very grateful for the support of the event officials and the community, which allows a newbie like me to experience and learn the development board of the RISC-V architecture, and to learn project development from other big guys, accumulating a lot of practical experience. At the same time, I am very grateful for the concern and patience of the official technical staff to answer questions and solve doubts. Thank you again!

Jacktang

The resolution of USB driver-free camera is not high

[The 3rd Xuantie Cup RISC-V Application Innovation Competition] Intelligent cigarette cabinet management system based on Lichee Pi 4A [Copy link]

Latest reply