# Sixth Lichuang Electric Competition#Autonomous visual unmanned gas station IoT system

Note: * is required

Please fill it out during the registration stage↓

*1. Introduction to project functions

   This autonomous visual unmanned gas station IoT system uses STM32F 103RCT6 as the main control MCU and HK32F103C8T6 as the co-processing MCU . It is equipped with OpenMV3 visual recognition module , Espressif WIFI module, SIM800C ( GSM ), TTS voice chip , and Gizwits Cloud IoT Platform and other special equipment. From the user's mobile phone, Gizwits IoT cloud, host computer management and other aspects of data interaction, a set of autonomous unmanned IoT refueling equipment with a large number of practical application scenarios is realized. The system solution framework is shown in Figure 1.

                     Figure 1: System solution framework

The specific functions of this project are as follows:

( 1 ) Establish a vehicle fuel tank port recognition algorithm model , correctly analyze and obtain the relative coordinate position of the fuel tank port through machine vision recognition algorithms, and link the robotic arm for accurate positioning .

( 2 ) Connect the device to the Gizwits IoT server and download the Gizwits client program through a mobile portable device to select the corresponding refueling package and obtain current refueling information .

( 3 ) Implement the host computer management software of the gas station park. Through the host computer management software, the autonomous refueling nozzle equipment in the park can be managed uniformly and real-time information of the equipment in the park can be obtained .

( 4 ) Friendly human-computer interaction equipment allows users and managers to understand the current operating status of the equipment in real time from the visual and voice aspects .

( 5 ) Active safety alarm equipment, real-time monitoring of fuel volatilization and fire detection in the park, so as to handle emergency incidents as soon as possible .

The physical picture of the project is shown in Figure 2 below:

                 Figure 2: Physical picture of autonomous vision unmanned gas station

*2. Project attributes

The project is disclosed for the first time; it is an original project; it has not participated in any competitions or school defenses.

* 3. Open source agreement

GPL3.0 open source agreement

Please fill in during the competition stage↓

* 4. Hardware part

   This design uses STMicroelectronics STM32F 103RC T6 as the main control chip and HK32F103C8T6 as the co-processing chip . The two controllers are mainly responsible for communication and data interaction with all functional modules . By collecting OpenMV3 feature detection data and image feature values, and using the BP neural network training method to improve the accuracy of fuel tank mouth recognition; configure the Espressif WIFI module to connect to the Gizwits IoT platform , and create the data node of the device on the Gizwits Cloud platform to achieve Gizwits data analysis and packetization, sensor data and communication data conversion logic ; handles the data interaction between the mobile App user control end, the host computer management end and each functional module.

         The OpenMV3 machine vision recognition module mainly uses feature detection ( find_keypoint ). It first saves the feature values ​​of the target object in KPTS1 and matches the various proportions, sizes and angles of the target features. The algorithm used in AGAST feature point detection is used to match the initial target feature value, and the feature value is used as the input of the BP neural network. The continuous iterative training of the neural network is used to output the final parameters, which can greatly improve the recognition accuracy.

   Through Espressif 's ESP8266 ( WiFi ) module, the Gizwits cloud platform can be connected to quickly realize hardware intelligence. Product Key and Product Secret parameters are assigned to the product through the complete solution of intelligent cloud platform, mobile APP and networking module provided by Gizwits . The Product Key parameter is written by the developer into the device MCU (device main control board) and notified to the WiFi module . After the WiFi module logs into Gizwits, Gizwits will identify the Product Key product. The Product Secret parameter is used during APP development or server docking. parameters used. The structure of Gizwits IoT platform is shown in Figure 3 below.

                                                      Figure 3: Gizwits IoT platform structure

This project design uses Lichuang EDA tools for PCB Layout:

The project schematic diagram is as follows:

                                                       Figure 4: Project schematic diagram

The project PCB diagram is as follows:

                                               Figure 5: Project PCB drawing

Note: It is recommended to use Lichuang EDA . If you choose other EDA tools, please upload schematics in PDF format, PCB drawings in PDF format, and PCB files in Gerber format in the attachments. Here you can explain in detail your project implementation principles and mechanisms, precautions, debugging methods, testing methods, etc. It is recommended to introduce your ideas to others in the form of pictures and texts.

* 5. Software part

   First, initialize the STM32 F103 chip and HK32F103 chip , and then initialize and configure the OpenMV3 camera recognition module, SIM800C ( GSM ) module , Espressif Systems WIFI Gizwits IoT module, TTS speech synthesis module , etc. After the initialization of each device is completed, the system enters normal working mode.

( 1 ) Determine whether to perform visual model training of the fuel tank port. If you choose to perform recognition model training on the fuel tank port, OpenMV3 extracts the characteristic value of the fuel tank port for BP neural network model training; otherwise, wait for the refueling instruction .

( 2 ) When the system receives the refueling command, it determines whether the refueling gun has been accurately positioned at the fuel tank mouth. If the positioning has not been successful, it will continue to perform visual positioning and attitude adjustment; if the positioning is completed, refueling will be started .

( 3 ) When refueling is started, the current refueling information is uploaded to the cloud through the Gizwits WIFI module through interruption . The device shakes hands with the cloud through the GAgent protocol and creates data nodes on the cloud. The equipment operating information is uploaded to the Gizwits server in real time, and users can obtain the equipment's real-time operating information through the mobile app and the host computer .

( 4 ) Determine whether the refueling amount matches the refueling package selected by the user. If it matches, the refueling operation will be completed. If the refueling amount is not reached, the refueling operation will continue. The specific system software flow chart is shown in Figure 6.

Figure 6: System software flow chart

Python machine vision recognition implementation:

   This time, the AGAST algorithm is used for feature extraction and target tracking. Extract the initial image model as the target object feature. KPTS1 saves the target object feature and will match various proportions of the target feature by default. Place the target object to be identified in the center of the camera for identification. Characteristic corners appear during the identification process, proving that the characteristics of the target have been identified and recorded. The specific debugging process is shown in Figure 7.

Figure 7  Python machine vision recognition implementation

The host computer management software implements:

   The host computer management software is developed through Visual Studio . The software is divided into two parts: user information area and field area intelligent management: the user information area records the specific consumption situation of refueled vehicles as well as vehicle license plates and locations; the field area intelligent management area Combining the real-time information of refueling option operations and equipment operation, the specific upper computer management software is shown in Figure 8 .

Figure 8  PC management software

Device access to Gizwits IoT platform:

          STM32F 103 communicates with the cloud through the serial port and GAgent module firmware. By creating devices and data nodes on the cloud, the underlying device operating information is uploaded to the Gizwits IoT server. You can log in to the Gizwits Cloud platform through a mobile portable device to obtain real-time information about the device. Operational information. The device is connected to the Gizwits IoT platform as shown in Figure 9.

                      Figure 9 Device connected to Gizwits IoT platform 

Mobile network access to Gizwits Cloud:

   Secondary development is carried out on the corresponding Android source code engineering framework generated by the Gizwits IoT platform to reconstruct the display interface and data node interaction. The mobile phone is connected to the Gizwits IoT platform through the mobile operator's 4G network to realize the user's refueling package selection. And the monitoring of real-time refueling progress data, as shown in Figure 10.

                      Figure 10 Mobile phone network access to Gizwits Cloud 

Note: If your project involves software development, please upload the corresponding project source code in the attachment. Here you can describe in detail your software flow chart, functional module block diagram, explanation or popular science of related algorithms, source code structure, construction and configuration of compilation environment, source code compilation method, program burning method, etc. It is recommended to introduce your ideas to others in the form of pictures and texts.

*6. BOM list

Note: BOM list involved in the project. Please upload a screenshot of the BOM at this location. Please upload the list details in PDF format to the attachment. Suggestions include model, brand, name, packaging, procurement channels, usage, etc. The specific content and form should be based on clearly expressing the project composition.

*7. Contest LOGO verification

Click the zip to download the competition logo! (Contest logo).zip

* 8. Demonstrate your project and record it as a video for uploading

Video requirements: Please shoot horizontally, with a resolution of no less than 1280×720, in Mp4/Mov format, and the size of a single video is limited to 100M;

Video title: Lichuang Electric Competition: {Project Name}-{Video Module Name}; such as Lichuang Electric Competition: "Autonomous Driving" - Team Introduction.

More details: https://diy.szlcsc.com/posts/06c94d90c2c447dfbd9ed7339ff4a5b1