[2024 DigiKey Creative Competition] Smart Trash Can Based on STM32MP157

zfw152666 · 2024-10-26 21:17 上传

[2024 DigiKey Creative Competition] Smart Trash Can Based on STM32MP157 [Copy link]

Smart Trash Can Based on STM32MP157

1. Introduction

Photos of the work; introduction to the work's functions; bill of materials and brief introduction, such as boards, chips, modules used, etc.; 100-200 words

The multifunctional intelligent trash can mainly uses the k210 development board for identification and control and the stm32mp157 for sensor data acquisition and image transmission. The main functions are as follows

Garbage classification identification + steering gear control

Voice control trash can switch

Detecting the overflow level of trash cans

The camera reads the image information and transmits it to the client through UDP. It uses vision to identify the type of garbage. The STM32F4 controls the servo and motor through RTOS to dump the garbage. The garbage data and sensor data are sent to MP157, which then uses socket programming to upload the data to the backend server and visualize it on the web page.

2. System block diagram design ideas, system hardware and software introduction and implementation block diagram, presented in a combination of pictures and texts

Model training can be done on the mxyolov3 platform or the official training platform (this is easier to use, but has a data set size limit of 20M). Before using the development board, you need to use KFLASH to burn the firmware package with the .bin suffix, burn the trained kmodel file to the development board, and then you need a piece of execution code. Subsequent function joint debugging also needs to be added based on the recognition code.

After identification, some control is required, otherwise the result is just a result. In my project, I used the pwm signal to control the servo to rotate to identify the recognition function of the trash can.

To prevent misjudgment, I added a 10-frame detection of the same object in the recognition code before driving the servo.

Using the CAN protocol to control the M2006 DJI motor with a PS2 controller, you can remotely control the trash can to move forward, backward, accelerate, decelerate, spin, etc.

The next step is to get started with it.

3. Functional description of each part The functional description and explanation of each part are presented in a combination of pictures and texts

ODYSSEY - STM32MP157C is a single board computer based on the STM32MP157C, a dual-core Arm-Cortex-A7 processor operating at 650Mhz. The processor also integrates an Arm Cortex-M4 coprocessor, making it suitable for real-time tasks. ODYSSEY - STM32MP157C is created in the form of a SoM (System on Module) and a carrier board. The SoM consists of an MPU, a PMIC, a RAM, and a carrier board in the form of a Raspberry Pi. The carrier board includes all the necessary peripherals, including Gigabit Ethernet, WiFi/BLE, DC power supply, USB host, USB-C, MIPI-DSI, DVP for camera, audio, etc. With this board, customers can quickly evaluate the SoM and deploy the SoM to their own carrier board quickly and easily.

Dual-core Arm-Cortex-A7 core processor with Cortex-M4 integrated som (system on module) including MPU, PMIC, RAM. Raspberry Pi 40-pin compatible carrier board. Compact size and powerfulOpen source Hardware/SDK/API/BSP/OSSpecification | |Item Value| |----------| | Peripheral InterfacesUSB Host1 | 2 x Gigabit Ethernet interface1 x 3.5mm Audio interface1 MIPI DSI Display interface1 x DVP Camera interface2 Grove (GPIO & I2C)1 x SD Card Interface (On Board)][WiFi/Bluetooth| WiFi 802.11 b/g/n 2.4GHzBluetooth 4.1)Onboard LEDs | 1 x Reset LED3 Defined LEDs1 Power LED]Power | 1 x DC Interface (12V/2A Power Input Recommended)1 x USB Type-C Key | 1 x Reset Key1 x User Key1 x Dial Key| Dimensions | 56mm x 85mm| | Operating Temperature | 0 ~ 75℃ | ApplicationsIndustrial (CAN-Ethernet Gateway, etc.)White Appliances (Refrigerator, Microwave Oven, etc.)Medical (Data Logger, etc.)High-End Wearables (VR Devices, etc.)Smart Home Devices

Garbage classification identification + steering gear control

The development board used for garbage classification is K210

Model training can be done on the mxyolov3 platform or the official training platform (this is easier to use, but has a data set size limit of 20M). Before using the development board, you need to use KFLASH to burn the firmware package with the .bin suffix, burn the trained kmodel file to the development board, and then you need a piece of execution code. Subsequent function joint debugging also needs to be added based on the recognition code.

After identification, some control is required, otherwise the result is just a result. In my project, I used the pwm signal to control the servo to rotate to identify the recognition function of the trash can.

To prevent misjudgment, I added a 10-frame detection of the same object in the recognition code before driving the servo.

Identification Control

The following is a function of the servo rotation angle, which mainly changes the duty cycle of the PWM signal

def Servo(servo,angle):
    servo.duty((angle+90)/180*10+2.5)

download attach save to album

2024-10-26 20:36 上传

To detect whether the trash can is full, the distance detection mainly uses the ap3216c sensor on the stm32mp157 development board.

Reading sensor data is to read device file data, which is achieved through file I/O. You can write code in the application layer to read the data under the device .

The following is the code for reading sensor data

QString Ap3216c::readPsData()
{
    char const *filename = "/sys/class/misc/ap3216c/ps";
    int err = 0;
    int fd;
    char buf[10];

    fd = open(filename, O_RDONLY);
    if(fd < 0) {
        close(fd);
        return "open file error!";
    }

    err = read(fd, buf, sizeof(buf));
    if (err < 0) {
        close(fd);
        return "read data error!";
    }
    close(fd);

    QString psValue = buf;
    QStringList list = psValue.split("\n");
    return list[0];
}

4. Image acquisition and transmission (STM32MP157) (UDP)

Get the data from the development board camera and transmit it to the client via UDP

Using UDP transmission process in QT:

Once the server creates a socket, it can directly use the writeDatagram function to send information. The function parameters need to include data, data size, the receiving end's IP, and port number.

(If TCP is used, the server generally needs to create a socket, bind, listen, and accept the client's connection. Here, when transmitting video information, the delay will be smaller when using UDP. Of course, this is just a theory, and I have not tested it.)

The client creates a socket, binds its own IP and port number, and can use the readDatagram function to receive data

Here, I use my own computer as the client to obtain the real-time status of the trash can. (In fact, this function is just imposed by me in order to learn network programming. The function is relatively useless and is mainly for learning)

Server-side code

//摄像头通过调用opencv库获取到的数据类型为mat 需要先转成QImage类型
//QImage类型的图像放入QByteArray中，然后进行base64编码的压缩
//接收端在进行base64解码
/* udp套接字 */
     QUdpSocket udpSocket;

/* QByteArray类型 */
     QByteArray byte;

/* 建立一个用于IO读写的缓冲区 */
     QBuffer buff(&byte);
    
/* image转为byte的类型，再存入buff */
     qImage.save(&buff, "JPEG", -1);

/* 转换为base64Byte类型 */
     QByteArray base64Byte = byte.toBase64();

/* 由udpSocket以单播的形式传输数据，端口号为8888 */
     udpSocket.writeDatagram(base64Byte.data(), base64Byte.size(), QHostAddress("192.168.10.200"), 8888);

Client code

udpSocket = new QUdpSocket(this);
udpSocket->bind(QHostAddress("192.168.10.200"), 8888);
QByteArray datagram;
udpSocket->readDatagram(datagram.data(), datagram.size());

//String-Base64编码转QByteArray
 QByteArray decryptedByte;
decryptedByte = QByteArray::fromBase64(datagram.data());


//比如读入一张BMP格式的文件到QByteArray对象中，再调用该函数，那么该函数就会根据QByteArray中数据进行解析，分析图像的格式等
QImage image;
image.loadFromData(decryptedByte);
videoLabel->setPixmap(QPixmap::fromImage(image));

CC := aarch64-linux-gnu-gcc
SRC := $(shell find src -name "*.c")
INC := ./inc \
./3rd/usr/local/include \
./3rd/usr/include \
./3rd/usr/include/python3.10 \
./3rd/usr/include/aarch64-linux-gnu/python3.10 \
./3rd/usr/include/aarch64-linux-gnu

OBJ := $(subst src/,obj/,$(SRC:.c=.o))

TARGET=obj/garbage

CFLAGS := $(foreach item,$(INC),-I$(item)) #-I./inc -I ./3rd/usr/local/include
LIBS_PATH := ./3rd/usr/local/lib \
./3rd/lib/aarch64-linux-gnu \
./3rd/usr/lib/aarch64-linux-gnu \
./3rd/usr/lib/python3.10

LDFLAGS := $(foreach item,$(LIBS_PATH) ,-L$(item))#-L./3rd/usr/local/lib
LIBS := -lwiringPi -lpython3.10 -pthread -lexpat -lz -lcrypt

obj/%.o:src/%.c
mkdir - p obj
$(CC) -o $@ -c $< $(CFLAGS)

$(TARGET) : $(OBJ)
$(CC) -o $@ $^ $(CFLAGS) $(LDFLAGS) $(LIBS)
scp obj/garbage src/garbage.py orangepi@192.168.0.113:/home/orangepi/garbage

compile: $(TARGET)

clean: rm $(TARGET) obj $(OBJ) -rf

debug:
echo $(CC)
echo $(SRC)
echo $(INC)
echo $(OBJ)
echo $(TARGET)
echo $(CFLAGS)
echo $(LDFLAGS)
echo $(LIBS)

Since the code of this work has been uploaded to the Download Center, I will not waste space to paste the specific code here. Friends who are interested can go to the Download Center to check out the complete and detailed code. Here I only list the file directory structure.

Source code link: https://en.eeworld.com/bbs/my/home.php?cur=myhome&act=download

Project source code description

5. Demonstration video of the work’s functions

6. Project Summary

Use vision to identify the type of garbage, use STM32F4 to control the servo and motor through RTOS to dump the garbage, and send the garbage data and sensor data to MP157, which then uses socket programming to upload the data to the backend server and display it visually on the web page.