BLE ultra-low power voiceprint door lock

xujinxi

BLE ultra-low power voiceprint door lock [Copy link]

 

BLE ultra-low power voiceprint door lock

Author:xujinxi

1. Project Background

The RSL10 Bluetooth module provided by OnSemi is the smallest Bluetooth low-power module in the industry. It also has the best low-power management in the industry, and the development kit has rich sensor peripherals, providing an ideal platform for the realization of various creative ideas.

II. Introduction

The RSL10 Bluetooth module has ultra-low power consumption and an easy-to-use audio interface. Equipped with sensors and related reference driver codes, it can reliably and conveniently realize engineers' creative ideas and is very popular among engineers. The creative design competition organized by EEWORLD+OnSemi provides a stage for engineers to display their talents.

This project is conceived to use the DMIC and IMU units carried by the RSL10 Bluetooth module. It is used for outdoor door knock recognition and voice pickup of smart door locks, and sends it to the indoor receiving end. The indoor RSL10 Bluetooth module receives the voice data from the outdoors and transmits it to the PC through the serial port. The PC uses the voice data to access the voice platform and recognize the voice command. If it is the correct "open the door" command, the "open the door" signal is transmitted through the serial port through the indoor RSL10 Bluetooth module, and then transmitted to the outdoor RSL10 Bluetooth module through BLE, and then the door lock opening signal is issued. The entire low-power voice door lock is realized.

3. System block diagram and function description of each part

The entire system is mainly composed of sensor interfaces such as RSL10, IMU, and DMIC. The following is the electronic system block diagram of the system: The USB port is connected to Jlink, and Jlink establishes a communication path with the PC through USB CDC. The RSL10 outdoor voice acquisition board transmits the collected voice data to the indoor RSL10 module via Bluetooth, and then uploads it to the PC Python code through the serial port of Jlink for data analysis and processing. The PC pushes the data to the Baidu voice recognition engine, and the recognition results are returned to the PC. Finally, it is sent to the indoor and outdoor RSL10 modules for corresponding operations.

4. Source code and sensor data processed in the case study

central_client_uart_on_evb-户内机.7z (106.3 KB, downloads: 6)

2021-7-28 09:29 上传

点击文件名下载附件

peripheral_server_uart_on_sensor-户外机.7z (491.09 KB, downloads: 6)

2021-7-28 09:29 上传

点击文件名下载附件

RSL10 BLE超低功耗声纹门锁.py (3.95 KB, downloads: 5)

2021-7-28 09:30 上传

点击文件名下载附件

/*------------------------------------------------ ----------------------------
* Copyright (c) 2015-2017 Semiconductor Components Industries, LLC (d/b/a
* ON Semiconductor ), All Rights Reserved
*
* Copyright (C) RivieraWaves 2009-2016
*
* This module is derived in part from example code provided by RivieraWaves
* and as such the underlying code is the property of RivieraWaves [a member
* of the CEVA, Inc . group of companies], together with additional code which
* is the property of ON Semiconductor. The code (in whole or any part) may not
* be redistributed in any form without prior written permission from
* ON Semiconductor.
*
* The terms of use and warranty for this code are covered by contractual
* agreements between ON Semiconductor and the licensee.
*
* This is Reusable Code.
*
* ----------------------- -------------------------------------------------- ----
* app.c
* - Main application file
* ------------------------------------ ------------------------------------------
* $Revision: 1.25 $
* $Date: 2017/12/05 16:02:54 $
* -------------------------------- -------------------------------------------------- */

//20210626 add dmic support by hhz
#include "app.h"
#include <printf.h>

#define DMIC_DATA_WIDTH_16 //16bit DMIC data-- DMIC_DATA_WIDTH_8 8bit DMIC data

#ifdef DMIC_DATA_WIDTH_8 //FS = 16000hz,data width = int8_t,2 second record data =16000*1 =16000 samples
#define DMIC_BUF_SIZE 16000 //100 sample, equal 200 bytes data(int16_t)
#else
#define DMIC_BUF_SIZE 16000 //FS = 16000hz,data width = int16_t,2 second record data =16000*2 =32000 s
#endif

#define BLE_TX_PACKET_SIZE 100 //200

uint32_t output_flag = 0;
uint32_t data_ready_flag = 0; //init not data ready until interruption receive DMIC_BUF_SIZE data from DMIC

int16_t dmic_value_scope_int16 = 0;

#ifdef DMIC_DATA_WIDTH_8
int8_t dmic_value_buf[DMIC_BUF_SIZE] = {0}; //d0_msb,d0_lsb,d1_msb,d1_lsb...
#else
int16_t dmic_value_buf[DMIC_BUF_SIZE] = {0}; //d0_msb,d0_lsb,d1_msb,d1_ lsb...
# endif

//dmic_value_buf[0] = lsb, dmic_value_buf[1] = msb
uint8_t* dmic_buf_pointer_uint8 = (uint8_t *)dmic_value_buf;

uint32_t j = 0, k = 0;

uint32_t dmic_packet_num = 0;
uint32_t tx_cnt = 0;
uint32_t record_sample_num = 0;
//+ lsb_num,~,~,msb_num packet num uint32 lsb fist
uint8_t* pointer_record_sample_num_uint8 = (uint8_t *)(&record_sample_num);

uint8_t syn_head_array[4] = {0x55,0x55,0x55,0x55}; //+ lsb_num,~,~,msb_num packet num uint32 lsb fist
uint8_t syn_end_array[4] = {0xaa,0xaa,0xaa,0xaa}; //+ lsb_num,~,~,msb_num packet num uint32 lsb fist
uint8_t head_sended_flag = 0;
uint8_t end_sended_flag = 0;

static uint8_t asr_flag = 0;

void DMIC_OUT_OD_IN_IRQHandler(void)
{
static int32_t i = 0;

//dmic_value_scope_int32 = (int32_t)AUDIO->DMIC0_DATA;
//dmic_value_scope_int16 = (int16_t)((dmic_value_scope_int32 >> 1) & 0xffff);
//dmic_value_scope_int16 = (int16_t)((dmic_value_scope_int32) & 0xffff);
//dmic_value_scope_ int16 = ( int16_t)(dmic_value_scope_int32);
//PRINTF("%d = %08x = %d\n",dmic_value_scope_int32,dmic_value_scope_int32,dmic_value_scope_int16);

//dmic_value_scope_int16 = (int16_t)AUDIO->DMIC0_DATA;
dmic_value_scope_int16 = (int16_t)((uint16_t)(AUDIO->DMIC_DATA & 0x0000ffff));
Sys_GPIO_Toggle(3); //test frame frequency

//dmic_value_scope_int16 = dmic_packet_num;
//PRINTF("%d = %08x\n",dmic_value_scope_int16,dmic_value_scope_int16);

if(dmic_packet_num < DMIC_BUF_SIZE)
{
#ifdef DMIC_DATA_WIDTH_8 //FS = 16000hz,data width = int8_t,2 second record data =16000*1 =16000 samples
dmic_value_buf[dmic_packet_num] = (int8_t)dmic_value_scope_int16;
#else
dmic_value_buf[d mic_packet_num] = dmic_value_scope_int16 ;
//dmic_value_buf[dmic_packet_num] = dmic_packet_num;
#endif

dmic_packet_num++;
}
else
{
data_ready_flag = 1;
//asr_flag = 2;
//NVIC_DisableIRQ(DMIC_OUT_OD_IN_IRQn); //close DMIC interrupt
}

}

void DIO0_IRQHandler(void)
{
//static uint8_t ignore_next_dio_int = 1;
PRINTF("DIO0 INTERRUPT\n");
asr_flag ++;

if(asr_flag == 2)
NVIC_EnableIRQ(DMIC_OUT_OD_IN_IRQn);
}

int main(void)
{
uint32_t length;
uint8_t temp[BUFFER_SIZE]; //for uart

static uint8_t time_cnt = 0;

/* Initialize the system */
App_Initialize();

//setting dio3 as dmic interrupt toggle output for sample rate conform
Sys_DIO_Config(3, DIO_MODE_GPIO_OUT_0); //CLOCK OUTPUT
//Sys_DIO_Config(3, DIO_MODE_AUDIOCLK);

//Sys_DIO_Config(LED_DIO_GREEN, DIO_MODE_GPIO_OUT_0); //GREEN LED
Sys_GPIO_Toggle(LED_DIO_GREEN);
Sys_GPIO_Set_High(LED_DIO_GREEN);

NVIC_EnableIRQ(DIO0_IRQn);

PRINTF("HARDWARE INIT OK !\n"); //hhz
PRINTF("into main loop !\n"); //hhz

/* Main application loop:
* - Run the kernel scheduler
* - Send notifications for the battery voltage and RSSI values
* - Refresh the watchdog and wait for an interrupt before continuing */
while (1)
{
Kernel_Schedule();

/*if(asr_flag == 2)
{
NVIC_EnableIRQ(DMIC_OUT_OD_IN_IRQn);
//PRINTF("NVIC_EnableIRQ\n");
}
else if(asr_flag == 0)
{
NVIC_DisableIRQ(DMIC_OUT_OD_IN_IRQn);
//PRINTF("NVIC_DisableIRQ\n" );
}*/

if (unhandled_packets != NULL)
{
if (UART_FillTXBuffer(unhandled_packets->length,
unhandled_packets->data) !=
UART_ERRNO_OVERFLOW)
{
unhandled_packets = removeNode(unhandled_packets);
}
}

if (ble_env.state == APPM_CONNECTED)
{
if (app_env.send_batt_ntf && bass_support_env.enable)
{
app_env.send_batt_ntf = 0;
Batt_LevelUpdateSend(0, app_env.batt_lvl, 0);
}

if (cs_env.sentSuccess)
{
// Copy data from the UART RX buffer to the TX buffer
length = UART_EmptyRXBuffer(temp);
if (length > 0)
{
// Split buffer into two packets when it's greater than
// packet size
if (length > PACKET_SIZE)
{
CustomService_SendNotification(ble_env.conidx,
CS_IDX_TX_VALUE_VAL,
temp,
PACKET_SIZE);
CustomService_SendNotification(ble_env.conidx,
CS_IDX_TX_VALUE_VAL,
&temp[PACKET_SIZE],
length - PACKET_SIZE);
}
else
{
CustomService_SendNotification(ble_ env.conidx,
CS_IDX_TX_VALUE_VAL,
temp,
length);
}
}
}

//test dmic first 0~1000 sample data
/*
if((data_ready_flag == 1) & (tx_cnt < DMIC_BUF_SIZE))
{
PRINTF("%d\n",dmic_value_buf[tx_cnt]);
tx_cnt++;
}
*/

//if (cs_env.sentSuccess)
if (cs_env.sentSuccess & (data_ready_flag == 1))
{
head_sended_flag = 1; //without head test
end_sended_flag = 1; //without end test

if(head_sended_flag == 0)
{
PRINTF("TX:syn_head_array !\n"); //hhz
CustomService_SendNotification(ble_env.conidx,
CS_IDX_TX_VALUE_VAL,
syn_head_array,
4);
//CustomService_SendNotification(ble_env.conidx,
// CS_IDX_TX_VALUE_VAL,
// pointer_record_sample_num_uint8,
// 4);
head_sended_flag = 1;
}

#ifdef DMIC_DATA_WIDTH_8
for(tx_cnt=0;tx_cnt < DMIC_BUF_SIZE/BLE_TX_PACKET_SIZE;tx_cnt++)
{
PRINTF("TX:%d ' DMIC packet !\n",tx_cnt);
CustomService_SendNotification(ble_env.conidx,
CS_IDX_TX_VALUE_VAL,
dmic_buf _pointer_uint8,
BLE_TX_PACKET_SIZE);
dmic_buf_pointer_uint8 += BLE_TX_PACKET_SIZE;

PRINTF("TX:tx dmic watchdog refresh !\n");
Sys_Watchdog_Refresh();
}
#else
if((data_ready_flag == 1) & (tx_cnt < DMIC_BUF_SIZE*2/BLE_TX_PACKET_SIZE))
{
//PRINTF("%d\ n",dmic_value_buf[tx_cnt]);
PRINTF("TX:%d ' DMIC packet !\n",tx_cnt);

//CustomService_SendNotification(ble_env.conidx,
// CS_IDX_TX_VALUE_VAL,
// syn_head_array,
// 2);

CustomService_SendNotification(ble_env.conidx,
CS_IDX_TX_VALUE_VAL,
dmic_buf_pointer_uint8,
BLE_TX_PACKET_SIZE);

dmic_buf_pointer_uint8 += BLE_TX_PACKET_SIZE;

//CustomService_SendNotification(ble_env.conidx,
// CS_IDX_TX_VALUE_VAL,
// syn_end_array,
// 2);

tx_cnt++;
}
else if((data_ready_flag == 1) & (tx_cnt >= DMIC_BUF_SIZE*2/BLE_TX_PACKET_SIZE))
{
data_ready_flag = 0;

//without head test
//head_sended_flag = 0;
//end_sended_flag = 0;
dmic_buf_pointer_uint8 = (uint8_t *)dmic_value_buf;
dmic_packet_num = 0;
tx_cnt = 0;
asr_flag = 0;
NVIC_DisableIRQ(DMIC_OUT_OD_IN_IRQn);
PRINTF("close od interrupt,in it all flag\n");
}
/*for(tx_cnt=0;tx_cnt < DMIC_BUF_SIZE*2/BLE_TX_PACKET_SIZE;tx_cnt++)
{
PRINTF("TX:%d ' DMIC packet !\n",tx_cnt);
CustomService_SendNotification(ble_env.conidx ,
CS_IDX_TX_VALUE_VAL,
dmic_buf_pointer_uint8,
BLE_TX_PACKET_SIZE);
dmic_buf_pointer_uint8 += BLE_TX_PACKET_SIZE;

PRINTF("TX:tx dmic watchdog refresh !\n");
Sys_Watchdog_Refresh();
}*/

#endif
if((end_sended_flag == 0) & (data_ready_flag == 0)) //all dmic data have been sent over BLE
{
PRINTF("TX:syn_end_array !\n"); //hhz
CustomService_SendNotification(ble_env.conidx,
CS_IDX_TX_VALUE_VAL,
syn_end_array,
4);
//end_sended_flag = 1;

head_sended_flag = 0;
end_sended_flag = 0;

dmic_buf_pointer_uint8 = (uint8_t *)dmic_value_buf;
dmic_packet_num = 0;
tx_cnt = 0;
asr_flag = 0;
PRINTF("close od interrupt,init all flag\n");
}

//PRINTF("TX dmic_value_buf\n");

//for(tx_cnt=0;tx_cnt < DMIC_BUF_SIZE;tx_cnt++)
//{
// PRINTF("%d\n",dmic_value_buf[tx_cnt]);
// Sys_Watchdog_Refresh();
//}

data_ready_flag = 0;
}

//Sys_Watchdog_Refresh();

} //connected

Sys_GPIO_Toggle(3);

/* Refresh the watchdog timer */
Sys_Watchdog_Refresh();

/* Wait for an event before executing the scheduler again */
SYS_WAIT_FOR_EVENT;

} //while(1)
} //main

5. Video Demonstration

1. Power on the indoor unit RSL10 module;

2. Power on the outdoor RSL10 sensor board;

3. Start the PC processing program and connect the indoor RSL10 module;

4. Press the button to start recording the outdoor RSL10 sensor module. The recorded data is transmitted to the indoor RSL10 module via BLE. The RSL10 module is transferred to the PC. The PC is recognized by Baidu speech recognition engine. The recognition result is returned to the PC and the recognition result is demonstrated.

The demonstration video link is as follows:

Station B : https://www.bilibili.com/video/BV1Ff4y1j7KW?share_source=copy_web

Youku: https://v.youku.com/v_show/id_XNTE4MzEzNDMwNA==.html

6. Project Summary

The RSL10 development board is loaded with multiple sensors: such as IMU, DMIC, temperature, humidity and other sensors. In simple or complex environments, multiple data can be collected for data analysis at the same time to find out the data relationship between the environment and the sensor, so as to facilitate electronic equipment to identify the environment or action and realize the perception of the electronic system to the environment and action. The relevant code routines and sensor hardware DEMO make it convenient for engineers to integrate sensors into the original system, thereby reducing time waste and quickly evaluating sensors. Thumbs up !

This project only completed the DMIC data collection of the outdoor RSL10 module, the BLE transmission of the DMIC data to the indoor RSL10 module, the RSL10 transmitted the audio data to the PC through the serial port, and completed the voice recognition through the Baidu voice recognition engine, and output the recognition results. The remaining functions are not completed. In addition, the DMIC data and transmission reliability need to be optimized, and the success rate is low.

w494143467

The first one to use the audio function!

cruelfox

What is the audio sampling rate? 8-bit or 16-bit?

It seems that BLE needs to transmit audio in real time, and the bandwidth is very tight.

xujinxi

cruelfox posted on 2021-7-19 20:58 What is the voice sampling rate? 8-bit or 16-bit? It seems that BLE needs to transmit audio in real time, and the bandwidth is very tight.

I didn't do real-time, I used 16bit 16khz configuration. In addition, my Python was not used well, the reception was unstable, and it would be even more difficult to do real-time.

客户专用

xujinxi posted on 2021-7-19 21:29 I didn't do real-time, I used 16bit 16khz configuration. In addition, my Python is not used well, the reception is unstable, and it is even more difficult to do real-time.

BLE voice real-time transmission, RSL10 has a set of remote mic samples that you can check out. The delay is as low as 40ms. The sampling rate is 16kHz and uses G722 encoding

xujinxi

Customer-only published on 2021-7-27 11:41 BLE voice real-time transmission, RSL10 has a set of remote mic samples that you can learn about. The latency is as low as 40ms. The sampling rate is 16kHz and uses G722 encoding...

That's good, I'll try it out on the weekend

jhearing

Hello, my DMIC has no data, I don't know much about hardware, I paid for help, what I want is DMIC in, OD out, DMA method