Wireless and Cloud Connectivity Technology Topic 2: Implementing BLE Full-Duplex Voice Streaming on STM32WB Using Opus
Latest update time：2022-04-15 11:53
Reads：
☝Click
on
"STMicroelectronics China"
above
to
follow us
In the field of wireless Bluetooth, the market has always been in demand for both ultra-low power consumption and full-duplex voice communication, such as wireless doorbells, short-range intercoms, etc. For BLE voice-related application scenarios, ST has launched the FP-AUD-BVLINKWB1 software package for customer evaluation.
This article will introduce this software package, which mainly includes the following features:
-
Full-duplex stereo audio streaming over BLE using Opus v1.3 encoding and decoding.
-
Use the custom BlueVoiceOPUS protocol to use Opus functions through the API.
-
The source code includes digital audio capture and processing.
-
Audio streaming can be played back via USB.
-
You can use the combination of the microphone expansion board X-NUCLEO-CCA02M2 + the Nucleo development board P-NUCLEO-WB55 or the discovery board STM32WB5MM-DK.
-
Compatible with STBLESensor App.
-
Free and open source.
If you are interested, please download the software package for evaluation.
Figure
1. Software architecture diagram
-
Hardware Abstraction: Hardware abstraction layer, using the STM32 HAL library to provide the underlying hardware driver.
-
BSP layer: Above the HAL layer, a board support package is provided, including drivers such as SPI, ADC, LED and User Button.
-
Middleware layer: The middleware layer mainly includes the following contents:
□
STM32 WPAN: Provides interface APIs related to protocols such as BLE, Thread, and Zigbee. This application only uses the BLE protocol.
□
USB Device: Provides implementations of various USB device classes. In this application, only the USB Audio class is used.
□
PDM Lib: Provides software implementation for converting digital microphone PDM signals to PCM signals.
□
OPUS: Open source third-party OPUS library, version v1.3.
Opus is a completely open source and free, highly versatile audio codec protocol standardized by the Internet Engineering Task Force (IETF).
Opus can handle a wide range of audio applications, including voice over IP, video conferencing, in-game chat, and even remote live music performances. It can scale from low bitrate narrowband speech to very high quality stereo music, and has the following features:
-
Bit rate range: 6 kb/s to 510 kb/s
-
Sampling rate range: 8kHz (narrowband) to 48kHz (fullband)
-
Frame size: 2.5 ms to 60 ms
-
Bit rate, sampling rate, frame size dynamically adjustable
-
Supports constant bit rate (CBR) and variable bit rate (VBR)
-
Support voice and music
-
Supports mono and stereo
-
Supports up to 255 channels
-
Good robustness and packet loss concealment capabilities
-
Supports floating point and fixed point
BlueVoiceOPUS is a custom protocol. Central and Peripheral are roles in the GAP layer. In point-to-point communication, the device that actively initiates the connection request is the Central. In the GATT protocol, the Server accepts requests and commands and saves the data in the attributes. The Client implements the service discovery process and requests data from the Server.
If it is a one-way asymmetric voice system, the device with a microphone can be defined as the server, and the client can actively or passively receive the voice data stream of the server.
Figure
2. GATT role allocation
As shown in Figure 2, FP-AUD-BVLINKWB1 provides a two-way system. Both devices have voice capabilities, so both sides implement the roles of GATT Server and Client. In addition, based on the notification nature, the voice data of the Server will be actively sent to the Client in an asynchronous manner.
✦
BlueVoiceOPUS Service
The ATT protocol is used to exchange data between devices. The smallest entity of ATT is called an attribute. GATT services consist of various services, where services start with service declaration attributes. Each service contains several characteristics, which are composed of any attributes or attribute descriptors.
Table 1 is a typical BLE service attribute table, which includes 1 service declaration and 3 characteristics. The Audio characteristic is used to notify the other device of audio data. The Ctrl characteristic notifies the other device of control data, such as play, pause, etc. The Music characteristic is used to notify the other device of music data. This characteristic is only implemented in the ST BLE Sensor APP, replacing the Audio characteristic, and is used to transmit compressed 48KHz stereo music.
▲
Table 1. BlueVoiceOPUS service attribute table
✦
BLUEVoiceOPUS implementation
In the Middleware layer, the files related to the BlueVoiceOPUS protocol include:
-
bvopus_service_stm: This file manages all BLE-related functions, including adding services and features, as well as receiving and sending data. It contains a simple data packet and parsing protocol.
-
opus_interface_stm: This file implements the interface between Opus encoder and BlueVoiceOPUS service. It provides simple API for Opus initialization, configuration, data compression and decompression, etc.
Application Layer Introduction
There are three main projects included in the software package:
-
BVLCentral: As the master device, it actively initiates the connection and provides BlueVoiceOPUS services.
-
BVLPeripheral: broadcasts as a slave device and provides BlueVoiceOPUS services.
-
BVLPeripheral_FullBand: Broadcasts as a slave device, provides BlueVoiceOPUS service, can receive stereo music via BLE, but currently can only be connected to ST BLE Sensor.
BVLCentral and BVLPeripheral are burned into the WB55 development board as the host and slave respectively. By starting or stopping the audio notification, three different types of communication can be achieved: simplex, half-duplex and full-duplex.
-
When the device is outputting an audio stream, the application layer is responsible for voice acquisition, data compression and packaging, and then sending it out through the BlueVoiceOPUS protocol.
-
When the device is receiving an audio stream, the application layer receives BLE packets from the BlueVoiceOPUS protocol, and then unpacks and decodes the OPUS voice data.
The voice streaming channel can be turned on or off by controlling SW1 on the P-NUCLEO-WB55.
The device status is displayed via LEDs.
-
Broadcast/discovery status: Green LED flashing
-
Connection status: Blue LED flashes slowly
-
Voice streaming status: blue LED flashes normally
-
Receiving status: Blue LED is steady on (not flashing)
-
Full-duplex status: Blue LED flashes quickly (both devices)
BVLCentral can be replaced by APP (ST BLE Sensor) to complete full-duplex voice communication between the device and the mobile phone.
▲
Figure 3. Application flow chart
The entire application process is shown in Figure 3. The entire process is described as follows:
-
The slave device broadcasts and the master device initiates a connection until the connection is successfully established.
-
Both parties complete the service and feature discovery process with each other.
-
The slave device requests to turn on the feature notification by pressing a button, the master device turns on the notification, and the slave device sends a voice stream. At this time, the state is the voice stream state.
-
On the contrary, the master device requests to turn on the characteristic notification by pressing the button, the slave device turns on the notification, and the master device sends the voice stream, and the state becomes full-duplex.
-
You can use buttons to turn on or off the voice stream and change the voice stream status.
The following will introduce the Demo deployment process from different aspects.
Full-duplex audio streaming between two STM32WB development boards
▲
Figure 4. Combination of NUCLEO+CCA02M2
As shown in Figure 4 above, you can use the P-NUCLEO-WB55 development board and a microphone expansion board X-NUCLEO-CCA02M2 to combine. Then burn the BVLCentral project and the BVLPeripheral project separately. After the two devices are connected, a half-duplex or full-duplex voice stream can be established through the SW1 button on the Nucleo. The microphone expansion board can collect voice signals at 8kHz or 16kHz and transmit them through the BlueVoiceOPUS protocol. When the other device receives the voice data, it is also transmitted to the PC via USB, and burned and saved using burning software (such as audacity),
and then played using Audacity or other voice software.
Figure
5. STM325MM-DK Discovery Board
As shown in Figure 5 above, you can also use the STM32WB5MM-DK to perform the same operation. This development board uses the STM32WB5MM module and has an OLED display on board for development.
Of course, you can also use the combination of Figure 4 and Figure 5, as long as one block is burned with the host program and the other block is burned with the slave program.
Full-duplex audio streaming between STM32WB development board and mobile phone
In addition to the above configuration method, ST also provides the ST BLE Sensor mobile APP for testing.
Figure
6. Audio streaming between the phone and the development board
As shown in Figure 6, the mobile phone can act as the master device, scan and connect to the STM32WB55 development board, and establish a full-duplex audio stream BLE link. Voice signals can be collected from the development board and sent to the mobile phone for playback. They can also be collected from the mobile phone, sent to the development board, and then played through USB.
Stereo music playback between STM32WB development board and mobile phone
The above introductions are all 8kHz/16kHz voice streams. The software package also provides a full-band 48kHz stereo music stream solution.
▲
Figure 7.48kHz stereo music streaming solution
As shown in Figure 7, after the phone scans the device, it initiates and establishes a connection, completes the GATT service discovery process, and then opens the notification attribute of the characteristic through the button to establish a full-band music stream. The APP uses OPUS to compress and package the music in the phone, and then sends it to the development board via BLE. The development board exports the data to the PC via USB, and finally uses Audacity or other voice software to play it.
Long press the QR code to follow and learn more information
Professional | Technology | Style
▲
Long press to follow STMicroelectronics
Semiconductor
China
京公网安备 11010802033920号