[ESP32-Audio-Kit Audio Development Board Review] 4. MIC Input Exploration + Help

wo4fisher

[ESP32-Audio-Kit Audio Development Board Review] 4. MIC Input Exploration + Help [Copy link]

After completing the "play-mp3-control" routine in the previous article, I thought the rest of the journey would be much smoother, but it wasn't. So here is a post for help.

There are not many experiences shared about ESP_ADF framework on the Internet, and most of them are just steps, without in-depth analysis of the framework. The few articles mainly involve two contents, one is the use of dueros, and the other is ESP32 voice recognition, both of which require the use of mic input.

1. Dueros operation

1.1 First copy the files in the esp-adf\examples\dueros directory to a new folder.

1.2 Then set the ssid and psw of WiFi in menuconfig.

There are two ways to enter menuconfig in vscode: 1) ctrl+shift+p; 2) the settings icon on the bottom left of vscode after installing the expressif plugin.

1.3 Register an account on Baidu Dueros Open Platform and authenticate as a developer. After authentication, apply for a profile and download it. After downloading, unzip one of the files to replace the duer_profile file in esp-adf\components\dueros_service, and change the new file name to duer_profile.

1.4 Comment out the content related to touch buttons. Touch buttons take up more CPU time and will affect the speech recognition rate.

For details, please refer to the link below [Anxinke ESP32 Voice Development Board Special Topic ②] ESP32-Audio-Kit development board is adapted to Baidu dueros routines to realize online voice function and chat with the development board.

1.4 build

1.5 load and run

1.5 Problem

According to the official readme file

## Supported Features

- Say "Hi Lexin" to trigger the DuerOS voice interaction, green LED will turn on for indicate wakeup, if the `DingDing.wav` exist in microSD, "Dingding" should be heard at same time.
- Press [Rec] button to talk to DuerOS. The device will play back the DuerOS response. You can say anything you want, e.g."今天天气怎么样?" or "现在几点了?", which means "What's the weather today?" or " what time is it?".
- The green LED indicates Wi-Fi status:
    - Green LED on if Wi-Fi connected
    - Green LED blinks normally if Wi-Fi disconnected
    - Green LED fast blinks if Wi-Fi is under setting
- Adjust volume via [Vol-] or [Vol+]
- Configure Wi-Fi by [Set] button

It should have the above effect. In fact, when using it, the board is powered on and initialized, and the wifi connection is successful.

I (5245,tid:3ffdfd40) lightduer_engine.c( 242):duer_engine_start, g_handler:3F80B7B0, length:1469, profile:3F80D8C0
W (6724) DUEROS: duer_callback: type:0, source:0x3f804eb4 data:2, data_len:0
I (5255,tid:3ffdfd40) lightduer_ca_conf.c(  38):    duer_conf_get_string: uuid = 21880000000001
I (5270,tid:3ffdfd40) lightduer_ca_conf.c(  38):    duer_conf_get_string: serverAddr = device.iot.baidu.com
I (5327,tid:3ffdfd40) baidu_ca_socket_adp.c( 136):DNS lookup succeeded. IP=163.177.151.84
I (5385,tid:3ffdfd40) lightduer_connagent.c( 206):will start latter(DUER_ERR_TRANS_WOULD_BLOCK)
W (5385,tid:3ffdfd40) lightduer_events.c(  80):[lightduer_ca] <== event end = 4012A740, timespent = 140
I (6512,tid:3ffdfd40) lightduer_connagent.c( 206):will start latter(DUER_ERR_TRANS_WOULD_BLOCK)
W (6513,tid:3ffdfd40) lightduer_events.c(  80):[lightduer_ca] <== event end = 4012A8F8, timespent = 1031
I (6520,tid:3ffdfd40) lightduer_connagent.c( 206):will start latter(DUER_ERR_TRANS_WOULD_BLOCK)
I (6529,tid:3ffdfd40) lightduer_connagent.c( 206):will start latter(DUER_ERR_TRANS_WOULD_BLOCK)
I (6539,tid:3ffdfd40) lightduer_connagent.c( 206):will start latter(DUER_ERR_TRANS_WOULD_BLOCK)
I (6546,tid:3ffdfd40) lightduer_connagent.c( 206):will start latter(DUER_ERR_TRANS_WOULD_BLOCK)
I (6554,tid:3ffdfd40) lightduer_connagent.c( 206):will start latter(DUER_ERR_TRANS_WOULD_BLOCK)
I (6562,tid:3ffdfd40) lightduer_connagent.c( 206):will start latter(DUER_ERR_TRANS_WOULD_BLOCK)
I (6571,tid:3ffdfd40) lightduer_connagent.c( 206):will start latter(DUER_ERR_TRANS_WOULD_BLOCK)
I (6578,tid:3ffdfd40) lightduer_engine.c( 242):duer_engine_start, g_handler:3F80B7B0, length:0, profile:00000000
I (6589,tid:3ffdfd40) lightduer_connagent.c( 206):will start latter(DUER_ERR_TRANS_WOULD_BLOCK)
I (6659,tid:3ffdfd40) lightduer_connagent.c( 206):will start latter(DUER_ERR_TRANS_WOULD_BLOCK)
I (6663,tid:3ffdfd40) lightduer_connagent.c( 206):will start latter(DUER_ERR_TRANS_WOULD_BLOCK)
I (6666,tid:3ffdfd40) lightduer_connagent.c( 206):will start latter(DUER_ERR_TRANS_WOULD_BLOCK)
I (6675,tid:3ffdfd40) lightduer_connagent.c( 206):will start latter(DUER_ERR_TRANS_WOULD_BLOCK)
I (6683,tid:3ffdfd40) lightduer_connagent.c( 206):will start latter(DUER_ERR_TRANS_WOULD_BLOCK)
I (6692,tid:3ffdfd40) lightduer_connagent.c( 206):will start latter(DUER_ERR_TRANS_WOULD_BLOCK)
I (6962,tid:3ffdfd40) lightduer_voice.c( 716):Mutex initializing
I (6962,tid:3ffdfd40) lightduer_connagent.c( 187):connect started!
I (6964,tid:3ffdfd40) lightduer_ds_log_cache.c(  85):no cache report
E (8448) DUEROS: event: 0
I (8451) DUEROS: duer_dcs_init
I (6977,tid:3ffdfd40) lightduer_dcs_router.c( 305):namespace: ai.dueros.private.protocol
I (6984,tid:3ffdfd40) lightduer_dcs_router.c( 305):namespace: ai.dueros.device_interface.screen_extended_card
I (6993,tid:3ffdfd40) lightduer_dcs_router.c( 305):namespace: ai.dueros.device_interface.screen
I (7002,tid:3ffdfd40) lightduer_dcs_router.c( 305):namespace: ai.dueros.device_interface.system
I (7011,tid:3ffdfd40) lightduer_dcs_router.c( 305):namespace: ai.dueros.device_interface.voice_input
I (7019,tid:3ffdfd40) lightduer_dcs_router.c( 305):namespace: ai.dueros.device_interface.voice_output
I (7028,tid:3ffdfd40) lightduer_dcs_router.c( 305):namespace: ai.dueros.device_interface.speaker_controller
I (7038,tid:3ffdfd40) lightduer_dcs_router.c( 305):namespace: ai.dueros.device_interface.audio_player
I (8528) AUDIO_WRAPPER: duer_dcs_get_speaker_state
I (8535) DUEROS: Dueros DUER_CMD_CONNECTED, duer_state:2
E (8535) DUEROS: event: DUER_EVENT_STARTED
W (8537) DUEROS: duer_callback: type:0, source:0x3f804eb4 data:3, data_len:0
W (7063,tid:3ffdfd40) lightduer_events.c(  80):[lightduer_ca] <== event end = 4012A8F8, timespent = 106
I (7080,tid:3ffdfd40) lightduer_connagent.c( 220):add resource successfully!!
I (7087,tid:3ffdfd40) lightduer_connagent.c( 220):add resource successfully!!
W (7093,tid:3ffdfd40) lightduer_engine.c( 697):data cache has not sent, pending..., dcache_len:2, qcache_len:3
W (7106,tid:3ffdfd40) lightduer_system_info.c( 306):Undefined memory type, 0
E (7110,tid:3ffdfd40) lightduer_system_info.c( 389):Sys Info: Get disk info failed
W (7123,tid:3ffdfd40) lightduer_engine.c( 697):data cache has not sent, pending..., dcache_len:2, qcache_len:4
W (7134,tid:3ffdfd40) lightduer_engine.c( 697):data cache has not sent, pending..., dcache_len:2, qcache_len:3
W (7140,tid:3ffdfd40) lightduer_engine.c( 697):data cache has not sent, pending..., dcache_len:2, qcache_len:2
W (7150,tid:3ffdfd40) lightduer_engine.c( 697):data cache has not sent, pending..., dcache_len:2, qcache_len:1
I (7383,tid:3ffdfd40) lightduer_dcs_router.c( 459):Directive name: ThrowException
E (7383,tid:3ffdfd40) lightduer_dcs_system.c( 314):error code: INVALID_REQUEST_EXCEPTION
E (7387,tid:3ffdfd40) lightduer_dcs_system.c( 315):error: didp product not exsists
I (7401,tid:3ffdfd40) lightduer_dcs_router.c( 459):Directive name: ThrowException
E (7403,tid:3ffdfd40) lightduer_dcs_system.c( 314):error code: INVALID_REQUEST_EXCEPTION
E (7410,tid:3ffdfd40) lightduer_dcs_system.c( 315):error: didp product not exsists
I (7422,tid:3ffdfd40) lightduer_dcs_router.c( 459):Directive name: DialogueFinished
I (7435,tid:3ffdfd40) lightduer_dcs_router.c( 459):Directive name: DialogueFinished
W (7449,tid:3ffdfd40) lightduer_engine.c( 697):data cache has not sent, pending..., dcache_len:2, qcache_len:2
W (7454,tid:3ffdfd40) lightduer_engine.c( 697):data cache has not sent, pending..., dcache_len:2, qcache_len:1

The log shows some error information.

It is known that it can play audio files in sdcard and connect to wifi correctly, but further research is needed. @安信可, @乐鑫

2 esp-adf\examples\speech_recognition\asr

This example is a speech recognition example. In order to test whether the problem is with the mic input, the ASR example is further tested. This example mainly uses the keyword "Hi, Espressif" as the wake-up word, and there are some preset command voices.

After menuconfig and build, there is still no response when downloading to the board.

3 esp-adf\examples\recorder\pipeline_recording_to_sdcard

The pipeline_recording_to_sdcard routine was further tested. After running, this routine records for 10 seconds by default, and then saves the recording file in the sdcard with the file name rec.wav.

The picture below is the audio file after recording. It is smooth and without any waves.

At this point, it can be concluded that the problem is with the mic voice input.

annysky2012

Anyone who knows, please answer

wo4fisher

annysky2012 posted on 2021-9-28 16:30 Anyone knows, please give me an answer

However, there is no

w494143467

Keep an eye out, maybe you can see a solution in the future.

mr金时

I'm also stuck, has the OP solved it?

wo4fisher

mr Jinshi published on 2021-11-22 23:33 I also fell into the trap, has the OP solved it?

No, it's busy at the end of the year.