[RVB2601 Creative Application Development] Part 4 Sound Histogram

oxygen_sh

[RVB2601 Creative Application Development] Part 4 Sound Histogram [Copy link]

 

The content of this evaluation experiment is to combine sound playback and display. While playing the sound, the screen also displays a histogram of the sound intensity.

First the video, then I'll explain how to do it.

video

4-1

This experimental code is developed based on the Offline version of the musicplay routine.

This routine plays the mp3 sound data (saved in array form) embedded in the program.

For example, the welcome_mp3 sound data used in the experiment is saved in include/welcome_mp3.h .

The data in the

array is arranged completely according to the mp3 file format. For example, the first four bytes represent the frame header of the sound data segment. Interested friends can study the mp3 file format by themselves, which is very helpful for understanding this sound playback demo .

MP3 sound data is compressed data and needs to be decoded into PCM before it can be played. The MP3 data decoding in the demo is implemented using the pvmp3dec open source MP3 decoding library.

The decoded data is sent to the AV_AO_ALSA PCM output driver for audio output.

After spending more than N hours of research, I finally located the location of the audio decoded data.

In the _ptask() task function in the Player.c file , ad_decode() is called to decode the mp3 data. The decoded data is stored in the dframe array.

rc = ad_decode(ad, dframe, &got_frame, &pkt);

Next, the audio data is processed to extract the intensity of the sound. The following processing is simply averaging the PCM amplitude values.

if(dframe->linesize[0] > 1024){

tempmaxvalue = 0;

for(uint8_t j=0;j<16;j++){

blocknumtemp[j] = 0;

temptotal = 0;

for(uint8_t i=0;i<32;i++){

temp = *(dframe->data[0]+1+j*64+i*2);

temp = temp << 8;

temp += *(dframe->data[0]+j*64+i*2);

temptotal += temp;

}

blocknumtemp[j] = temptotal;

if(blocknumtemp[j] > tempmaxvalue){

tempmaxvalue = blocknumtemp[j];

}

}

if(tempmaxvalue >0){

for(uint8_t j=0;j<16;j++){

blocknum[j] = (blocknumtemp[j] * 8)/tempmaxvalue;

}

}

}

The following is a histogram display of the data representing the sound intensity on the screen.

void draw_block(uint8_t x, uint8_t y)

{

for(uint8_t i = x; i >= x-4; i--){

for(uint8_t j=y; j<= y+7; j++){

buf[j] = 1;

}

}

for(uint8_t j=y; j<= y+7; j++){

buf[x-4][j] = 0;

}

for(uint8_t i = x; i >= x-3; i--){

buf[y] = 0;

buf[y+7] = 0;

}

}

void draw_blocks()

{

uint8_t block_row, block_col, blockrownum,x,y;

for(uint8_t i=0;i<64;i++){

for(uint8_t j=0;j<128;j++){

buf[j] = 0;

}

}

for(block_col=1;block_col<= BLOCKCOLNUM;block_col++){

for(block_row=1;block_row<= blocknum[block_col-1];block_row++){

x = 63 - (block_row -1)*5;

y = (block_col - 1)*8;

draw_block(x, y);

}

}

}

nmg

Thought of my mp4, haha

xinmeng_wit

Please tell me how the audio data in the array is generated?
For example, if I have an mp3 audio, how do I generate the corresponding data?

oxygen_sh

xinmeng_wit posted on 2022-5-15 08:29 Please tell me how the audio data in the array is generated? For example, if I have an mp3 audio, how do I generate the corresponding data?

That is to convert the mp3 file data into a U16 array; you can use the "DataToHex" tool, which can save the file-generated array into a .h file.

xinmeng_wit

oxygen_sh posted on 2022-5-15 10:01 is to convert the mp3 file data into a U16 array; you can use the "DataToHex" tool, which can save the file to generate an array...

Got it, thanks