Design and implementation of streaming media player based on Symbian platform

1 Introduction

Following voice calls, SMS services, and data communications, wireless music downloading and real-time playback are becoming one of the emerging services provided by mobile operators around the world, attracting widespread attention. Compared with traditional music players based on personal computers, the advantage of mobile music players is that they are convenient and fast in downloading and playing, which can bring users a relaxed and comfortable experience. With mobile music players as the medium, operators can rely on their own strong control over mobile communication network services to provide better services to customers. Due to the flexibility and diversity of mobile communication services, they also have strong scalability in value-added services.

The Multimedia Framework (MMF) API of the Symbian S60 platform can provide multimedia services for upper-layer applications and create and distribute rich and colorful music applications for users. As a plug-in set of multimedia processing units, MMF provides a common interface for operating terminal hardware, supports different audio operations, can realize local playback and streaming playback modes, and can play multiple audio formats.

In the existing literature on multimedia players based on Symbian OS, the design principles of the mobile client of the streaming media player based on the Symbian platform are discussed, guiding opinions are given, and optimization suggestions are given in terms of structure and buffering. On this basis, a user interface design scheme is proposed. However, the literature does not give the specific implementation of multimedia technology. This paper designs and implements a multimedia player based on the Symbian platform, but it only supports local playback and does not involve the relevant technologies of streaming playback.

MMusic is a multimedia player based on MMF architecture, used to play a variety of music files. It not only realizes local playback of music files, but also supports streaming playback. In addition, the music player also supports music download, lyrics download and playback synchronization. It has a beautiful interface, simple operation and multiple functions.

2 System overall framework design

When designing the system framework, we first identified the four basic functions that the player needs to implement: ① Support local audio playback in multiple formats such as mp3, wmv, aac, etc.; ② Support streaming audio playback; ③ Support song search and download functions; ④ Support lyrics synchronization function.

The MMusic player is implemented based on the application architecture of the Symbian S60 platform, using a multi-view framework to construct different views according to different functions. Its views include the playback interface (main interface), song search interface, music media library interface, and radio station list interface. The playback interface is the core interface of the entire player, and is mainly responsible for the playback of audio media, including local playback and streaming playback.

According to the different playback modes, the corresponding menu and playback engine are dynamically selected. Local playback uses the CPlayerAdapter engine. The engine uses the audio playback class CMdaAudioPlayerUtility class and its related observer MMdaAudioPlayerCallback to implement local audio playback. Streaming playback is implemented using the CPluginAdaper engine, which uses the CMdaAudioOutputSTream and MMdaAudioOutputStreamCallback classes.

MMusic music player can be divided into six modules according to its functions:

①UI module, responsible for displaying each interface and switching between interfaces, and processing user keystrokes; ②local playback module, which can provide services such as local audio file playback and volume control; ③streaming playback module, which realizes interaction with the server and streaming playback; ④data download module, responsible for downloading and storing audio data; ⑤lyrics synchronization module, which ensures the synchronization of audio playback and lyrics playback; ⑥server module, which provides download resources such as music files and lyrics. The relationship between different modules is shown in Figure 1.

Figure 1 MMusic player function relationship diagram [page]

3 Introduction of key technologies

3.1 Audio playback

The audio playback function can be implemented through the interface provided by MMF. The audio playback class CMdaAudioPlayerUtility provides the API for local playback, while the CMdaAudioOutputStream class provides the API for streaming playback.

3.1.1 Local playback engine

The MMusic player encapsulates the initialization, playback, and pause operations of audio files in the local playback engine CPlayer-Adapter. The engine contains the audio playback class CMdaAudio-PlayerUtility that supports audio playback operations and simple metadata reading operations. Since all its operations are asynchronous, a client is required to perform audio playback operations. The playback engine CPlayerAdapter must inherit the MMdaAudioPlayerCallback interface class. This interface class provides two methods, MapcInitComplete() and MapcPlayComplete(). The local playback process is shown in Figure 2. First, an audio player instance is constructed and created by calling the CMdaAudioPlayerUtility::NewFilePlayerL() function. When the work of opening and initializing an audio sample is completed, the system automatically calls the MapcInit-Complete() method to notify the client of the result of creating the instance. If it is successfully created, the CMdaAudioPlayerUtility::Play() method can be called to play the audio file. When the playback of an audio sample is completed, the system notifies the client that the playback is complete through the MapcPlayComplete() callback function.

3.1.2 Streaming Playback Engine

For streaming audio playback, the CMdaAudioOutputStreamAPI of the multimedia framework is used. CPluginAdaper is an encapsulated streaming playback engine, which calls the related interfaces of streaming media playback. The main differences between local content playback and streaming playback are the file opening method, initialization method and the processing method of audio data stream.

To implement streaming playback, you need to write a client class that implements MMdaAudioOutputStreamCallback in the program.

This class provides three callback functions to inform the client of the result of the audio output streaming process so that the program can handle possible errors. These callback functions are MaoscOpenComplete(), MaoscBufferCopied() and MaoscPlay-Complete(), all of which must be implemented by the user class of the CMdaAudioOutputStream class. Therefore, CPluginAdaper must inherit the observation class MMdaAudioOutputStreamCallback to obtain the result of the streaming playback process.

Figure 2 Local playback flow chart

Before streaming, you need to connect to the server to get the audio data stream to be played. In the MMusic player, sockets are used to implement communication between the server and the client. The client first needs to connect to the Symbian OS socket server, then open a socket and specify TCP as the transmission protocol. Then get the server's IP address and port from the list file and send a request to the server. After the server responds, read the response data and analyze it to initialize the playback engine. The client continues to request the server to play the audio data.

After getting the audio data stream to be played, start streaming playback. [page]

The playback process is shown in Figure 3. First, initialize the CMdaAudioOutputStream class. After initialization, the Multimedia framework calls the MaoscOpenComplete() callback function to indicate that the audio output is available. The parameter given by the framework is an error value, which indicates whether the initialization is successful. If successful, KErrNONe is given. The sampling rate and volume can be set here. After successful initialization, the PlayL() function can be called on the client to play the audio data stream. It calls the FillBufferL() function to fill the data into the buffer and uses CMdaAudioOutputStream::WriteL(const TdesC8＆aData) for playback. The parameter aData is the buffer data.

WriteL() is an asynchronous function. After copying the data in the descriptor aData to the audio hardware, the framework will call the MMdaAudioOutputStreamCallback::MaoscBufferCopied() callback method to notify the client application that aData has been received and copied to the playback stream. At this point, you can call the FillBufferL() function to continue filling the buffer, thereby achieving continuous streaming playback.

Figure 3 Streaming playback flow chart

3.2 Audio file download and breakpoint resume

MMusic uses the Hypertext Transfer Protocol (HTTP) to download audio files and lyrics. HTTP is an application layer protocol implemented on the TCP/IP protocol, used to transmit information on the Internet. It uses Socket to transmit data between the server and the client.

In the Symbian OS version, API support for HTTP client is provided.

When using HTTP for communication, the client needs to establish an HTTP client session to handle communication with the HTTP server on the session. HTTP communication transactions are established on the session. Transactions consist of requests and responses. One or more transactions can be established on the same session. Multiple sessions can also be established at the same time to achieve different connections.

When creating an HTTP engine, first create an object of the RHTTPSession class, then call the Open() method of the class to open the newly established session. Then use the RHTTPTransaction class to represent the HTTP transaction. The transaction is opened in the HTTP client session. When opening a transaction, you need to specify the data transmission method used by the transaction, GET or POST, and pass in the URL to be opened. After the server completes the processing, it returns the response data. The HTTP client processes the response result and receives the response data by retransmitting the MHTTP-TransactionCallback.

The download of songs is implemented in two steps. First, the name of the song to be searched is sent to the server using the HTTP engine, and the server returns an XML file to the client. The XML file contains information such as the name of the song searched, the download address, the audio file format and size. The client parses the file using Symbian's built-in XML parser and stores the parsed song information in an array.

Each element of the array is a custom structure, including song name, download address and other content.

After parsing is completed, switch to the list interface. The list interface displays the parsing results of the XML file just downloaded. Users can choose the songs to download according to their preferences. At this time, the client extracts the song download address and sends a download request to the server using the encapsulated HTTP engine. The client first determines whether the file to be downloaded already exists. If not, it uses the HTTP engine's normal download mode to download the audio file. If the file to be downloaded already exists, the existing file size is obtained and passed as a parameter to the HTTP engine and sent to the server to achieve breakpoint resuming. The specific code is as follows:

[page]

3.3 Lyrics synchronization

First, search for lyrics files with the same name as the music file being played locally on the user terminal. If a qualified file is found, it is parsed. Each line of the file is stored in an array as an array unit, and the data unit includes the play time and lyrics content. The GetPosition() function of the CMdaAudioPlayerUtility class obtains the play position of the current audio file and compares it with the time of the lyrics file recorded in the array to achieve lyrics synchronization. If no lyrics file is found, the lyrics download engine is used to download and parse it from the server.

4 Software Implementation and Effect

According to the overall architecture design of the player, the music media player is implemented on the Symbian platform and tested in the actual mobile communication network. The test environment is China Mobile's GPRS/EDGE network. The test phone is Nokia N95, the operating system is V9.2, and the user interface is S60 3rd Edition. Its ARM main frequency is 11 332MHz and the memory is 160MB. When tested on the real machine, the local audio file plays smoothly. When playing the content of the online radio station (i.e. streaming playback), its effect will be restricted by the network conditions, but it is relatively smooth overall.

The effect of the player on a real machine is shown in Figure 4. (a) shows the function menu of the player, (b) shows the effect of local playback and lyrics synchronization, (c) shows the status of connecting to the server during streaming playback, and (d) shows the effect of streaming playback after successfully connecting to the server. The interface can display the current connection speed, audio file format, traffic, etc.

(a) Player function menu (b) The effect of local playback and lyrics synchronization (c) The status of connecting to the server during streaming playback (d) The effect of streaming playback after successfully connecting to the server

Figure 4 Software renderings

5 Conclusion

MMusic music player has well realized the functions of audio playback, music download, lyrics synchronization, etc. On this basis, some audio effects can also be added to give users a better audio-visual experience. At the same time, in terms of lyrics synchronization, the program can also add the function of adjusting lyrics. When the lyrics are not synchronized, users can adjust them according to the actual situation.