Design of H.264 Streaming Media Player Based on Directshow

DirectsHow application framework completes the underlying work of streaming media processing, so that programmers do not need to worry about how to input data and how to output it after processing, but only need to worry about how to process the input data. The H.264 video codec standard has a high compression ratio and excellent network affinity, and is generally considered to be the most influential streaming video compression standard. A streaming media player that combines Direct-show and H.264 will undoubtedly have excellent performance.

1 Introduction to DirectShow Technology and H.264 Video Compression Standard

Directshow is a streaming media development software package provided by Microsoft. It provides a complete solution for multimedia applications with high performance requirements such as playback of media files in various formats and audio and video acquisition on the Windows platform.

Directshow is a complete COM-based application system located in the application layer. It uses the FilterGraph model to manage the entire data flow processing process. Each functional module involved in data processing is called a Filter. Each Filter is connected to form a "pipeline" in a certain order in the FilterGraph to work together. Filter is a COM component, and its functions can be implemented by the user. DirectShowSDK also provides some standard Filters for users to use. Each Filter is connected to each other through Pin in a certain order in the FilterGraph, and Pin is also a COM object.

H.264 is the latest international video coding standard developed by the joint development group of ITU-T and ISO/IEC. The H.264/AVC video coding standard has significantly improved the coding quality and compression ratio compared with the original video coding standards. At the same visual perception quality, the coding efficiency is about 50% higher than that of H.263, MPEG-2 and MPEG-4. H.264 not only has excellent compression performance, but also has good network affinity. Therefore, H.264 is generally regarded as the most influential streaming video compression standard.

2 System Design Framework

This system is based on the Directshow application framework and H.264 video compression standard, and realizes the function of receiving streaming media data from the network and playing it in real time on the client. The streaming media file is an AVI file encoded with H.264. Since Directshow provides AVISplitteRFilter, AudioDecoder and standard Video/AudioRenderer, this system only needs to design a custom network source filter and H.264 decoding filter.

Multimedia streaming actually involves two aspects of technology. One is the communication technology between the server and the client, including the transmission of multimedia data, command control, etc.; the other is the technology for the client to decode and play the received multimedia stream in real time. Obviously, network communication can use windowssocket technology, and multimedia stream decoding and playback can use direct-show technology. This paper adopts the directshow application framework, designs the network source filter and h.264 decoding filter, and constructs a streaming media player through FilterGraph.

The FilterGraph for playing local H.264-encoded AVI files is shown in Figure 1. Simply replace the local FileSourceFilter with the network source Filter and the CoreAVCVideoDecoder with the h.264 decoding Filter to implement a network H.264 video player.

3 System Design and Implementation

3.1Filter Design Process

The coding implementation of Filter includes Filter registration information, framework function implementation on Filter, logic control class implementation, custom interface implementation, property page implementation, property rights protection, etc.

First, we need to analyze the functions that the Filter needs to complete and its position in the FilterGraph to determine the filter model and select a suitable base class; then, define the input and output PINs and custom interfaces, and register the Filter information; finally, implement all pure virtual functions and custom interface functions of the base class, and rewrite the related functions of the base class to customize the filter functions.

3.2 Design of Network Source Filter

The main function of the source filter is to receive streaming media data from the server and provide it to other filters in the FilterGraph.

Since the AVIsplitter that comes with directshow works in pull mode, the source Filter also works in pull mode.

This source filter uses double buffer circular queue technology to receive data and pass data to the next level filter. The reasons for using this technology are as follows:

(1) During the connection between SourceFilter and SplitterFilter, a portion of data will be read from SourceFilter to obtain the format description of the data, otherwise FilterGraph cannot complete the connection. Before the source filter is connected to SplitterFilter, a waiting thread should be started. When the data cache of SourceFilter has received enough data in advance, the complete FilterGraph will be constructed.

(2) When the complete FilterGrapH is built and is in operation, SourceFilter must dynamically receive data and continuously provide new data to SplitterFilter. The double-buffered circular queue can fully utilize the memory space and provide a stable data source for SplitterFilter.

(3) The buffer queue can stabilize the bit rate and effectively reduce the impact of network delay, congestion and jitter.

The working process of the source filter is as follows: establish a circular buffer queue, the queue tail pointer is used to buffer the data received from the network, and the queue head pointer is used for the Splitter to read the data, separate the audio and video, and pass it to the next level Decoder for processing; when the Socket receives the network data, the data is inserted into the tail of the queue and the tail pointer is moved backward; when the Splitter needs to read data, the data is read from the head of the queue and the head pointer is moved backward.

Streaming media transmission adopts the client/server architecture. There is also a Socket communication protocol problem between the server and the client. Since streaming media is continuous, its synchronization point cannot be selected arbitrarily. Therefore, in order to transmit streaming media data, a connection-oriented reliable transmission protocol (TCP) must be used. The control and feedback messages between the client and the server can be transmitted using (UDP). The server first creates a listening Socket to listen for connection requests from the client. Once the client's request is heard, the server creates a Socket for data transmission and binds it to the client requesting a connection. At this time, the server is ready for data transmission. When the client issues a command, the server performs corresponding operations according to the type of command, such as data sending, stopping, disconnecting, etc.

On the server side, the continuous H.264 stream is first divided into small packets of payload data, and a header is added to send them using the TCP protocol; on the client side, the payload data of the small packets is assembled according to the header description, and then H.264 decoding and playback is performed. The Socket data transmission structure is: payload type (8bit), payload data length (16bit) and payload data packet (2324Byte).

The client implementation can refer to the MEMFileFilter in the SDK. Copy the four files asyncrdr.cpp, asyncrdr.h, asyncio.cpp, and asyneio.h directly from the SDK example, and then derive the stream processing class CMemStream from CAsyncStream in the MemFilter.h file to customize the data source and perform data reading operations; derive the filter class CMemReader from CAsyncReader, implement the SourceFilter framework with 1 output PIN, and complete the connection with SplitterFilter.

The data flow between network source filter classes is shown in Figure 2, and the data flow of the "pull" thread of the next level Filter is shown in Figure 3. Among them: CAsyneStream is used to mark the data flow; CAsyneRequest marks the input and output requests; CAsyncIo implements the control of data input and output. The programming mainly implements the CMemStream: Read() function to read data from a specific data source to the Sample of the next level Filter.