Real-time system of brain-computer interface based on FPGA

Brain -computer interface (BCI) is a novel human-computer interface method. It is defined as a brain-computer (computer or other device) communication system that does not rely on the normal output pathways of the brain (peripheral nervous system and muscle tissue) [1]. To realize the brain-computer interface, there must be a signal that can reflect the different states of the human brain and can extract and classify this signal in real time or in a short time [2]. Transient visual evoked potential has a strict time-locked synchronization relationship with the stimulus, can be detected more accurately, and the visual stimulation frequency it requires is relatively low, which is not easy to cause visual fatigue. Therefore, this study uses transient visual evoked potential to realize brain-computer interface.

The brain-computer interface system is a closed-loop system consisting of humans and machines[3]. In addition to the human, the brain-computer interface system includes: signal acquisition, signal processing, external devices and control parts. The solution in this paper uses FPGA to replace the computer as the control and information processor of the brain-computer interface. It mainly includes three parts: EEG acquisition circuit, FPGA-based VGA visual stimulator and FPGA development board, as shown in Figure 1.

The electrode cap and EEG acquisition circuit constitute the EEG signal acquisition part. The VGA visual stimulator belongs to the external device and control part. The FPGA development board is the core of the whole system. It not only realizes signal processing, but also realizes the control of EEG acquisition circuit and VGA controller. The FPGA development board uses Cyclone II EP2C35 DSP development board, and the FPGA chip is Cyclone II EP2C35F672.

1 System Design

1.1 Design of EEG acquisition circuit

EEG signals are relatively weak and are drowned in strong background noise and interference. Appropriate circuits must be designed to amplify EEG signals while suppressing noise through filtering, improving the signal-to-noise ratio, obtaining better analog EEG signals, and converting them into digital signals suitable for FPGA processing through AD conversion.

The EEG acquisition circuit designed in this paper mainly includes active electrodes, right leg drive circuit, analog circuit, digital circuit and power supply. The active electrode is used to conduct EEG signals, and the right leg drive circuit can reduce the common mode voltage. The analog circuit includes preamplifier circuit, high-pass filter circuit, amplifier filter circuit and voltage boost circuit to achieve analog signal conditioning and improve signal-to-noise ratio. The digital circuit includes AD conversion circuit and optocoupler isolation circuit, and its main function is to convert analog signals to digital signals.

1.2 Design of VGA visual stimulator based on FPGA

Brain-computer interface visual stimulators can usually be generated on a computer monitor or television through hardware and software, or a dedicated graphic display circuit can be designed [4]. The biggest advantage of using hardware to achieve visual stimulation is stable performance, but it is difficult to modify and upgrade.

The scheme in this paper is a VGA visual stimulator based on FPGA. It uses a VGA display as a visual stimulator and generates graphic stimulation signals through VHDL programming. FPGA is a semi-custom circuit with strong online modification capabilities. The design can be modified at any time without changing the hardware circuit. Therefore, different stimulation modes can be flexibly generated through programming, and modification and upgrading are very convenient. According to the working principle of VGA display line-by-line scanning and VGA industrial standards [5], a VGA controller is designed in FPGA, including a frequency division module, a scanning timing generation module, an image description module, a stimulation frequency control module and a cursor control module. Its block diagram is shown in Figure 2.

The frequency division module generates the pixel output frequency and the frequency required for stimulation. The CycloneⅡEP2C35 chip has 4 enhanced embedded phase-locked loops (PLLs), each of which can provide 3 output clocks with very stable frequency, and can also be divided and multiplied. The embedded phase-locked loop is called by the phase-locked loop IP core, and the pixel output frequency is directly divided to ensure the accuracy and stability of the system clock. The counter divides the frequency required for stimulation and can generate different stimulation frequencies.

The scanning timing generation module provides the timing of VGA line scanning and field scanning. The pixel output frequency is counted and divided to generate the line scanning frequency, which is 31.469 kHz. Then, the line scanning frequency is counted and divided to obtain the field scanning frequency, which is 59.94 Hz.

The image description module describes the stimulus graphics that need to be generated, including 4 squares and numbers 0, 1, 2, and 3 located in the four directions of the screen, up, down, left, and right. The 4 squares are the stimulus targets for multiple target selection, and the cursor is used to feedback the results of the selection. The position and size of the squares and numbers are described by row coordinates and field coordinates. Numbers are marked on the squares to distinguish different squares. Changes or movements of the graphics can cause effective evoked potentials, so the color of the squares is alternating between black and white. In order to make the squares more prominent, the background color is dark slate gray.

The stimulation frequency control module designs the stimulation mode. It adopts the same frequency composite stimulation mode, that is, the number of flashes of each visual stimulation module is the same in unit time, but the flashing moments of each stimulation module are staggered. The clock of module flashing is generated by counter frequency division. The staggered flashing moment is realized by phase delay. A counter is generated by the system clock. The delay is controlled according to the count of the counter to achieve the requirement of precise timing. When the flashing clock signal is low level, the square is black; when it is high level, the square is white. At the same time when the stimulation module changes from black to white, a trigger signal is issued.

The cursor control module describes the cursor graphics and controls the movement of the cursor. The direction and speed of the cursor movement are controlled by changing the cursor coordinates and frequency division. The color of the cursor is red.

The FPGA-based VGA visual stimulator makes full use of the characteristics of FPGA semi-custom circuits and adopts a synchronous timing design method. It not only has stable performance, but also can flexibly design different visual stimulators according to needs. It is easy to implement and powerful, and has the advantages of implementing a visual stimulator in hardware or software.

1.3 Extraction and identification of transient visual evoked potentials

The cumulative average method is the most classic evoked potential signal processing method and is also the most commonly used method to improve the signal-to-noise ratio in electrophysiological measurements [2]. The block diagram of the cumulative average algorithm designed in FPGA is shown in Figure 3, which includes a trigger signal detection module, a RAM address control module, an asynchronous dual-port RAM, an accumulator, and a divider.

The trigger signal detection ensures the time-locked synchronization between the stimulation and the visual evoked potential. Only when the trigger signal of the earliest flashing stimulation module is detected, the EEG data is stored in the RAM. When the required number of flashes is reached, the data writing stops. According to the delay relationship between the stimulation modules, the starting address of the data corresponding to each stimulation module in the RAM can be determined. Then, the data is read out to the accumulator according to the starting address, and the number of accumulated times is the same as the number of flashes. Finally, the divider is used to divide the accumulated number to obtain the average result after accumulation.

Intellectual property (IP) cores refer to verified, reusable IC modules with certain functions. FPGA has a large number of IP cores for various purposes. These IP cores parameterize the cores, and the parameters can be easily operated through header files or graphical user interfaces ( GUIs ). The asynchronous dual-port RAM IP core calls the on-chip RAM to cache EEG data, ensuring the synchronization of the data interface and the data processing speed. The accumulator IP core and the divider IP core are used to implement the accumulator and divider in the algorithm.

If only the cumulative averaging method is used, hundreds of times are needed to obtain a reliable evoked potential waveform, and the signal extraction time is too long. When a small number of cumulative averages are used, in order to further improve the signal-to-noise ratio and extract a more ideal evoked potential waveform, a digital filtering method is used to reduce the influence of noise. FIR filter has a strict linear phase, good stability, and a relatively flat passband. Therefore, FIR filter is used to implement digital filtering. A 15-order low-pass FIR filter is designed using the window function design method. The window function is a Hamming window and the cutoff frequency is 10 Hz. The filter is designed using FDATool in the MATLAB toolbox and converted into HDL code. The FIR filter can be easily implemented in FPGA. The multiplication operation in FIR filtering is implemented by calling the embedded multiplier used for DSP operation in the CycloneⅡ chip. Compared with the multiplier based on logic units, the embedded multiplier has higher performance and occupies fewer logic units. The embedded multiplier can be seamlessly integrated with the M4K RAM block of the CycloneⅡ device to implement efficient DSP algorithms [6].

The recognition of transient visual evoked potentials is to convert the evoked potential signals into a series of control commands, so as to realize the communication and control between the human brain and the outside world. Template matching is one of the traditional pattern recognition methods. The correlation coefficient is an indicator of the degree of correlation between variables, which can be used to judge the degree of curve fitting. Using the correlation coefficient to measure the quality of the template and the unknown pattern match is an effective and feasible method.

First, a transient visual evoked potential waveform with obvious characteristics is selected as the reference template for template matching; then, the correlation coefficient between the real-time extracted evoked potential waveform and the reference template is calculated. If the correlation coefficient value is greater than the set threshold, it is considered that the evoked potential is detected, and a control command is issued to move the cursor to the corresponding stimulation module, thereby realizing the recognition of transient visual evoked potential.

The calculation formula of the correlation coefficient is as follows:

Assuming that the data of the template is y, the value of the y term in formula (1) can be calculated offline first and stored in ROM, thereby improving the speed of real-time calculation. Only one multiplier-adder IP core is needed to call the embedded multiplier in the FPGA chip to implement the multiplication and addition operations in the numerator and denominator, saving device resources. The square root IP core is used to implement square root operations.

FPGA runs fast, internal programs run in parallel, and it has DSP operation IP cores and embedded multipliers, which can quickly and accurately complete EEG processing algorithms and meet the real-time requirements of signal processing.

2 Experimental Results

The brain-computer interface system constructed by the scheme in this paper was used for experiments. Brain-computer interface experiments are usually a group of experiments, in which four different stimulation modules are selected (including the selection of all targets). In the experiment, the subject wears an electrode cap, with his eyes about 70 cm away from the screen, and controls the cursor to move to the target he is looking at. In each experiment, after the four stimulation modules complete a round of flashing, they pause for a few seconds before the next round of flashing. After four rounds of flashing, that is, after all four stimulation modules can be selected, the flashing stops automatically.

The transient visual evoked potential waveform extracted from the experiment is shown in Figure 4. In Figure 4(a), the prototype of the visual evoked potential can be seen, but the characteristics of the visual evoked potential are not very obvious. In Figure 4(b), the evoked potential signal is further processed by FIR filtering, and a visual evoked potential with relatively obvious characteristics is obtained.

The experimental results of the brain-computer interface are shown in Table 1. The subjects were 5 healthy men. When the judgment was correct, the cursor immediately moved to the module that the subject was looking at when the stimulus module stopped flashing.

The brain-computer interface experiment based on transient visual evoked potentials shows that the FPGA-based brain-computer interface real-time system solution given in this article is feasible. The FPGA-based VGA visual stimulator has a very stable stimulation frequency, an easy-to-accept stimulation interface, and is easy to modify and upgrade. The FPGA-based EEG signal processing algorithm uses a small number of cumulative averages combined with FIR filtering to extract evoked potentials, and identifies them through template matching methods. It can quickly and accurately convert transient evoked potential signals into control commands, realizing a real-time brain-computer interface system. The FPGA-based brain-computer interface system is a new method and a useful exploration of the implementation method of brain-computer interfaces.