Implementation of Remote Upgrade of Multi-node System Based on Large Capacity FPGA

　　introduction

　　Multi-node systems can be seen in many current electronic system applications. This multi-node system has been increasingly widely used due to its structural scalability, flexibility in function configuration, and good maintainability such as easy to find faulty nodes. Usually, the main hardware components of each node in a multi-node system are very similar. In recent years, the powerful FPGA platform has made the hardware components of each node in a multi-node system more versatile. In particular, with the promotion of IP core technology and the emergence of some powerful CPU cores, such as PicoBlaze and MicroBlaze provided by Xilinx, the main functions of many current systems can be implemented by large-capacity FPGAs without the need to use more expensive high-performance microprocessors.

　　For node systems, this will greatly reduce system costs. For multi-node systems composed of large-capacity FPGAs, system upgrades are a time-consuming and laborious task. At this time, the remote upgrade capability of FPGAs is particularly important. For multi-node systems containing high-performance embedded microprocessors such as ARM, DSP, and PowerPC, these embedded microprocessors use SELECTMAP to load FPGAs and implement remote upgrades without adding too many external devices, which is very economical and efficient. However, for systems whose core components are large-capacity FPGAs, if the FPGA itself receives upgrade data and writes to the non-volatile memory that stores configuration data, once the upgrade fails, it will not be able to upgrade again, thus limiting its application in many places that require high reliability requirements or are inconvenient for maintenance personnel to reach; and if expensive embedded microprocessors are added simply to achieve remote upgrades of the system, the system cost and complexity will be greatly increased. In response to this situation, the author designed a remote upgrade system consisting of an ATmega64 microcontroller and an RS485 bus, plus a host computer connected to Ethernet, to solve the above problems.

　　1 System Design

　　1.1 System structure design

　　The remote upgrade system is mainly composed of a master computer connected to Ethernet, a half-duplex RS485 bus, and various sub-nodes. The composition structure is shown in Figure 1. The master computer obtains the upgrade data of each node of the entire system from the outside through Ethernet, and then sends the corresponding upgrade data to each sub-node through the RS485 bus to complete the upgrade of the multi-node system. The RS485 bus adopts a half-duplex mode of one master and multiple slaves. The master computer is the main controller of the bus. It can only initiate communication connections to each node, and the remaining sub-nodes can only respond to the control commands of the master computer. Figure 1 Structural block diagram of the remote upgrade system of a multi-node large-capacity FPGA system

　　1.2 Node upgrade principle

　　Each node is connected to the main control computer through the RS485 bus. When there is no upgrade data, the bus can be used to transmit the query and control commands of the main control computer to each node. The communication protocol between the main control computer and each node can adopt a protocol with three frame types: query control frame, response frame and data frame. Only the main control computer can send query control frames to query or set the remote upgrade status or working condition information of each node. The child node receives the command and data of the main control computer, sends a response frame, and completes its own upgrade.

　　2 Subnode Hardware Design

　　2.1 Subnode Hardware Structure Diagram

　　As shown in Figure 2, taking a system composed of Xilinx's XC3S4000 large-capacity FPGA as an example, the hardware circuit of the remote upgrade system sub-node mainly includes: ATmega64 single-chip microcomputer, Flash for storing large-capacity FPGA configuration data, and half-duplex bus transceiver connected to the RS485 bus. The ATmega64 single-chip microcomputer is used to load the large-capacity FPGA XC3S4000, receive the FPGA upgrade data from the RS485 bus and write it into the Flash memory SST36VF1601C with 2 MB storage space. The RS485TTL level conversion circuit uses the RS485 transceiver SP485R. Figure 2 Sub-node hardware structure block diagram [page]

　　2.2 Subnode Hardware Design Description

　　In the hardware design of the sub-node, the ATmega64 microcontroller is the key to realizing the entire upgrade function. ATmega64 is a low-power 8-bit CMOS microcontroller based on the enhanced AVR RISC structure. Due to its advanced instruction set and single-clock cycle instruction execution time, the data throughput of ATmega64 is as high as 1 MIPS/MHz, which can alleviate the contradiction between power consumption and processing speed in the system. The AVR core has a rich instruction set and 32 general-purpose working registers, and all registers are directly connected to the arithmetic logic unit (ALU), so that one instruction can access two independent registers simultaneously in one clock cycle. This structure greatly improves code efficiency and has a data throughput of up to 10 times that of ordinary CISC microcontrollers. ATmega64 has the following characteristics:

　　① 64 KB of in-system programmable Flash (with simultaneous read and write capability, i.e. RWW), 2 KB of EEPROM, 4 KB of SRAM, and 32 general-purpose working registers;

　　② 53 general I/O lines;

　　③ Real-time counter (RTC);

　　④ 4 flexible timer/counters (T/C) with compare mode and PWM, programmable watchdog timer with on-chip oscillator;

　　⑤ 2 USARTs, byte-oriented two-wire serial interface (TWI), 1 SPI serial port;

　　⑥ 8-channel 10-bit ADC with optional differential input stage and programmable gain;

　　⑦ JTAG interface compatible with IEEE 1149.1 standard, which can be used to access the on-chip debugging system and programming.

　　ATmega64 is produced with Atmel high-density non-volatile memory technology. The on-chip ISP Flash allows the program memory to be programmed through the ISP serial interface (or a universal programmer) or through the bootloader running in the AVR core. By integrating an 8-bit RISC CPU with in-system programmable Flash in one chip, ATmega64 becomes a powerful single-chip microcomputer that provides a flexible and low-cost solution for many embedded control applications.

　　There are three main technical issues involved in using the ATmega64 microcontroller to remotely upgrade a large-capacity FPGA: first, how to connect the ATmega64 microcontroller to the RS485 bus; second, how to use the ATmega64 microcontroller to read and write large-capacity Flash memory after receiving the upgrade data and loading the FPGA; third, how to implement SELECTMAP loading of the FPGA with the ATmega64 microcontroller.

　　2.2.1 RS485 interface circuit design

　　As listed in Table 1, the RS485 bus standard has the advantages of easy control, low price, high noise suppression, relatively high transmission rate, long transmission distance and wide common mode range. In the past 20 years, the recommended standard RS485, as an electrical specification for multi-point differential data transmission, has been used in many different fields as a data transmission link.

　　Table 1 Performance of TIA/EIA485 serial communication standard The ATmega64 microcontroller of the sub-node is connected to the RS485 bus through a high-performance RS485 transceiver designed and produced by Sipex.

　　SP485R is a RS485 transceiver that is fully compatible with popular standard RS485 chips and includes features such as higher ESD protection and high receiver input impedance. The high receiver input impedance allows 400 transceivers to be connected to the same transmission line without causing attenuation of the RS485 driver signal. The features of this transceiver are as follows:

　　① Allow more than 400 transceivers to be connected to the same transmission line;

　　② Receiver input high impedance (standard value RIN=150 kΩ);

　　③ Half-duplex configuration is consistent with industrial standard pinout;

　　④ The common mode input voltage range is -7 to +12 V;

　　⑤ Low power consumption (250 mW);

　　⑥ Independent driver and receiver enable pins.

　　Its typical application circuit is shown in Figure 3. Figure 3 Half-duplex RS485 circuit using SP485R

　　The standard serial port of the ATmega64 microcontroller is directly connected to the RO pin of the SP485R chip through RXD, and directly connected to the DI pin of the SP485R chip through TXD. The R/D signal output by the microcontroller directly controls the transmitter/receiver enable of the SP485R chip: when the R/D signal is 1, the transmitter of the SP485R chip is valid and the receiver is disabled. At this time, the microcontroller can send data bytes to the RS485 bus; when the R/D signal is 0, the transmitter of the SP485R chip is disabled and the receiver is valid, the microcontroller can receive data bytes from the RS485 bus. The pull-up resistor R1 and the pull-down resistor R2 are used to ensure that the SP485R chip is in an idle state when the bus is not connected, so as to improve the working reliability of each RS485 node. The 6.8 V TVS tubes V1, V2, and V3 are used to protect the RS485 bus to prevent the high voltage generated when the RS485 bus is disturbed by the outside world from damaging the RS485 transceiver.

　　2.2.2 ATmega64 storage space expansion

　　The addressing space of ATmega64 is 64 KB. Using the addressing system of ATmega64 itself, only the address space of 0x0000~0xFFFF can be accessed. Obviously, this is far from enough to store and load the configuration data of large-capacity FPGA. Taking Xilinx's Spartan3 series FPGA as an example, the storage space required for its configuration data is listed in Table 2. [page]

　　Table 2 Spartan3 series FPGA configuration file size Taking the XC3S4000 FPGA as an example, the storage space required for its configuration file is about 1.35 MB, which is far beyond the addressing space of the ATmega64 microcontroller. Therefore, in order to use the ATmega64 microcontroller to read and write FPGA configuration data, its addressing space must be expanded. The expanded hardware connection block diagram is shown in the connection part between the microcontroller and the Flash in Figure 2.

　　In this design, general I/O is used to expand the addressing space of ATmega64 microcontroller. The total addressing space of ATmega64 microcontroller is 64 KB, but the 4 KB SRAM and various registers on the chip occupy part of the addressing space in front, so the addressing space of its off-chip memory is 0x1100~0xFFFF. Therefore, the lower 15 bits of the Flash address are directly connected to the lower 15 bits of the ATmega64 microcontroller address bus, and the remaining 6 high bits of the address are selected by the general I/O of the microcontroller. When addressing, the dedicated address port of the microcontroller only outputs address data from 0x8000 to 0xFFFF, which is combined with the address output by the general I/O to give the read and write address of the Flash.

　　By using the above method to expand the addressing space of ATmega64 microcontroller, the addressing operation of ATmega64 microcontroller to the appropriate Flash memory can be realized, thus solving the problem of storing and reading large-capacity FPGA configuration data.

　　2.2.3 Implementing SELECTMAP loading of FPGA using ATmega64

　　Xilinx's Spartan3 series FPGA loading methods are divided into five types: Master Serial, Slave Serial, Master Parallel, Slave Parallel, and JTAG.

　　According to whether the FPGA controls the loading process, the loading mode can be divided into Master mode and Slave mode; according to the data bit width when loading data, it can be divided into Serial mode and Parallel mode. The pins used for loading are mainly: PROG_B, CCLK, RDWR_B, DONE, INIT_B, CS_B, BUSY, D[0~7], M[0~2], HSWAP and JTAG interface (TDI, TMS, TCK, TDO). The loading process is generally divided into three steps: configuration establishment, configuration data loading and loading completion.

　　SELECTMAP is the Slave Parallel mode, which is a loading method in which an external controller controls the loading process of the FPGA and writes the loading data to the FPGA in the form of 8-bit data. The connection between the FPGA and ATmega64 in Figure 2 is a hardware connection block diagram of the ATmega64 microcontroller using the SELECTMAP method to load the FPGA. The specific implementation process is as follows:

　　ATmega64 starts the loading process by setting the PROG_B, CS_B, and RDWR_B pins of the FPGA low. After PROG_B is set low, the FPGA starts to clear the internal configuration RAM and sets the INIT_B pin low. After PROG_B is set back to 1, at the rising edge of INIT_B changing from low to high, the FPGA samples the M[0-2] pins to obtain the configuration mode information. ATmega64 monitors the INIT_B pin of the FPGA. When the INIT_B pin changes from low to high, it means that the FPGA has completed the clearing of the internal configuration RAM and is ready to receive configuration data. At the rising edge of the CCLK configuration clock given by ATmega64, the configuration data D[0-7] is written to the configuration RAM. When the FPGA receives all the configuration data, the DONE pin is set high by the FPGA. ATmega64 can determine whether the FPGA is loaded by monitoring the DONE pin. For Spartan3 series FPGAs, if the frequency of the CCLK configuring the FPGA is higher than 50 MHz, the external controller also needs to monitor the BUSY pin of the FPGA. When the BUSY pin is high, it means that the FPGA has not completed the processing of the previous configuration data. At this time, the external controller needs to continue to keep the previous configuration data on the D[0~7] pins until the BUSY pin returns to a low level. For the application of this design, the configuration clock frequency provided by ATmega64 is much lower than 50 MHz, so there is no need to consider the control function of the BUSY pin.

　　3 Software Design

　　3.1 Software Design of the Main Control Computer

　　The software running status of the main control computer should be divided into two types: one is the normal query control status, which is used to query and control the working status of each node in the system; the other is the system upgrade status, which is used to perform upgrade control on each sub-node. As shown in Figure 4, these two states can be converted to each other. Figure 4 Main states of the main control computer

　　The query control state of the software is determined by the main functions to be achieved by the system and is not within the scope of this article. In the upgrade state of the system, the main control computer must first obtain the remote upgrade data of each node of the system through Ethernet. After all the upgrade data is received, an upgrade instruction is sent to a node of the system. After the node responds and establishes a communication connection, all the upgrade data of the node is sent to the node. Next, the main control computer determines whether the previous node is the last node to be upgraded. If not, it continues to transmit the upgrade data of the next node. After all the nodes in the system have been upgraded, wait for external input control commands. For example, restart the entire system and load new data; or do not restart temporarily and return to the query control state. The software flow is shown in Figure 5. Figure 5 System upgrade state flow

　　3.2 Software Design of Subnodes

　　As for the software design of the sub-node, like the main control computer, it is also divided into the usual query control state and the system upgrade state, and they can also be converted with the main control computer; but the ATmega64 microcontroller also has to undertake the task of loading the FPGA. After starting up, the ATmega64 microcontroller first loads the sub-node FPGA so that the sub-node can work normally. After the sub-node works normally, it monitors the RS485 bus and determines whether there is any communication with this node. When the main control computer requires to establish a communication connection with this node, it sends feedback information to establish a communication connection with it. The sub-node enters the query control mode or the remote upgrade mode according to the command sent by the main control computer. After entering the remote upgrade mode, the sub-node receives the remote upgrade data sent by the main control computer, and the upgrade data is written to the Flash after verification. After the upgrade is completed, continue to wait for the command of the main control computer to restart or continue running. The specific software design process is shown in Figure 6. Figure 6 Sub-node software process

　　Conclusion

　　FPGA not only inherits the advantages of ASIC in terms of large scale, high integration and high reliability, but also overcomes the disadvantages of ordinary ASIC in terms of long cycle, large investment and poor flexibility, and gradually becomes the ideal choice for many system implementations. In particular, with the improvement of FPGA capacity and performance, coupled with its unique hardware upgrade capability, its application scope is becoming wider and wider. The remote upgrade method for multi-node systems composed of large-capacity FPGA proposed in this paper has a simple system structure, mature technology, and obvious cost advantages.