Kernel transplantation and file system production (4): Summary of UBIFS root file system production

Introduction to UBIFS file system:

Unsorted Block Image File System (UBIFS) is used on solid-state drive storage devices and competes with LogFS as one of the successor file systems of JFFS2 . The real development began in 2007 and was first added to the stable version of Linux kernel 2.6.27 in October 2008. UBIFS was first designed by Thomas Gleixner and Artem Bityutskiy, engineers from IBM and Nokia , in 2006 to solve the bottlenecks encountered by MTD (Memory Technology Device) devices. Due to the surge in Nand Flash capacity, YAFFS and other systems can no longer control the space of Nand Flash. UBIFS handles the actions between MTD devices through the subsystem UBI. Like JFFS2 , UBIFS is built on MTD devices and is therefore incompatible with general block devices . JFFS2 runs on MTD devices, while UBIFS can only work on UBI volumes . It can also be said that UBIFS involves three subsystems:

1. MTD subsystem, providing access interface to flash chip. MTD subsystem provides the concept of MTD device, such as /dev/mtdx. MTD can be considered as raw flash.

2. UBIsubsystem, which provides wear-leveling and volume management functions for flash devices; UBI works on top of MTD devices and provides UBI volumes; UBI is a high-level representation of MTD devices, shielding the upper layer from some issues that MTD has to deal with, such as wearing and bad block management

3. UBIFS file system, working on UBI, must select support for MTD in Device Drivers

The following are some features of UBIFS:

Ø Scalability: UBIFS has good scalability to flash size; that is, mount time, memory consumption and I/O speed are not dependent on flash size (it is not completely accurate for memory consumption, but the dependence is very low); UBIFS can adapt well to GB flashes; of course, UBI itself has scalability issues, but UBI/UBIFS is better scalable than JFFS2 anyway. In addition, if UBI becomes a bottleneck, you can upgrade UBI without changing UBIFS.

Ø Fast mount: Unlike JFFS2, UBIFS does not need to scan the entire file system during the mount phase. The time it takes to mount the UBIFS medium is only in milliseconds, and the time does not depend on the size of the flash ; however, the initialization time of UBI does depend on the size of the flash, so this time must be taken into account

Ø     write-back支持：回写或者叫延迟写更准确些吧，同JFFS2的write-through(立即写入内存)相比可以显著的提高文件系统的吞吐量。

Ø Abnormal unmount adaptability: UBIFS is a log file system that can tolerate sudden power failure and unclean reboot. UBIFS recovers from unclean unmount by replaying logs. In this case, replay will consume some time, so the mount time will increase slightly. However, the replay process does not scan the entire flash medium, so the mount time of UBIFS is about a few tenths of a second.

Ø Fast I/O - Even if we disable write-back (using the -o syncmount option when unmounting), UBIFS still performs close to JFFS2; remember, JFFS2's synchronous I/O is amazing, because JFFS2 does not need to maintain indexing data structures on flash, so there is no burden; UBIFS does have index data. UBIFS is fast because of the way UBIFS commits logs: instead of moving data from one place to another, it just adds the address of the data to the file system index, and then chooses a different eraseblock as the new log block, in addition to multi-headed log methods and other tricks.

Ø on-the_flightcompression - Data stored on the flash media is compressed ; there is also the flexibility to turn compression on and off for individual files; for example, you may want to turn compression on for a specific file, or you may enable compression by default but turn it off for multimedia files.

Ø Recoverability - UBIFS can recover from index corruption; each piece of information in UBIFS is described by a header, so the file system can be reconstructed by scanning the flash medium, which is very similar to JFFS2; imagine that if you erase the FAT table of a FAT file system, it is a fatal error for FAT FS, but if you erase the UBIFS index, you can still reconstruct the file system, of course, this requires a special user space program to do this recovery

Ø Integrity - UBIFS ensures data integrity by writing a checksum to the flash media. UBIFS will not ignore corrupted file data or meta-data. By default, UBIFS only checks the CRC of the meta-data, but you can force a data CRC check with a mount option.

UBIFS file system production process:

1. Add kernel support (linux-3.8.0):

Device Drivers  --->

<*> Memory Technology Device (MTD) support  --->

<*>   Enable UBI - Unsorted block images  --->

--- Enable UBI - Unsorted block images

(4096) UBI wear-leveling threshold

(1)   Maximum expected bad eraseblock count per 1024 eraseblocks

[ ]   UBI Fastmap (Experimental feature)

< >   MTD devices emulation driver (gluebi)

File systems  --->

[*] Miscellaneous filesystems  --->

<*>   UBIFS file system support

[*]     Advanced compression options

[*]       LZO compression support

[*]       ZLIB compression support

Make compile

2. Create a UBIFS file system image

The file system image created by the mkfs.ubifs tool is too complicated to burn in uboot. It is necessary to make uboot support nandflash partition, activate this partition in uboot, and burn this image through the ubi write command. mkfs.ubifs, ubinize tool Its function is to convert the image created by mkfs.ubifs into an image file that can be burned directly using the nand write command.

Since the mkfs.jffs2 and mkfs.ubifs tools are created, the ubinize tool will be generated at the same time. I just create the tools and generate the UBIFS file system image according to the script.

(1) Production mkfs.jffs2 and mkfs.ubifs Tool link: http://blog.csdn.net/fulinus/article/details/8860836

(2) Generate UBIFS script:

Note two points:

The mkfs.ubifs tool requires explicit parameters:

m: page size, as above: 2K

x: image compression format, optional LZO

e: logical erasable block size

c: Maximum number of logical erasable items

r (or d): specifies the root file directory tree

o: Specify the production file name

The ubinize tool requires explicit parameters:

You need to specify a configuration file in the following format:

[ubifs-volume]
mode=ubi
image=filename //File name, i.e., mkfs.ubifs file to create production file
vol_id=0
vol_size= filesize //File size, determined according to actual situation
vol_type=dynamic
vol_name=rootfs
vol_flags=autoresize
vol_alignment=1

m: page size, 2k

p: physical block erase size

s: Usually the same as the page size 2k

v: Header size, usually 512

o: specify output file

mkfs.ubifs and ubinize usage Reference Links:

http://wenku.baidu.com/link?url=ZjS_qifTxWr68prdj780aT6yFp0rD8IbZGFvbT0ASwUhZbvdQBBzyIGTp_d5bbJx5IAb4TJX4CDmdUJ26mRwn2hij7G58X-WHPZVYCpR1MG

UBIFS file system introduction:

http://wenku.baidu.com/view/eebeacd9a58da0116c174991.html

Make the script:

[zhouguangfeng@localhost ubifs]$ vim build_ubifs.sh

#!/bin/sh

CPU=zhou
rootfs_dir=../rootfs //Specify the root file directory, according to the root file directory tree
partition_sizeM=40
image_name=rootfs_${CPU}.ubifs


if [ ! -d $rootfs_dir ] ; then
echo "Miss rootfs source code tree "$rootfs_dir" exit..."
exit;
fi

#Default setting by UBIFS
sub_page_size=512
vid_head_offset=512


#-m, minimum I/O unit size, it's 2K(the Page size) on K9F2G08, refer to "UBI: smallest flash I/O unit:    2048"
page_size_in_bytes=2048
#echo "Page size [$page_size_in_bytes] bytes."


#It's 64 pages per block on K9F2G08
pages_per_block=64
block_size_in_bytes=`expr $page_size_in_bytes * $pages_per_block`
#echo "[$pages_per_block] pages per block and [$block_size_in_bytes] bytes"


#It's 2048 blocks on K9F2G08
blocks_per_device=2048
#echo "Blocks per device  [$blocks_per_device]"


#-e, logical erase block size, fixed on K9F2G08, refer to u-boot information "UBI: logical eraseblock size:  129024 bytes"
# logical erase block size is physical erase block size minus 1 page for UBI
logical_pages_per_block=`expr $pages_per_block - 1`
logical_erase_block_size=`expr $page_size_in_bytes * $logical_pages_per_block`
#echo "Logical erase block size:  [$logical_erase_block_size] bytes."

#The rootfs partition size in bytes
partition_size_in_bytes=`expr $partition_sizeM * 1024 * 1024`
partition_physical_blocks=`expr $partition_size_in_bytes / $block_size_in_bytes`
#echo "Partition size [$partition_size_in_bytes] bytes and [$partition_physical_blocks] blocks."


#Logical blocks on a partition = physical blocks on a partitiion - reserved for wear level
patition_logical_blocks=`expr $partition_physical_blocks - $wear_level_reserved_blocks`
#echo "Logical blocks in a partition [$patition_logical_blocks]"


#File-system volume = Logical blocks in a partition * Logical erase block size
fs_vol_size=`expr $patition_logical_blocks * $logical_erase_block_size`
#echo "File-system volume [$fs_vol_size] bytes."


config_file=rootfs_ubinize.cfg
image_tmp=ubifs-${CPU}.img