【i.MX6ULL】Driver Development 11——LCD Driver Practice

DDZZ669

【i.MX6ULL】Driver Development 11——LCD Driver Practice [Copy link]

The LCD driver was mentioned before when porting the Linux system. This article will look at how to configure the LCD driver in the Linux device tree.

1 Knowledge Points

First, you need to understand a new concept: Framebuffer

1.1 Framebuffer

Framebuffer literally means frame buffer, abbreviated as fb. Linux brings together all display-related hardware and software in the system, and abstracts the underlying LCD virtual device into a /dev/fbX device. Applications can control the screen display by operating /dev/fbX.

The NXP official Linux kernel has enabled the LCD driver by default. You can see a device like fb0 in the dev/directory.

Framebuffer is represented in the kernel as the fb_info structure:

The complete structure definition is as follows:

struct fb_info {
    atomic_t count;
    int node;
    int flags;
    struct mutex lock;      /* Lock for open/release/ioctl funcs */
    struct mutex mm_lock;       /* Lock for fb_mmap and smem_* fields */
    struct fb_var_screeninfo var;   /* 当前的可变参数 */
    struct fb_fix_screeninfo fix;   /* 当前的固定参数 */
    struct fb_monspecs monspecs;    /* Current Monitor specs */
    struct work_struct queue;   /* Framebuffer event queue */
    struct fb_pixmap pixmap;    /* Image hardware mapper */
    struct fb_pixmap sprite;    /* Cursor hardware mapper */
    struct fb_cmap cmap;        /* Current cmap */
    struct list_head modelist;   /* mode list */
    struct fb_videomode *mode;  /* current mode */

#ifdef CONFIG_FB_BACKLIGHT
    /* assigned backlight device */
    /* set before framebuffer registration, 
      remove after unregister */
    struct backlight_device *bl_dev;

    /* Backlight level curve */
    struct mutex bl_curve_mutex;    
    u8 bl_curve[FB_BACKLIGHT_LEVELS];
#endif
#ifdef CONFIG_FB_DEFERRED_IO
    struct delayed_work deferred_work;
    struct fb_deferred_io *fbdefio;
#endif

    struct fb_ops *fbops;    /* 帧缓冲操作函数集 */
    struct device *device;      /* This is the parent */
    struct device *dev;       /* This is this fb device */
    int class_flag;       /* private sysfs flags */
#ifdef CONFIG_FB_TILEBLITTING
    struct fb_tile_ops *tileops;  /* Tile Blitting */
#endif
    char __iomem *screen_base;    /* 虚拟内存基地址(屏幕显存) */
    unsigned long screen_size;    /* 虚拟内存大小(屏幕显存大小) */ 
    void *pseudo_palette;         /* 伪16位调色板 */ 
#define FBINFO_STATE_RUNNING    0
#define FBINFO_STATE_SUSPENDED  1
    u32 state;                /* Hardware state i.e suspend */
    void *fbcon_par;        /* fbcon use-only private area */
    /* From here on everything is device dependent */
    void *par;
    /* we need the PCI or similar aperture base/size not
      smem_start/size as smem_start may just be an object
      allocated inside the aperture so may not actually overlap */
    struct apertures_struct {
        unsigned int count;
        struct aperture {
            resource_size_t base;
            resource_size_t size;
        } ranges[0];
    } *apertures;

    bool skip_vt_switch; /* no VT switch on suspend/resume required */
};

Note the fb_fops item in the structure . /dev/fb0 is a character device, and fb_fops is its file operation structure. Its file_operations operation set is in the drivers/video/fbdev/core/fbmem.c file:

static const struct file_operations fb_fops = {
    .owner =    THIS_MODULE,
    .read =     fb_read,
    .write =    fb_write,
    .unlocked_ioctl = fb_ioctl,
#ifdef CONFIG_COMPAT
    .compat_ioctl = fb_compat_ioctl,
#endif
    .mmap =     fb_mmap,
    .open =     fb_open,
    .release =  fb_release,
#ifdef HAVE_ARCH_FB_UNMAPPED_AREA
    .get_unmapped_area = get_fb_unmapped_area,
#endif
#ifdef CONFIG_FB_DEFERRED_IO
    .fsync =    fb_deferred_io_fsync,
#endif
    .llseek =   default_llseek,
};

You can see familiar function interfaces such as open and release.

Therefore, the focus of LCD driver is to initialize each member in fb_info .

There are many member variables of the fb_info structure, and the following ones need to be focused on:

• var: current variable parameter

• fix: current fixed parameters

• fbops: frame buffer operation function set

• screen_base: virtual memory base address (screen video memory)

• screen_size: virtual memory size (screen video memory size)

• pseudo_palette: pseudo 16-bit palette

After initializing fb_info, register the newly initialized fb_info with the kernel through the register_framebuffer function .

1.2 Introduction to LCD driver file mxsfb

The driver file of LCD is mxsfb.c , which is a platform driver framework. After the driver and device are matched, the mxsfb_probe function will be executed.

LCD initialization is implemented through the mxsfb_probe function , the main functions of this function are:

• Apply for fb_info

• Initialize each member variable in the fb_info structure

• Initialize the eLCDIF controller

• Use the register_framebuffer function to register the initialized fb_info with the Linux kernel

This function is located in: /drivers/video/fbdev/mxsfb.c

The implementation of this function is as follows:

static int mxsfb_probe(struct platform_device *pdev)
{
    const struct of_device_id *of_id =
            of_match_device(mxsfb_dt_ids, &pdev->dev);
    struct resource *res;
    struct mxsfb_info *host; //<-----NXP的fb_info
    struct fb_info *fb_info; //<-----Linux的fb_info
    struct pinctrl *pinctrl;
    int irq = platform_get_irq(pdev, 0);
    int gpio, ret;

    if (of_id)
        pdev->id_entry = of_id->data;

    gpio = of_get_named_gpio(pdev->dev.of_node, "enable-gpio", 0);
    if (gpio == -EPROBE_DEFER)
        return -EPROBE_DEFER;
    //省略...

    fb_info = framebuffer_alloc(sizeof(struct fb_info), &pdev->dev);//<--------申请fb_info
    if (!fb_info) {
        dev_err(&pdev->dev, "Failed to allocate fbdev\n");
        devm_kfree(&pdev->dev, host);
        return -ENOMEM;
    }
  host->fb_info = fb_info; //<---将mxsfb_info与fb_info联系起来
  fb_info->par = host;
    //省略...

    ret = mxsfb_init_fbinfo(host);
    if (ret != 0)
        goto fb_pm_runtime_disable;

    mxsfb_dispdrv_init(pdev, fb_info);
    //省略...
  
    ret = register_framebuffer(fb_info); //<------------注册
    if (ret != 0) {
        dev_err(&pdev->dev, "Failed to register framebuffer\n");
        goto fb_destroy;
    }

    console_lock();
    ret = fb_blank(fb_info, FB_BLANK_UNBLANK);
    console_unlock();
    if (ret < 0) {
        dev_err(&pdev->dev, "Failed to unblank framebuffer\n");
        goto fb_unregister;
    }

    dev_info(&pdev->dev, "initialized\n");
    
}

Among them, the prototype of the register_framebuffer function is as follows:

Function parameters and return value meaning:

• fb_info: fb_info to be reported

• Return value: 0 - success, negative value - failure

1.3 LCD driver programming

NXP has already written the eLCDIF interface driver for 6ULL, so we don't need to modify the LCD driver part. All we need to do is modify the device tree according to the LCD used.

1.3.1 View the device tree

1.3 Let's first look at the LCD driver for Linux officially written by NXP. Open imx6ull.dtsi and find the content of the lcdif node:

lcdif: lcdif@021c8000 {
  compatible = "fsl,imx6ul-lcdif", "fsl,imx28-lcdif";
  reg = <0x021c8000 0x4000>;
  interrupts = <GIC_SPI 5 IRQ_TYPE_LEVEL_HIGH>;
  clocks = <&clks IMX6UL_CLK_LCDIF_PIX>,
  <&clks IMX6UL_CLK_LCDIF_APB>,
  <&clks IMX6UL_CLK_DUMMY>;
  clock-names = "pix", "axi", "disp_axi";
  status = "disabled";
};

The address 021c8000 can be found in the reference manual:

1.3.2 Screen IO Configuration

Open the imx6ull-myboard.dts file and find the following content in the iomuxc node:

Specifically:

pinctrl_lcdif_dat: lcdifdatgrp {
            fsl,pins = <
                MX6UL_PAD_LCD_DATA00__LCDIF_DATA00 0x79
                MX6UL_PAD_LCD_DATA01__LCDIF_DATA01 0x79
                MX6UL_PAD_LCD_DATA02__LCDIF_DATA02 0x79
                MX6UL_PAD_LCD_DATA03__LCDIF_DATA03 0x79
                MX6UL_PAD_LCD_DATA04__LCDIF_DATA04 0x79
                MX6UL_PAD_LCD_DATA05__LCDIF_DATA05 0x79
                MX6UL_PAD_LCD_DATA06__LCDIF_DATA06 0x79
                MX6UL_PAD_LCD_DATA07__LCDIF_DATA07 0x79
                MX6UL_PAD_LCD_DATA08__LCDIF_DATA08 0x79
                MX6UL_PAD_LCD_DATA09__LCDIF_DATA09 0x79
                MX6UL_PAD_LCD_DATA10__LCDIF_DATA10 0x79
                MX6UL_PAD_LCD_DATA11__LCDIF_DATA11 0x79
                MX6UL_PAD_LCD_DATA12__LCDIF_DATA12 0x79
                MX6UL_PAD_LCD_DATA13__LCDIF_DATA13 0x79
                MX6UL_PAD_LCD_DATA14__LCDIF_DATA14 0x79
                MX6UL_PAD_LCD_DATA15__LCDIF_DATA15 0x79
                MX6UL_PAD_LCD_DATA16__LCDIF_DATA16 0x79
                MX6UL_PAD_LCD_DATA17__LCDIF_DATA17 0x79
                MX6UL_PAD_LCD_DATA18__LCDIF_DATA18 0x79
                MX6UL_PAD_LCD_DATA19__LCDIF_DATA19 0x79
                MX6UL_PAD_LCD_DATA20__LCDIF_DATA20 0x79
                MX6UL_PAD_LCD_DATA21__LCDIF_DATA21 0x79
                MX6UL_PAD_LCD_DATA22__LCDIF_DATA22 0x79
                MX6UL_PAD_LCD_DATA23__LCDIF_DATA23 0x79
            >;
        };

        pinctrl_lcdif_ctrl: lcdifctrlgrp {
            fsl,pins = <
                MX6UL_PAD_LCD_CLK__LCDIF_CLK      0x79
                MX6UL_PAD_LCD_ENABLE__LCDIF_ENABLE 0x79
                MX6UL_PAD_LCD_HSYNC__LCDIF_HSYNC  0x79
                MX6UL_PAD_LCD_VSYNC__LCDIF_VSYNC  0x79
            >;
        };

        pinctrl_pwm1: pwm1grp {
            fsl,pins = <
                MX6UL_PAD_GPIO1_IO08__PWM1_OUT  0x110b0
            >;
        };

There are 3 nodes here:

• The child node pinctrl_lcdif_dat is the configuration item for the 24 data lines of the RGB LCD

• The child node pinctrl_lcdif_ctrl is the configuration item for the four control lines of the RGB LCD , including CLK, ENABLE, VSYNC and HSYNC

• The child node pinctrl_pwm1 is the backlight brightness configuration item of the RGB LCD

1.3.3 Screen parameter configuration

Find the lcdif node in the imx6ull-myboard.dts file and modify it to the corresponding parameters according to the LCD you are using.

Below are the parameters of the NXP official board:

I use a 7-inch Wildfire screen (GT911, 800x480), and its parameters are:

parameter	value
width	800
height	480
HBP	46
HFP	twenty two
VBP	twenty three
VFP	twenty two
HSPW	1
VSPW	1

The modified lcdif node is as follows:

&lcdif {
    pinctrl-names = "default";       /* 使用到的 IO */ 
    pinctrl-0 = <&pinctrl_lcdif_dat
           &pinctrl_lcdif_ctrl
           &pinctrl_lcdif_reset>;
    display = <&display0>;
    status = "okay";

    display0: display {         /* LCD 属性信息 */ 
        bits-per-pixel = <16>;      /* 一个像素占用几个bit */ 
        bus-width   = <24>;      /* 总线宽度 */ 

        display-timings {
            native-mode = <&timing0>;  /* 时序信息 */ 
            timing0: timing0 {
            clock-frequency = <9200000>; /* LCD像素时钟，单位Hz */ 
            hactive   = <800>;    /* LCD X轴像素个数 */ 
            vactive   = <480>;    /* LCD Y轴像素个数 */ 
            hfront-porch = <22>;     /* LCD hfp 参数 */ 
            hback-porch = <46>;     /* LCD hbp 参数 */
            hsync-len  = <23>;     /* LCD hspw 参数 */ 
            vback-porch = <22>;     /* LCD vbp 参数 */
            vfront-porch = <4>;     /* LCD vfp 参数 */ 
            vsync-len  = <1>;     /* LCD vspw 参数 */ 

            hsync-active  = <0>;    /* hsync 数据线极性 */ 
            vsync-active  = <0>;    /* vsync 数据线极性 */ 
            de-active    = <1>;    /* de 数据线极性 */ 
            pixelclk-active = <0>;    /* clk 数据线极性 */ 
            };
        };
    };
};

1.3.4 Screen backlight configuration

Control the brightness of the LCD screen backlight through PWM signal

pinctrl_pwm1: pwm1grp {
  fsl,pins = <
    MX6UL_PAD_GPIO1_IO08__PWM1_OUT  0x110b0
    >;
};

LCD backlight uses PWM1, so you also need to set the PWM1 node. Find the following in the imx6ull.dtsi file:

This node information does not need to be modified, and the default configuration can be used. If you want to modify it, do not modify it here, but modify it in the imx6ull-myboard.dts file.

pwm1 node in imx6ull-myboard.dts:

&pwm1 {
    pinctrl-names = "default";
    pinctrl-0 = <&pinctrl_pwm1>;
    status = "okay";
};

Backlight node in imx6ull-myboard.dts:

backlight {
  compatible = "pwm-backlight";
  pwms = <&pwm1 0 5000000>;
  brightness-levels = <0 4 8 16 32 64 128 255>;
  default-brightness-level = <6>;
  status = "okay";
};

2 Experimental tests

2.1 Enable Linux logo display

When uboot is started, the NXP logo will be displayed on the upper left corner of the LCD, and when the Linux kernel is started, a small penguin will be displayed on the upper left corner of the LCD. Therefore, the display of the small penguin logo can be used to verify whether the LCD driver is normal.

By default, the logo display is turned on, you can confirm it again.

In the Linux kernel source directory, enter the following command to open the kernel's graphical configuration:

make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig

Linux kernel configuration interface:

Then follow the path to find the corresponding configuration item:

-> Device Drivers 
   -> Graphics support  
   -> Bootup logo (LOGO [=y])

Finally, you reach this interface:

These three options correspond to the logo in black and white, 16-bit, and 24-bit color formats respectively.

2.2 Compile device tree

After modifying the lcdif node in the device tree (mainly modifying the screen parameters), execute the following command in the Linux kernel source directory to recompile the device tree and copy it to the network boot location.

make imx6ull-myboard.dtb
cp arch/arm/boot/dts/imx6ull-myboard.dtb ~/myTest/tftpboot/nxp/

Then restart the development board, and you can see the penguin icon on the screen when the Linux kernel is driven:

2.3 Setting up LCD as a terminal console

Previously, the serial port was used to display the startup and debugging information of the board. In fact, the LCD can be set as a terminal for synchronous display:

2.3.1 Setting uboot bootargs

Restart the development board, and press recharge during the countdown to enter ubout. You can first look at the previous bootargs configuration:

Just add it on the original basis console=tty1:

setenv bootargs 'console=tty1 console=ttymxc0,115200 root=/dev/nfs nfsroot=192.168.5.104:/home/xxpcb/myTest/nfs/rootfs,proto=tcp,nfsvers=4 rw ip=192.168.5.102:192.168.5.104:192.168.5.1:255.255.255.0::eth1:off'
saveenv

Then restart the development board, and you can see the output information on the screen when the Linux kernel is driven:

Comparing the information output by the serial port, we can see that the screen output is nothing after the sentence Freeing unused kernel memory: 400K (8090e000 - 80972000). There is no prompt to press the Enter key to continue , and there is no display to start the printing of the self-starting hello word test program. This is because some settings have not been completed.

2.3.2 Modify the /etc/inittab file

This modification is used to set up screen to interact as a terminal.

Open the /etc/inittab file in the root file system and add the following line:

tty1::askfirst:-/bin/sh 

After saving, restart the board and plug a keyboard into the USB port of the board. Then you can interact with the board through the keyboard:

Now you can also operate the board on the screen by plugging a keyboard into the board.

Note that the print of the hello word program that was set to start up previously does not appear on the screen because the input of printf is not set in the LCD. We can print on the LCD screen by directing the output to /dev/tty1, such as testing the screen output of hello linux:

echo hello linux > /dev/tty1 

2.4 Other issues

2.4.1 Automatic screen off problem

When the LCD screen is not operated for a period of time, the screen will automatically go black. At this time, you can wake it up by connecting a keyboard and pressing the Enter key (you can also wake it up through the board's ON/OFF button , because this button is also assigned the function of the Enter key).

This time is set in drivers/tty/vt/vt.c in the Linux source code, and the default is 10 minutes (10*60 seconds).

If you want the screen to stay on all the time, you can set the value to 0, re-edit the Linux kernel to get the zImage, and then use the new zImage to start the development board.

If you do not want to modify zImage, another way is to create an application that starts at startup to control the screen from turning off. The content of lcd_always_on.c is:

#include <fcntl.h> 
#include <stdio.h> 
#include <sys/ioctl.h> 

int main(int argc, char *argv[]) 
{ 
  int fd; 
  fd = open("/dev/tty1", O_RDWR); 
  write(fd, "\033[9;0]", 8); 
  close(fd); 
  return 0; 
} 

Compile the program in Ubuntu, and then copy the executable program to the root file system of the board:

arm-linux-gnueabihf-gcc lcd_always_on.c -o lcd_always_on 
cp lcd_always_on ~/myTest/nfs/rootfs/usr/bin/

Then, set the program to start automatically at boot in /etc/init.d/rcS.

After saving, restart the development board and the screen will not turn off automatically.

2.4.2 Screen Brightness Adjustment

The brightness of the screen can also be adjusted. The backlight node in the device tree has 8 levels set, and the brightness can be adjusted in the range of 0 to 7. Enter the following directory to view the current screen brightness:

/sys/devices/platform/backlight/backlight/backlight 

The following command can be used to modify the screen brightness in real time, for example, to change the brightness to 1:

echo 1 > brightness

Summarize

This article introduces the knowledge related to LCD screen driver and conducts experiments. Because the LCD pins of NXP official board are the same as mine, the main modification is to modify the screen parameters of the lcdif node in the device tree.

Through the experiment, the penguin logo can be displayed, and the output information of the board can be directed to the LCD screen display. By connecting the keyboard, the interaction with the Linux board can be realized. Finally, the screen off and brightness adjustment functions are also tested.

okhxyyo

【i.MX6ULL】 Driver development series articles summary entry:

【i.MX6ULL】Driver Development——by DDZZ669 - ARM Technology - Electronic Engineering World - Forum (eeworld.com.cn)

Sub-directories:

【i.MX6ULL】Driver Development 1——Character Device Development Template

【i.MX6ULL】Driver Development 2——New Character Device Development Template

【i.MX6ULL】Driver Development 3——GPIO Register Configuration Principle

【i.MX6ULL】Driver Development 4——Light up the LED (Register Version)

【i.MX6ULL】Driver Development 5——Device Tree Principle and Lighting LED

【i.MX6ULL】Driver Development 6——Pinctrl subsystem and GPIO subsystem light up LED

【i.MX6ULL】Driver Development 7——Key Input Capture

【i.MX6ULL】Driver Development 8——Interrupt Method to Detect Buttons

【i.MX6ULL】Driver Development 9——Linux IO Model Analysis

【i.MX6ULL】Driver Development 10——Blocking & Non-blocking Key Detection

【i.MX6ULL】Driver Development 11——LCD Driver Practice

【i.MX6ULL】Driver Development 12——Capacitive Touch Driver Practice (Part 1)

【i.MX6ULL】Driver Development 13——Capacitive Touch Driver Practice (Part 2)

Fred_1977

The author is very serious and attentive, and has a great spirit! If this program is written by the author himself, the technical level is still very high!

Jacktang

This Framebuffer is a collection of all display-related hardware and software in the system by Linux. It is a good feature