[Jia Nan CanMV K230] Evaluation ⑦ Line segment and various shape detection

不爱胡萝卜的仓鼠

[Jia Nan CanMV K230] Evaluation ⑦ Line segment and various shape detection [Copy link]

This post was last edited by Hamster who doesn't like carrots on 2024-10-5 17:26

The most basic image recognition is the recognition of various lines and shapes. Let’s complete this first step today.

The basic framework of the code used today is the same as the code in the previous article (but I removed the frame rate code, which is useless to me and consumes resources). If nothing goes wrong, this will be a basic template for subsequent tests. Since using the previous 1920*1080 resolution and adding image recognition code will cause insufficient memory, I lowered the resolution to 320x240 recommended by the tutorial. The specific code is as follows:

import time, os, sys

from media.sensor import *
from media.media import *
from media.display import *

def camera_test():
    print("camera_test")

    #用默认配置构造一个Sensor对象
    sensor = Sensor()
    #复位摄像头
    sensor.reset()
    
    #设置摄像头水平翻转
    sensor.set_hmirror(False)
    #设置摄像头垂直翻转
    sensor.set_vflip(False)

    #设置指定通道的输出图像尺寸, 320x240
    sensor.set_framesize(width=320, height=240)
    #设置指定sensor设备和通道的输出图像格式
    sensor.set_pixformat(Sensor.RGB565)

    #通过IDE缓冲区显示图像
    Display.init(Display.VIRT, sensor.width(), sensor.height())
    
    #初始化media manager
    MediaManager.init()
    #摄像头开始工作
    sensor.run()

    try:
        while True:
            os.exitpoint()
            
            #摄像头拍一张照片
            img = sensor.snapshot()
            Display.show_image(img)

    except KeyboardInterrupt as e:
        print("user stop: ", e)
    except BaseException as e:
        print(f"Exception {e}")
        
    #摄像头停止工作
    sensor.stop()
    # deinit display
    Display.deinit()
    os.exitpoint(os.EXITPOINT_ENABLE_SLEEP)
    time.sleep_ms(100)
    #media manager去初始化
    MediaManager.deinit()

if __name__ == "__main__":
    os.exitpoint(os.EXITPOINT_ENABLE)
    camera_test()

1. Line detection

There are two functions for line detection, find_lines and find_line_segments. Their links in the official documentation: https://developer.canaan-creative.com/k230_canmv/main/zh/api/openmv/image.html#find-lines、https://developer.canaan-creative.com/k230_canmv/main/zh/api/openmv/image.html#find-line-segments

The former is to find a straight line, and the latter is to find a line segment.

Let's use it to find line segments here.

The code is as follows

import time, os, sys

from media.sensor import *
from media.media import *
from media.display import *

def camera_test():
    print("camera_test")

    #用默认配置构造一个Sensor对象
    sensor = Sensor()
    #复位摄像头
    sensor.reset()
    
    #设置摄像头水平翻转
    sensor.set_hmirror(False)
    #设置摄像头垂直翻转
    sensor.set_vflip(False)

    #设置指定通道的输出图像尺寸, 320x240
    sensor.set_framesize(width=320, height=240)
    #设置指定sensor设备和通道的输出图像格式
    sensor.set_pixformat(Sensor.RGB565)

    #通过IDE缓冲区显示图像
    Display.init(Display.VIRT, sensor.width(), sensor.height())
    
    #初始化media manager
    MediaManager.init()
    #摄像头开始工作
    sensor.run()

    try:
        while True:
            os.exitpoint()
            
            #摄像头拍一张照片
            img = sensor.snapshot()
            
            ###################图像识别代码#######################
            #循环找线段，把所有线段找出来
            for l in img.find_line_segments(merge_distance = 10, max_theta_diff = 15):
                #在画面上画线，把找到的线段画出来
                img.draw_line(l.line(), color = (255, 0, 0), thickness=2)
            ###################图像识别代码#######################
            
            Display.show_image(img)

    except KeyboardInterrupt as e:
        print("user stop: ", e)
    except BaseException as e:
        print(f"Exception {e}")
        
    #摄像头停止工作
    sensor.stop()
    # deinit display
    Display.deinit()
    os.exitpoint(os.EXITPOINT_ENABLE_SLEEP)
    time.sleep_ms(100)
    #media manager去初始化
    MediaManager.deinit()

if __name__ == "__main__":
    os.exitpoint(os.EXITPOINT_ENABLE)
    camera_test()

I drew three horizontal, vertical and diagonal line segments on paper, and successfully recognized them by adjusting two parameters.

2. Rectangle Detection

Rectangle detection function official documentation: https://developer.canaan-creative.com/k230_canmv/main/zh/api/openmv/image.html#find-rects

The code is as follows:

import time, os, sys

from media.sensor import *
from media.media import *
from media.display import *

def camera_test():
    print("camera_test")

    #用默认配置构造一个Sensor对象
    sensor = Sensor()
    #复位摄像头
    sensor.reset()
    
    #设置摄像头水平翻转
    sensor.set_hmirror(False)
    #设置摄像头垂直翻转
    sensor.set_vflip(False)

    #设置指定通道的输出图像尺寸, 320x240
    sensor.set_framesize(width=320, height=240)
    #设置指定sensor设备和通道的输出图像格式
    sensor.set_pixformat(Sensor.RGB565)

    #通过IDE缓冲区显示图像
    Display.init(Display.VIRT, sensor.width(), sensor.height())
    
    #初始化media manager
    MediaManager.init()
    #摄像头开始工作
    sensor.run()

    try:
        while True:
            os.exitpoint()
            
            #摄像头拍一张照片
            img = sensor.snapshot()
            
            ###################图像识别代码#######################
            #循环找矩形，把所有矩形找出来
            for r in img.find_rects(threshold = 10000):
                #把矩形画到图片上
                img.draw_rectangle(r.rect(), color = (255, 0, 0),thickness=2)
            ###################图像识别代码#######################
            
            Display.show_image(img)

    except KeyboardInterrupt as e:
        print("user stop: ", e)
    except BaseException as e:
        print(f"Exception {e}")
        
    #摄像头停止工作
    sensor.stop()
    # deinit display
    Display.deinit()
    os.exitpoint(os.EXITPOINT_ENABLE_SLEEP)
    time.sleep_ms(100)
    #media manager去初始化
    MediaManager.deinit()

if __name__ == "__main__":
    os.exitpoint(os.EXITPOINT_ENABLE)
    camera_test()

3. Circle detection

Rectangle detection function official documentation: https://developer.canaan-creative.com/k230_canmv/main/zh/api/openmv/image.html#find-rects

The code is as follows:

import time, os, sys

from media.sensor import *
from media.media import *
from media.display import *

def camera_test():
    print("camera_test")

    #用默认配置构造一个Sensor对象
    sensor = Sensor()
    #复位摄像头
    sensor.reset()
    
    #设置摄像头水平翻转
    sensor.set_hmirror(False)
    #设置摄像头垂直翻转
    sensor.set_vflip(False)

    #设置指定通道的输出图像尺寸, 320x240
    sensor.set_framesize(width=320, height=240)
    #设置指定sensor设备和通道的输出图像格式
    sensor.set_pixformat(Sensor.RGB565)

    #通过IDE缓冲区显示图像
    Display.init(Display.VIRT, sensor.width(), sensor.height())
    
    #初始化media manager
    MediaManager.init()
    #摄像头开始工作
    sensor.run()

    try:
        while True:
            os.exitpoint()
            
            #摄像头拍一张照片
            img = sensor.snapshot()
            
            ###################图像识别代码#######################
            for c in img.find_circles(threshold = 3500, x_margin = 10, y_margin= 10,
                                              r_margin = 10,r_min = 2, r_max = 100, r_step = 2):
                #画红色圆做指示
                img.draw_circle(c.x(), c.y(), c.r(), color = (255, 0, 0),thickness=2)
            ###################图像识别代码#######################
            
            Display.show_image(img)

    except KeyboardInterrupt as e:
        print("user stop: ", e)
    except BaseException as e:
        print(f"Exception {e}")
        
    #摄像头停止工作
    sensor.stop()
    # deinit display
    Display.deinit()
    os.exitpoint(os.EXITPOINT_ENABLE_SLEEP)
    time.sleep_ms(100)
    #media manager去初始化
    MediaManager.deinit()

if __name__ == "__main__":
    os.exitpoint(os.EXITPOINT_ENABLE)
    camera_test()

I put a one-yuan coin on the paper and it was recognized successfully

4. BUGs that occurred during the process

When I first started testing, I encountered a very strange BUG. When the code stopped running, even if the deinitialization code had been called, as long as there was image recognition code during this running, such as line detection, as soon as the line detection code was run, the log reported "Exception Out of fast frame buffer stack memory", as shown in the figure below

It means "exceeded fast frame buffer stack memory", in simple terms, the memory is blown. But our development board has 1G memory, so no matter what, it will not be said that the memory is insufficient.

I checked online, and it basically said that the code consumes too much memory and needs to downgrade the resolution. However, I tried to adjust the resolution to the smallest 88x72 QQCIF and turned off the display, but it didn't work. When I ran the line detection function, an error message was reported. If I removed this sentence, it would work normally. I spent almost an hour trying to figure it out, but I couldn't get any results. I was so upset.

The final solution is: restart (press the RST button on the development board). . . . . . . As expected, restart when encountering problems