[K230 Embedded AI Development Board Review] + Simple Sign Language Recognition (Part 1)

jinglixixi

[K230 Embedded AI Development Board Review] + Simple Sign Language Recognition (Part 1) [Copy link]

 

Sign language is the basic way for hearing-impaired people to communicate. It is a special form of language that mainly expresses semantics through changes and movements of gestures.

FIG. 1 is a schematic diagram showing characters in sign language, and FIG. 2 is a schematic diagram showing numbers.

Figure 1 Character diagram

Figure 2 Digital schematic

With the help of the palm and key point detection function provided by CanMV K230 AI vision, it can provide certain support for sign language recognition.

In order to detect and identify key points, the palm must be detected first, and then the positions of the finger joints are extracted for classification and identification.

The processing flow of palm detection is as follows:

When the palm detection program is run, the detection result is shown in FIG3 .

Figure 3 Palm detection

The process flow for performing key point detection is as follows:

When the key point detection program is run, the detection result is shown in FIG4 .

Figure 4 Key point detection

Based on the extraction of key points, the gesture can be interpreted as follows:

The test results of gesture recognition are shown in Figures 5 to 7, and three gestures can be recognized.

Figure 5 Fist recognition

Figure 6 yeah recognition

Figure 7 Five recognition

After the right hand recognition is completed, the recognition results for the left hand are shown in FIGS. 8 and 9 , indicating that the recognition for the left hand is still effective.

Figure 8 Fist identification

Figure 9 yeah recognition

Later we will explore how to define gestures to expand the recognized content and lay a solid foundation for simple sign language recognition.

wangerxian

What is the video frame rate that can be achieved when the K230 runs the gesture recognition model?

jinglixixi

wangerxian published on 2024-10-15 18:21 What is the video frame rate that can be achieved when K230 runs the gesture recognition model?

The video frame rate is around 8.634869

wangerxian

jinglixixi posted on 2024-10-15 21:05 The video frame rate is around 8.634869

That's not bad, it's good to do some control instructions.

jinglixixi

wangerxian posted on 2024-10-16 09:14 That's not bad, it's good to make some control instructions.

Yes, this board is really good and has both research and practical value!