International Conference on Information and Knowledge Technology

Home / چهاردهمین کنفرانس بین المللی فناوری اطلاعات و دانش

Persian deaf sign language recognition system using deep learning

Authors :

Mohammad Ebrahimi¹

1- دانشگاه آزاد اسلامی واحد هرند (اصفهان)

Keywords :

Deep learning،Machine vision،Dynamic programming،Sign language،Tracking

Abstract :

Vision based communicating with compare of speech-based communicating is more complex and meaningful. Direct communication between deaf and other people is very difficult, so there are attempts for making a sign language interpreting system. Deep learning uses linear and nonlinear transformations to model the concepts in the data hierarchically. Deep learning also learns long time dependencies in sequential frames. A deep network has been created that has trained 14 words, and recognizes that words with its speed during playback. In this paper, data are given to two distinct systems and the results are compared with each other. The first system consists of three basic stages of hand and head tracking, feature extraction, and finally identification. Dynamic programming is used for tracking, that it is very timely. Right hand tracking is an important step. Possibility of placing two hands together causes the tracking system to be erroneous and occlusion. Dynamic algorithm, for hand tracking, has been explained. Difference frames has been used as score function and movement between two frame times has been used for penalty function in dynamic programming. Experiments from deaf news, has been tested, and well able to track right hand. The second system is a deep learning network with multiple layers that recognizes sign language word in a much higher speed than the first system.