International Conference on Information and Knowledge Technology

Home / شانزدهمین کنفرانس بین المللی فناوری اطلاعات و دانش

From Faces to Words: An Efficient Persian Visual Lip Reading

Authors :

Mana Amini¹ Sajjad Aemmi² Azadeh Ashouri³ Reza Akhoundzadeh⁴ Kourosh Hassanzadeh⁵ Mohammad Reza Mohammadi⁶

1- PART AI Research Center 2- PART AI Research Center 3- PART AI Research Center 4- PART AI Research Center 5- PART AI Research Center 6- Iran University of Science and Technology

Keywords :

Lip Reading،Visual Speech Recognition،CTC Loss،LSTM،Video-Based Authentication

Abstract :

Visual speech recognition, or lip reading, is the task of transcribing spoken content directly from video frames of a speaker’s mouth without relying on audio. We develop an end-to-end visual lip reading system that processes cropped mouth regions from video sequences and decodes them into text using recurrent neural networks trained with CTC loss. To extend beyond existing English datasets, we collected and manually annotated a new Persian Lip Reading Dataset (PLRD), providing valuable resources for studying morphologically rich languages. Our experiments show that the proposed system achieves competitive word error rates on our custom Persian dataset. Beyond transcription, the model can also be employed in authentication scenarios, where it verifies whether a spoken phrase in a video matches a given reference text. This demonstrates the potential of lip reading systems not only for accessibility and robust speech recognition in noisy environments, but also for secure user verification.