Abstract
This project develops a deep learning system to recognize American Sign Language (ASL), helping bridge communication gaps for deaf and hard-of-hearing individuals. The system combines convolutional and recurrent neural networks to analyze both the spatial positioning of hands and the temporal movement patterns essential for sign language interpretation. Training utilized three complementary datasets featuring static alphabet signs, detailed finger positions, and continuous signing sequences, exposing the system to diverse signing conditions.
Testing with standard cross-validation protocols demonstrated 97.8% accuracy across more than 11,000 previously unseen samples. Built with open-source frameworks, the system can operate on conventional devices without specialized hardware requirements.
The technology processes video input to identify hand regions, extract relevant features, and classify signs in real-time while preserving user privacy through on-device processing. Applications include educational tools, accessibility solutions for public services, healthcare communication support, and personal communication assistance.Field testing in community settings confirmed the system's effectiveness under varied lighting conditions and with signers of different proficiency levels. Technical innovations include optimized hand segmentation pre-processing, custom data augmentation techniques addressing image variability, and efficient models suitable for mobile deployment.
Future development will focus on sentence-level recognition, support for additional sign languages, and integration with existing accessibility tools. This project advances technological inclusion by making digital interactions more accessible to sign language users across multiple contexts.