How It Works
- Video Capture: Records sign language gestures using your camera
- Vision Processing: Uses a Vision Transformer model to analyze hand movements and gestures
- Word Recognition: Identifies individual sign language words with confidence scores
- Sentence Construction: An NLP transformer converts recognized words into coherent sentences
- Speech Synthesis: Converts the generated text to speech for audio output
Features
- Real-time sign language recognition
- Confidence scoring for predictions
- Intelligent sentence formation
- Customizable text-to-speech settings
- Support for multiple voices and languages
- Responsive design for all devices
Technology Stack
This application demonstrates a complete AI pipeline using modern web technologies, computer vision, and natural language processing to bridge communication gaps for the deaf and hard-of-hearing community.
