IYRC PAPER ARCHIVE
  • FRONT
  • PAST CONFERENCES
    • IYRC Spring 2023 >
      • Authors
    • IYRC Fall 2022 >
      • Authors
    • IYRC Fall 2021 >
      • Authors
    • IYRC Spring 2021 >
      • Authors
    • IYRC 2020 >
      • Authors
    • IYRC 2019 >
      • Authors
    • IYRC 2018 >
      • Authors
  • PAST SUMMER PROGRAMS
    • IYRC Summer 2022
    • IYRC Summer 2021
  • IYRC HOME PAGE

LangAI: Real-Time Sign Language Recognition and Translation using Convolution Neural Networks and Image Processing Algorithms

SHRIMALI, Samyak
DOI: http://doi.org/10.34614/2022IYRC56
Category: STEM
Abstract – People with hearing disabilities have extreme difficulties with communication and are often times unable to share information, ideas, and emotions. Currently, the most effective way to bridge this communication barrier is through the use of hand gesture-based sign language. But using hand gestures requires sign language interpreters to help translate hand signals into spoken word, and if a sign language interpreter is unavailable, people with hearing disabilities remain unheard to the rest of the population. This research aims to develop, LangAI, a smart mobile application that efficiently and accurately translates American Sign Language into the English alphabet in real-time to reduce the dependence people with hearing disabilities have on sign language translators. This application utilizes a fine-tuned VGG-16 CNN model that was found to be the most optimal after a comparative analysis of five different CNN architectures where it possessed the highest validation accuracy of 96.14% and F1 score of 0.975. LangAI’s supplemental features include a learning tab where users are connected to open-source sign language learning resources and a mindfulness tab where users can learn fun facts about this history of sign language and other communication methods. LangAI has the ability to significantly aid people with hearing disabilities around the world. It solves all of the shortcomings of current solutions and research to create direct positive human impact which has the potential to progressively improve the lives of countless people who are often marginalized due to their disability. 
​
  • PAPER
  • PRESENTATION VIDEO
<
>
Download PDF
  • FRONT
  • PAST CONFERENCES
    • IYRC Spring 2023 >
      • Authors
    • IYRC Fall 2022 >
      • Authors
    • IYRC Fall 2021 >
      • Authors
    • IYRC Spring 2021 >
      • Authors
    • IYRC 2020 >
      • Authors
    • IYRC 2019 >
      • Authors
    • IYRC 2018 >
      • Authors
  • PAST SUMMER PROGRAMS
    • IYRC Summer 2022
    • IYRC Summer 2021
  • IYRC HOME PAGE