Jan 2022 - Jun 2022
Street-Crossing Guide for Blind People
Developed a wearable assistive device using the RaspberryPi platform that is able to detect pedestrian traffic lights and aid street-crossing.
(Team project under NYU EG1004)
There is a wide array of assistive technology for visually-impaired people on the market. However, very rarely do them focus on the use-case of street-crossing. Often, street-crossing by a visually impaired individual is accomplished in the following steps: 1) identifying an intersection with the help of curbs or tactile pavings, 2) making references to their mental maps and environmental cues to deduce the current road structure, 3) acquiring information about road traffic and the time between two stoplights through many light cycles, and 4) finally, attempting to cross the street. However, no matter how carefully planned out this meticulous maneuver is, one is still vulnerable to the unpredictable traffic patterns caused by left turns, pushbuttons, and stopsign-regulated intersections. Conventional methods, like white canes or guide dogs, could not address this situation in a reliable way, leading to this product idea.
The product is a wearable device with a built-in camera, speaker, and microcontroller that is able to 1) respond correctly to pre-trained pictures of pedestrian traffic lights and identify their color, 2) relay that information to the user through auditory feedback, and 3) help the user to stay on the designated safe street-crossing path through a course correction algorithm.
The wearer is able to elicit the control loop through a button and hear the sound feedback through the speaker. The device is able to recognize the light post from background distractions and classify images into two categories, white and red, with an average accuracy of 65% on naive data and up to 98% on the trained images with a detection interval of 5 seconds.
By developing this product, we want to encourage people with visual impairments to go out more and participate in social scenarios like everyone else.
User wearing the device
Set-up with components labeled
Boundary boxes drawn around traffic lights for course correction
RaspberryPi can be used without connection to keyboard and monitor