Assisting deaf passengers with train travel: Introducing AI sign language

“….” Silence is all that can be heard as people around you on the station platform suddenly flock to a different location. Some are checking their watches or phones, others are arguing amongst themselves. “…..” Still all you can hear is silence.

This is the experience that deaf individuals who have missed audio-based station announcements will encounter. An experience full of uncertainty, anxiety, and vulnerability as they are left on station platforms unsure as to the information that other passengers have received.

These situations result in deaf passengers looking for the closest information board so they are able to understand the situation and if their journey has been impacted. To combat these stressful situations faced by passengers, experts are currently working on AI technology which will allow sign language to be presented via an avatar that can translate sentences and instructions into animations.

 

The power of sign

The project, which is currently at prototype stage, sees the development of an avatar which can translate simple sentences and instructions into animations. Prior to involvement in the rail industry the avatar was tested by healthcare professionals and patients who were unable to bring their interpreter with them into Covid-19 isolation wards. This technology allowed deaf patients to have sentences signed to them by the avatar such as “where are you feeling pain”?

The project is part of SignLab Amsterdam – a cross facility research lab at the University of Amsterdam which aims to develop technology to translate text into sign language using animated avatars – who want to further develop this technology to support parents of deaf children, assist in airport environments and much more.

Although the technology is making headway in its development its aim is not to replace sign language interpreters but to aid and assist deaf individuals where situations are not feasible for a human interpreter to be present.

Dr Floris Roelofsen, associate professor at the university’s institute for logic, language and computation and founder of SignLab began exploring this area of study after his daughter was born deaf which saw him learning sign language early on to be able to communicate with her. He hopes this project will not only assist deaf passengers in a variety of environments but will also heighten awareness of the situation.

 

 

“It raises awareness and understanding of this phenomenon. The awareness and the understanding of this situation among the general population seems to be very low. In the Netherlands around 8% of the population have disabling hearing loss. I think even the people who have disabling hearing loss think that they’re the only ones, whereas in fact it’s a huge group.”

“I think it would help to break the barrier for everyone. To break the isolation that deaf people and hard of hearing people experience. Awareness and understanding are a very welcome side effect of implementing and using this technology.”

“I think it would help to break the barrier for everyone. To break the isolation that deaf people and hard of hearing people experience. Awareness and understanding are a very welcome side effect of implementing and using this technology.” Roelofsen said.

“Governments, companies, and anyone providing services should take these passengers into account, certainly governments. If they’re providing a service that is paid out of tax money that is also paid by deaf people, then they should have equal access to those services.”

 

Current railway trials

The technology is currently being discussed with Dutch railway company, NS, who have their own passenger app that updates passengers with platform changes, delays, and other announcements.

Although the company have their own apps that provide passengers with information, last minute quick changes are not always updated instantly which sees information being presented to passengers in an audio format.

These last-minute messages are the area which Roelofsen wishes to focus on: “It’s the last-minute messages that we want to translate and then generate. When there is an announcement at the train station they would get a translation on their phone in sign language.”

“There would be a buzzer, indicating that there is a new message. That way they would be up to date on information that was coming in that couldn’t have been made available through more standardised written text.”

Alongside the last-minute announcements another area of focus is looking into when trains get stuck or stop on tracks. In this situation the passengers need to know why the train is not proceeding which generally sees announcements being made over the on-board tannoy system. Roelofsen hopes that as the technology develops situations like this will see deaf passengers being alerted as to what is happening via the app.

Roelofsen explained: “When the speaker is used to give this information there are standardised messages that also appear on screens in the train, but those are very generic messages – they don’t have very specific information. With this technology we could provide more specific information for deaf passengers about how long the delay is, what the cause is, how long it’ll say stationary, if there’s any advice.”

 

Stressful rail travel: Naomi’s experience

“It makes me really anxious travelling as a deaf person, and I think that’s an experience that’s shared by a lot of deaf people particularly in rail travel because things do change.”

PHD student Naomi Smart has been deaf since birth on her right side which has resulted in her relying on lip reading and captions going about her day-to-day life. However, due to announcements in railway stations being primarily audio based this can result in her missing important announcements – this has caused Naomi to alter the way she travels to ensure announcements are not missed. “It makes me really anxious travelling as a deaf person, and I think that’s an experience that’s shared by a lot of deaf people particularly in rail travel because things do change.”

 

 

“If I know I’m going to be travelling somewhere by train I won’t read a book because I’ll be looking around and I won’t listen to music because I could miss something. I’m sort of like on edge every time I’m in a train station because I really want to make sure I don’t miss anything.”

The development of this technology aims to assist passengers such as Naomi to ensure they have a stress-free, relaxing travel experience without the worry of missing audio-based announcements. Discussing the technology Smart said: “I think it’s great technology, it’s really important and I think above anything else, it’s really nice to have that sort of visibility, and being considered and being thought about.”

With 11 million people with the UK being deaf or hard of hearing the question is raised as to why more inclusive technology or services have not been included sooner within the rail environment. Issues of funding, outdated systems and technological advances only occurring within the last few years are key players when it comes to this discussion; however, Smart stated that in her experience the deaf community are often seen as a minority.

“Quite often in my experience I find that the deaf community and disabled people are often expected to just sort of cope. We’re seen as a minority, so we don’t fit with the norm and we’re expected to just sort of make it work.”

“Something that I’ve often noticed is the emphasis is often on me to make things work for me. Particularly with travelling I think a lot of deaf and disabled people find that people sort of expect you to be travelling with someone. I often have people saying: “Well why don’t you just take someone with you.” Why can’t I just have an independent train journey on my own without someone else being there?”

Looking into the future Smart hopes that she will be able to take a train and for it not to be a stressful, anxiety inducing experience where she has to pre-plan the entire journey but something that she can look forward to and enjoy.

“For me as a deaf traveller I want to go and take a train and not be worried. I would like it to be fully accessible.”

“For me as a deaf traveller I want to go and take a train and not be worried. I would like it to be fully accessible. I would like to be able to have the knowledge that with any announcements I would also to be able to maybe read or have a sign language interpretation. I think equal access is the main thing, I would like to be reassured and know that I’m going to be able to access everything that a hearing person is.”

“I know we’re a minority, but we exist, we’re here and we deserve an accessible public service.”

The post Assisting deaf passengers with train travel: Introducing AI sign language appeared first on Railway Technology.