I'm a (hearing) son in an all-deaf family, and am a bit tired to wait for every single service to become accessible.
Why not just be platform-agnostic? It's the same problem that has been going on for non-captioned videos for years.
If a captioning service is going to be exclusive to only Google Meet or Teams, then by definition, it's not inclusive.
So when we come on Zoom with my parents, they use Ava http://www.ava.me to caption what I say with their speakers on.
It looks like a caption bar, floating on top of the screen, so when I share my screen they can still see captions.
My dad started using Ava at work where they use GoToMeeting. They just won't move to Zoom easily so he gets access without having to shift the whole company on it.
The software also separates who says what neatly if you share a link in the chat and people connect to it. Also voices out what's typed on the other end.
I'm going to take this as an excuse to ask some questions about how the hearing impaired use video chat:
How much does the video component help? I'd assume that most modern video chat solutions are adequate for sign language (but you know what they say about assumptions!). I've heard that reading lips over video is difficult. Is this a case where it's possible to read maybe half of what people say, or is it just impossible to read anything?
What happens when a deaf person receives an unexpected call/message when they're not looking at their PC or handheld? Is vibration from a mobile device usually enough? Are there any visual systems for a desktop to signal the user at a distance? (I'm thinking of something like the flashing light alarm clocks.)
Is communication via sign language over video chat feasible with a mobile device? I honestly don't know how much can be communicated with only one hand free. If one hand isn't enough, is propping the device up on a stable surface enough to be able to communicate this way? (Common sense says that at this point everyone would just be texting, but sometimes you just need that human connection.)
I don't have any practical need for any of these answers. It's just something I've been curious about for a while.
I ignore all calls to my phone, my wife has set my voicemail up to say I'm deaf, to text me instead.
I've got a rather expensive system set up to every doorbell, and smoke alarm in the house that also functions as my alarm clock in which it'll vibrate the mattress (just a small circular vibration device to place under the bed).
Lip-reading over video is good enough IF you can hear enough, so because I have a cochlear implant I can do both and that's usually good enough. Without the processor on, I'm profoundly deaf - so video would be quite useless. (thus the reasoning for a vibrating smoke alarm thingy).
I keep my phone on vibrate-only.
At the end of the day I'm a little different because I rely on my cochlear implant and for some people in some communities (see deaf vs Deaf), there's quite a lot of stigma.
We say deaf & hard-of-hearing please, not hearing-impaired.
Lip reading is hard, period. Over video makes it harder when network connection isn’t great, and more people speak, so mostly difficult yeah.
Flashes / vibration are common alerts yes. I prefer flash since you might not touch the surface that vibrates.
And yes one-handed sign over video is very frequent but maybe best for 1-1 or familiar groups because indeed you can lose a bit in precision/fluidity. If you want to be accessible over video, find a stable place and sign with dark colored shirt so contrast is better ;)
Not to belittle the whole PC thing -- I get it: repeat to someone that they are at a disadvantage their whole life and they'll grow up believing it. My girl understands that her hearing makes her different but doesn't believe that it has hindered her life in any way. Having four kids pretty close in age, it's hard not to compare. My hard-of-hearing daughter is the one out of the four that will surprise us every few months with some new thing she decided to learn how to do from YouTube[0].
She's the most self-directed learner of all of my kids. I think she knows the right terminology. I know she doesn't care at all, though, and we've raised all of our kids to assume that people aren't out to offend you.
[0] She sings incredibly well -- something we didn't expect -- so a few years back she wanted to sing for the talent show but she wanted to do so with a puppet. We thought the puppet idea was a distraction and like the good parents we were, we let her demo it for us and didn't pay a whole lot of attention and tried to discourage her from using the puppet. She was super-discouraged and, thankfully, we asked her to do it again. It was at that point that we realized she was singing beautifully and ... not moving her lips at all. I think she was 8.
> We say deaf & hard-of-hearing please, not hearing-impaired.
Whoops, my apologies.
> Flashes / vibration are common alerts yes.
A quick search reveals that there's a mishmash of different ways to use a phone's flash for notifications. Are there any equivalent options for desktop/laptop systems?
In any case, thanks for taking the time to engage with me.
> questions about how the hearing impaired use video chat
I am not deaf or hard of hearing, but have a teenage daughter who is old enough to articulate some of these things. She doesn't sign (used to, and there's reasons beyond "didn't use it" that she doesn't) and what she's told us is:
(1) Video calls are better than audio-only, but just barely. If you watch her with a group of friends, she's not always looking at the person who she's directly communicating with. She uses her vision to keep up with fast moving, multi-party conversation and to fill in missing sounds. She hears sounds well into mid-range tones. Deeper and high-pitched tones are not heard at all, but she can feel the deep tones[0]. Through lousy laptop/phone speakers, it's all mid-to-high tones and -- I get the impression that people sound much different to her over speakers than they do in person, but I have no way to confirm that.
(2) Without closed captioning, she misses a lot. She doesn't read the captioning word-for-word but uses it like lip-reading -- to fill in gaps for sounds she can't hear. This happens more frequently than to me (I have very mild hearing loss; a little worse than a normal person at my age) -- sounds might be heard, but a competing sound ruins the "signal" (signal/noise ratio) more for her than it does for me. Worse, because video conferencing software likes to focus on the speaker, and the latency involved in switching to the speaking is very long, it can make the calls even more confusing.
The second point can't be stressed enough. My daughter is not completely deaf. She doesn't "lip read" in the way you see on TV[1] -- she can hear enough that she does it unconsciously. My daughter (and Mom and Dad) had no idea how much she relied on lip reading and closed captioning (it's on by default on all of our TVs so we don't even notice it). When Zoom meetings and mandatory mask wearing came along, it took very little time for us to notice she was struggling way more than the other three kids. It took about a month before we figured out why.
In her schooling situation right now (online-only public school -- one not setup specifically for COVID), video/audio conferences are somewhat minimal. Teams works good enough for her, and she's adjusted by pre-reading the materials -- basically learning whatever it is that's about to be taught to the other kids -- because it's easier than trying to learn it from the live lesson. If she didn't have to attend these sessions, we wouldn't make her. Thankfully, video/audio/live lessons are very minimal in the program her and my son are enrolled in.
[0] Something I learned (my daughter is more accurately my step-daughter, so I met her when she was about 4 years old) -- she had a terrible fear of fireworks until she was about eleven. We found out this is a really common thing with people who are hard of hearing. After a lot of questioning, it would seem that she feels the "boom" of the explosion but the corresponding sound is very quiet or just missing entirely. Combined with the difference between the sight of the explosion and the time it takes for the sound to reach your body, it resulted in her being startled, constantly.
[1] ala Seinfeld - You couldn't have her spy on someone's conversation without hearing their voice -- while she's better at it than I am, she's still about a time zone away from the actual words that were said.
We're a unique breed: a deep tech (4 PhDs), mission-driven (100,000 deaf/hard-of-hearing users relying on our work) and VC-funded startup (SV Angel, Initialized, Khosla), and we're doubling our team to 15.
Our goal is to improve how we humans connect & communicate with each other using human & artificial intelligence. First through using a patented multi-peer speech recognition technology app that empowers 450M deaf and hard-of-hearing people to have exceptionally accurate captions for any conversation they're a part of. That unique focus is because I (CEO/Founder) grew up in a Deaf family & witnessed how the world was not accessible for them, then cofounded Ava with my CTO Skinner, deaf since age 2.
Specifically, we're hiring makers (engs + designer) who want to be at the very beginning of a rocketship. All jobs are here: http://www.ava.me/jobs but these following 3 positions will be filled quickly (it's my #1 focus):
1) a Full-Stack Developer (~2-5+ years exp), with React and Node background, to take the lead on building out the Ava experience on the web. Really powerful B2C & B2B applications through it, with real-time & scalability challenges. Should care about all the details that makes a first experiences become magical for a new user, and committed to make a huge impact on society.
2) a Backend Developer (~2-5+ years exp) with DevOps, Node, Python skills as their strong suit. Will own designing and scaling Ava's AI infrastructure, supporting mission-critical algorithms for our users.
3) a Product Designer, exp. in B2C & B2B platforms, mobile & web. Open to animations, UI/UX. Will be working with deaf users & probably learning sign language as a result :) Huge role in leading redesign, branding & empowering our community - very active because it's often life-critical for them to use Ava.
Feel free to email me directly: thibault@ava.me if any question, or to apply here: http://www.ava.me/jobs I'll review HN applications personally!
Ava (www.ava.me) | San Francisco and Paris | FullStack, Software & Backend Engineers, Product Designer | ONSITE (in 9 months, can be REMOTE until then) | Positions: http://www.ava.me/jobs
We're a unique breed: a deep tech (4 PhDs), mission-driven (100,000 deaf/hard-of-hearing users relying on our work) and VC-funded startup (SV Angel, Initialized, Khosla), and we're doubling our team to 15.
Our goal is to improve how we humans connect & communicate with each other using human & artificial intelligence. First through using a patented multi-peer speech recognition technology app that empowers 450M deaf and hard-of-hearing people to have exceptionally accurate captions for any conversation they're a part of. That unique focus is because I (CEO/Founder) grew up in a Deaf family & witnessed how the world was not accessible for them, then cofounded Ava with my CTO Skinner, deaf since age 2.
Specifically, we're hiring makers (engs + designer) who want to be at the very beginning of a rocketship.
All jobs are here: http://www.ava.me/jobs but these following 3 positions will be filled quickly (it's my #1 focus):
1) a Full-Stack Developer (~2-5+ years exp), with React and Node background, to take the lead on building out the Ava experience on the web. Really powerful B2C & B2B applications through it, with real-time & scalability challenges. Should care about all the details that makes a first experiences become magical for a new user, and committed to make a huge impact on society.
2) a Backend Developer (~2-5+ years exp) with DevOps, Node, Python skills as their strong suit. Will own designing and scaling Ava's AI infrastructure, supporting mission-critical algorithms for our users.
3) a Product Designer, exp. in B2C & B2B platforms, mobile & web. Open to animations, UI/UX. Will be working with deaf users & probably learning sign language as a result :) Huge role in leading redesign, branding & empowering our community - very active because it's often life-critical for them to use Ava.
Feel free to email me directly: thibault@ava.me if any question, or to apply here: http://www.ava.me/jobs
I'll review HN applications personally!
Phone calls - try RogerVoice app to have them transcribed for you. It's pretty easy but not free.
Ava - we focus on any in-person discussions, including meetings. The trick with group conversations is the cross-talk, and people speaking fast/casually. We developed over the last 3 years an AI that eliminates these issues to make it work exactly for this situation. Others will need to download the app (Ava), but it's free to join a host, and hosts get 5 hours every month to try the app 100% free.
Regarding the embarrassing situation of having to ask others, I see this a lot - that's really normal.
First, know that you can use Ava by itself with your phone, a bit like a "talking stick". It can look like you're checking your phone when you are actually checking what the person actually said if what you heard.
Just hold your phone within 1-2 meters from the speaker.
If you have a Bluetooth microphones, those make Ava really awesome: you can find an earpiece or a clip on (we recommend a few here: www.ava.me/store) so you can follow presentations/meetings when one person speak for a long time and is hard to lip read. Airpods or such work too, and of course wired microphones (for lunch for example).
For more situations, I really encourage you to read our FAQ: http://help.ava.me - we list like 95% of daily situations a deaf/hard-of-hearing person finds themselves into and how Ava can help there.
PS: feel free to give us any feedback on this, we're all ears & eyes :)
Awesome, let us know via the app how you find it.
The key is for group conversations: multiple people talking, which is really dope for accuracy vs. your understanding, is my guess :)
hi tsm, founder of Transcense here. beyond the impact we want to do, we always at some point built on top of others/existing technologies. Innovation definition is tricky. Is it in the technical implementation (an Instagram is not that complicated after all) or in the productization/distribution to market?
We're humbled to have been posted on HN, not from us. But stay updated, what's coming next will be even more interesting.
So when we come on Zoom with my parents, they use Ava http://www.ava.me to caption what I say with their speakers on. It looks like a caption bar, floating on top of the screen, so when I share my screen they can still see captions.
My dad started using Ava at work where they use GoToMeeting. They just won't move to Zoom easily so he gets access without having to shift the whole company on it.
The software also separates who says what neatly if you share a link in the chat and people connect to it. Also voices out what's typed on the other end.