After watching the video on facial recognition using ML + Meta glasses I started wondering if we could use LLMs + Vision models to help blind people augment their surroundings.
While I admittedly have no experience with the blind or the methods taught to them that assist them in navigating the world, I have a hunch there's potential to leverage the current LLM/ AI stack to improve upon those methods.
Are their any cool companies or open source projects experimenting with this?
https://www.ayes.ai
The leader in the field is BeMyEyes, of course. They've been working with Microsoft to integrate GPT-4o vision models into their app, with some great success. What we haven't seen yet is the move to live-video image recognition that could come from something like an OrCam or Meta glasses (they recently announced a partnership with Meta). I'm guessing there are serious safety issues with the model missing important information and leading someone vulnerable astray.
https://www.bemyeyes.com https://www.bemyeyes.com/blog/be-my-eyes-meta-accessibility-...
OrCam has a new product (woe upon those of us who have the paltry OrCam MyEye2) that the Meta glasses will be competing against at an eye-watering > $4K price point, that seems to do less.
https://www.orcam.com/en-us/orcam-myeye-3-pro
As with the hearing aid industry which recently went over-the-counter causing prices to plummet, the vision aid product category is in temporary disarray as inexpensive new technologies makes their way into a premium-price market.