How AI is making the digital world a more accessible place

How AI is making the digital world a more accessible place

There are 1.3 billion people with disabilities in the world, that is one in seven individuals. An estimated 70 percent of these people have non-apparent disabilities, which means they go largely unnoticed and hence are not often addressed. Every one of us will face some type of disability at some point in our lives, be it temporary, situational, or permanent.

“Technology can empower people to achieve more, help strengthen education opportunities, and make the workplace more inviting and inclusive for people with disabilities. And with more than one billion people with disabilities in the world, Microsoft believes accessibility and inclusion are essential to delivering on our mission to empower every person and every organization on the planet to achieve more,” says Jenny Lay-Flurrie, Chief Accessibility Officer at Microsoft.

Take for instance, Windows 10, which offers several built-in capabilities that make it easier for people with disabilities to access content. For example, the Ease of Access settings in Windows 10 offers features like Narrator that reads aloud text, Magnifier that enhances the size of text and images, and a high-contrast mode to assist users with vision impairments.

Nidhi-768x902For Nidhi Jain, Program Manager – AI Products Cortana at Microsoft India, User Interface (UI) designs are easier to see with more distinct colors in the high-contrast mode. Jain was diagnosed with retinitis pigmentosa at the age of 17, a degenerative, incurable genetic disorder that causes varied degrees of loss of peripheral vision and night blindness.

“Product designs have to be pixel-perfect. Comprehending images and designs are a challenge for me since I have a partial visual impairment. I work around it by using a high-contrast black mode which helps the text and images stand out,” she explains.

For people who are deaf or hard of hearing, the closed captions feature in the Ease of Access settings makes the device easier to use without sound by displaying audio as text. Users can also choose to show audio alerts or notifications visually.

Applying AI to improve accessibility

We believe that AI can be a force multiplier in unlocking solutions to some of the biggest challenges facing people with disabilities. Our AI services have reached 96 percent human parity in object recognition, 88.5 percent in machine reading comprehension, 69.9 percent in machine translation and the Switchboard speech recognition benchmark for human parity at 5.1 percent. Having reached parity with an accuracy at par with humans, we’re also applying AI capabilities to assist and empower people with disabilities, helping make their workplace and everyday life more inclusive.

Here’s a look at AI-enabled services, apps, and features, that are helping people with disabilities unlock their potential.

automatic-image-description-768x511Intelligent Alternative Text in Word and PowerPoint: While alternative text started primarily as an attribute in HTML code to provide image context to search engine crawlers helping them index an image properly, we are leveraging AI to make visual content more accessible for people with visual impairments. Using the same technology that powers PowerPoint’s Designer feature, Microsoft Computer Vision Cognitive Service, users can now get automatic alternative text for images in both PowerPoint and Word.

For Shriram Parthasarathy, Social Media Lead at Microsoft India, intelligent alternative text is an invaluable feature. Diagnosed with macular degeneration during early childhood, he uses intelligent alternate text while running Microsoft’s social media and digital networks.

“Being a marketer, one thing you cannot live without is the presence of visual content, both in terms of static images and motion graphics. In my daily routine, one of the biggest challenges is interpreting images and graphics for what they are and how they will end up. Intelligent image descriptions help me interpret every image,” he reveals.

Shriram Parthasarthy, Social Media Lead, Microsoft India, working on his laptop

Seeing AI: Apps like Seeing AI have unlocked new capabilities for people with low-vision or blindness helping them navigate their surroundings like never before. Powered by Microsoft’s Cognitive Services APIs, Seeing AI essentially leverages machine vision and text-to-speech systems to recognize and understand the facial expressions of friends and family, read out sentences, identify colors, currencies and more.

Apart from the obvious use cases, Shriram also relies on Seeing AI to access images on Twitter. “I use Seeing AI to interpret images on tweets as they happen without any dependency. Even when I’m on the move I’m able to pick up an image and use Seeing AI to describe it for me,” he reveals.

Dictate: A helpful feature in Windows 10, Dictate enables hands-free typing. Powered by Microsoft’s speech-to-text technology, Dictate has been making it easier for people with low or no vision to type their emails, write blogs and more. The feature works in any text field either in Windows 10 or in an app, which users can invoke by pressing Windows key and H simultaneously.

Jain often uses it to dictate her responses to emails, instead of using a keyboard to type in every letter. “I think Dictate is a powerful tool, though it was not developed specifically for someone with a visual impairment. I use it because I can just speak to my computer and it types everything on the screen. I respond to a lot of emails using Dictate, especially when I’m having a tiring day,” she adds.

RELATED: Design for disability, empower everyone

Live captions and subtitles: We realize the growing need for captions. Not only do they empower those of us who are deaf or hard-of-hearing, but everyone benefits when captions are included—from diverse groups and organizations where multiple languages are spoken to helping people in a loud room from being left out, the benefits are endless.

Live-sub-titles-for-inclusive-presentationsLive captions and subtitles are now available in Skype and PowerPoint. Skype users can turn these on for a single call or choose to see them on every call they are part of, whether it’s a one-on-one or group chat.

PowerPoint users will also be able to display captions and subtitles in the same language or in a different one with Microsoft Translator.

In addition, we also announced that live captions and subtitles are coming to Teams, helping to make your meetings more inclusive for all attendees.

Microsoft Stream also auto transcribes the audio using our speech to text capabilities, which makes it easier for people who are deaf or hard of hearing to access video content.

Background blur in Teams and Skype: What is now considered to be an important feature for people who work remotely, background blur in Teams and Skype actually originated from a Microsoft engineer’s need to be connected to her parents in India.

Screenshot of an open computer put on a table
Background blur feature in Teams and Skype was originally created by Swetha Machanavajhala who wanted to be able to read the lips of her parents without distraction on Skype calls

Swetha Machanavajhala, deaf since birth, had just moved to the U.S. and was trying her best to stay in close touch with her parents back home, calling them on Skype every week. But their internet connection in India was poor, and she struggled to read their lips over the glitchy video. She always had to ask her parents to turn off the lights in the background to help her focus better on their faces.

“I kept thinking, ‘Why can’t we build technology that can do this for us instead?’” Machanavajhala recalls. “So I did.”

It turned out her background-blurring feature was useful for privacy reasons as well, helping to hide messy offices during video conference calls or curious café customers during job interviews. So Machanavajhala’s innovation was integrated into Microsoft Teams and Skype.

Tell Me: One of the challenges that people with low vision face is to reach the command they need, thus slowing them down. Tell Me lets you quickly access commands in several Office 365 applications without navigating the command ribbon. You can use Tell Me to assist with formatting, discover the difficult-to-find capabilities and even get scoped help in Office 365 using everyday language. Jain vouches for this feature in her daily work life.

“As a program manager, I’m expected to create long product spec documents, which must be formatted properly for designers, coders and everyone else in the team. With a simple ‘Alt + Q’ on the keyboard, the Tell Me tool lets me add bullet points, change the font and do everything else I need to make a presentable document using Microsoft Office,” she reveals.

These are just a few examples of how AI is revolutionizing accessibility. Given the right tools, anyone – be it a person with a visual, hearing, speech, mental, cognitive or mobility-related disability – can achieve more. You can find out more about our accessibility tools for people with other disabilities here.

Contributing to a more accessible world

At Microsoft, our commitment to accessibility is not limited to our products. We are supporting developers, NGOs, academics, researchers and inventors to bring their accessibility ideas to life. The AI for Accessibility program is an endeavor to provide access to cutting-edge Microsoft Azure cloud computing resources and make it easier to bring accessible products and services to the market.

You too can build on the latest advancements in AI and Machine Learning to develop accessible and intelligent solutions that see, hear, speak, understand and better interpret people’s needs.