Apple's new accessibility feature lets Vision Pro users control a wheelchair with their eyes

Source: Engadget

Apple is previewing new accessibility features powered by Apple Intelligence, including natural‑language voice input and a Vision Pro app that lets users control motorized wheelchairs with their eyes. The updates were announced ahead of Global Accessibility Awareness Day (GAAD) on May 21 and are expected later this year.

Image Explorer & VoiceOver

The Image Explorer in VoiceOver now leverages Apple Intelligence to generate richer, more detailed descriptions of images across the system—covering photos, scanned bills, personal records, and other content that often lack thorough alt‑text.

Live Recognition has also been updated: users can press the Action button on iPhone to ask a question about what’s in the camera viewfinder and receive a detailed response, with the ability to ask follow‑up questions.

Magnifier

Magnifier now incorporates Apple Intelligence to provide visual descriptions within its high‑contrast interface for low‑vision users. The Action button can be used to quickly obtain answers to questions and to control the app with spoken commands such as “zoom in” or “turn on the flashlight.”

Voice Control (Natural Language)

Apple’s Voice Control gains a natural‑language input mode. Users can describe on‑screen buttons and controls in plain language instead of memorizing exact labels or numbers. This works across any app, including those with visual layouts like Apple Maps or Files. Example commands include:

• “Tap the guide about best restaurants.”
• “Tap the purple folder.”

The feature also helps when app elements are not properly labeled for accessibility.

Accessibility Reader

Accessibility Reader now handles more complex material such as scientific articles, multi‑column text, images, and tables. It can provide on‑demand summaries so readers get an overview before diving into details. A new built‑in translation function lets users read content in their native language while preserving custom formatting, fonts, and colors.

Generated Subtitles

Generated subtitles offer captions for personal videos, content shared by friends and family, and other media—not just videos from platforms like YouTube and Facebook. Subtitles are created on‑device for privacy and appear automatically for un‑captioned videos on iPhone, iPad, Mac, Apple TV, and Apple Vision Pro.

Wheelchair Control Feature (Vision Pro)

Apple introduced a new wheelchair control feature for Vision Pro. Using the headset’s precision eye‑tracking system, the feature provides a responsive input method for compatible alternative drive systems. It requires no frequent recalibration and works under various lighting conditions. Compatibility includes TOLT and LUCI drive systems, with accessory support for Bluetooth and wired connections.

“The option to control my power wheelchair on my own is gold to me,” said Pat Dolan, who has lived with ALS for 10 years and is a founder of GeoALS. “With this new feature, Apple is developing life‑enhancing technology for the people who need it most.”