Argo is an augmented reality application for improving the accessibility of Cooper Hewitt, Smithsonian Design Museum. 
Accessible Museum Experience
This project is directed towards visitors with visual impairments and aims to enable independent exploration, participation, and empowerment through design.
The team is prototyping a mobile application that uses an augmented reality to recognize objects in an exhibit using the phone camera. When an object is detected and identified, the app will then ask the user if they want additional information, such as an audio description.
AR assistive app
Augmented reality is growing field in assistive technology. Image recognition software trained with photos of exhibit objects would recognize an object in view and offer the user a verbal description of the object.​​​​​​​
User Flow
With our concept solidified, we moved on to the task of creating the initial user flows for the app. We progressed through several iterations of the flow and ended with a flow that incorporates the following basic features:

• ADD TO COLLECTION – The ability to save items to “My Collection".
• OBJECT DETECTION – The ability to identity objects in an exhibit through image detection.
• VERBAL DESCRIPTION – The ability to play, pause, and skip verbal description of an object.

User Research
Site visit with a user
The team visited the Cooper Hewitt in the company of Dr. Leona Godin, an accomplished writer, artist, and performer in her own right, to see the museum through her eyes. Leona is legally blind, navigates with the assistance of a white cane, and her partner.
Dr. Leona Godin listening to a Cooper Hewitt curator describe an object in the ‘Scraps’ exhibit
Dr Leona Godin at the World of Radio exhibit at the Cooper Hewitt
Challenges with a Verbal Description Only Approach
Relying on verbal description only is taxing for users – Processing information only through verbal description is mentally tiring. 
Effective and evocative verbal description is difficult – The curators are subject matter experts on the objects in the exhibits, and even they struggled to convey all the information needed using verbal description only. 
We deployed a 10-question online survey to gather feedback from persons who are blind and visually impaired on their experiences in museums, comfort in using a smartphone device, and the types of information they would prefer to receive about objects in an exhibit. In total, we received 11 survey replies. 
Key findings from the survey follow.​​​​​​​
1. Museum Experiences
The majority of participants would prefer a tactile experience; however the second most desired is an audio-based tour and/or smart phone app.

2. Augmented Reality
Most of our participants have either never heard of augmented reality, or have not experienced it.

3.Smartphone Ease of Use
Most of our survey participants indicated they would be comfortable using a mobile app and downloading an app with a large size if it was effective. Most survey participants indicated they would prefer voice control for an application.
Minimum Viable Product
This MVP is for the proof of concept that image recognition and Cooper Hewitt's API can work together. When argo detects an art piece, it pulls the information of the art from the Cooper Hewitt's API with large print and read the info for users. 
We settled on on the combination of Unity, which allows us to make AR app, and Vuforia, an image-recognition augmented reality development kit.
User Testing 
With the accessibility community
We were fortunate to have the opportunity to test our early prototype on experts in the accessibility community. 
We asked users to perform a simple task: see if they could locate two target objects mounted on the wall by pointing the camera on a mobile phone. This is the basic action the fully-finished app will request of users. And below is what we've heard from them.

1. Requirement for a constant feedback mechanism
2. Distracting from navigation - Hands free!
3. Group of Objects: If there are multiple objects in a display case, a good option would be to receive description on a group of the objects.
4. Mental Fatigue: Concentration on just one sense, such as audio feedback, is mentally taxing. 
How It Works
* Users can control all features with their voice, too.
Changing Verbosity: 
To solve the taxing experience with wordy audio description, argo has 'Verbosity' feature to let the user choose either short length or long length of description.
Large Print:
For the people with low vision, argo offers font-size option. Users can set text size depending on their needs and type of vision loss.

Ideas to Explore​​​​​​
Navigational assistance: Create the ability to guide a user to a specific piece of art they are interested in or the ability to detect objects from farther away so that someone with a visual impairment can stand in the center of a room and get information about objects without having to walk from piece to piece
Extending the concept to the gift shop: Implement with the Cooper Hewitt gift shop to enable persons with visual impairment to shop for items for the full experience.
Back to Top