Xuemei
Design
About
Xuemei
Design
About
Xuemei
Design
About

TouchStone

TouchStone

TouchStone

Designing an accessible app

Designing an accessible app

TouchStone is an interactive app for those interested in learning Braille. Fueled by our shared passion for education and accessibility we let this guide us as we designed and developed this app for Google A.I's 2024 Hackathon.

Joining a team of 5 developers, I played a key role in shaping the app's look and feel from the early stages. I was consulted later on to check on UI and usability of their prototype as well as creating a voiceover walkthrough tutorial of the app in which we submitted to Google!

TouchStone is an interactive app for those interested in learning Braille. Fueled by our shared passion for education and accessibility we let this guide us as we designed and developed this app for Google AI's 2024 Hackathon.

I joined a team of five developers in the early stages of the app's development, where I contributed to shaping the concept, enhancing usability, and refining the app's design.

My Role

My Role

Lead UX/UI Designer

Timeline

Timeline

1 Month Sprint

Methods Used

Methods Used

Wireframes, Prototyping

Sketching, Prototyping

Tools Used

Tools Used

Figma

Project

Project

Google A.I Hackathon

Inspiration

Inspiration

Inspiration

The inspiration for TouchStone lies behind a shared passion for education and accessibility, alongside our interests in language and communication. Our motivation to create a learning app for Braille stemmed from a memory shared by our peer as they reminisced about their childhood best friend, who happened to be visually impaired. Despite their differences, they formed an inseparable bond exchanging handwritten notes in Braille by poking holes through pieces of paper. The friend created an entire cheat sheet of Braille representations that they knew in order to preserve that precious connection. Within this story, and our own experiences, we realized the profound impact that accessible communication can have on fostering connections and understanding. This greatly portrayed the importance of inclusive education and the need for accessible resources for those eager to learn. We empathize with and recognize a multitude of things: the emotional journey of individuals facing the risk of visual impairment, the overwhelming feelings that come from times of uncertainty and the connection that can stem from learning. With an enhanced learning experience of instant translations of words and phrases through the integration of AI, TouchStone becomes not only a learning app for those at risk of visual impairment but also a tool to empower educators, parents and caregivers in providing effective support and guidance.

The inspiration for TouchStone lies behind a shared passion for education and accessibility, alongside our interests in language and communication. Our motivation to create a learning app for Braille stemmed from a memory shared by our peer as they reminisced about their childhood best friend, who happened to be visually impaired. Despite their differences, they formed an inseparable bond exchanging handwritten notes in Braille by poking holes through pieces of paper. The friend created an entire cheat sheet of Braille representations that they knew in order to preserve that precious connection.

Within this story, and our own experiences, we realized the profound impact that accessible communication can have on fostering connections and understanding. This greatly portrayed the importance of inclusive education and the need for accessible resources for those eager to learn. We empathize with and recognize a multitude of things: the emotional journey of individuals facing the risk of visual impairment, the overwhelming feelings that come from times of uncertainty and the connection that can stem from learning. With an enhanced learning experience of instant translations of words and phrases through the integration of AI, TouchStone becomes not only a learning app for those at risk of visual impairment but also a tool to empower educators, parents and caregivers in providing effective support and guidance.

What it does

What it does

What it does

TouchStone offers a unique and engaging way to learn Braille. Through the combined use of AI assistance and interactive quizzes, the app creates an enhanced experience that guides users through the basics of Braille, covering the representations of simple numbers, the alphabet, special characters and everyday words that are likely to be encountered.

Our interactive quiz feature includes:

  • FlashCards: Learn letters, numbers, or words, and see their correct Braille translation when you flip the card. Matching: Match six random words, letters, or numbers with their Braille translations to test your memory and recognition. Fillin’ the Braille: Apply what you’ve learned by filling in the Braille characters for specific words, letters, or numbers.

With the integration of Gemini AI in TouchStone’s search bar, you can easily inquire about the Braille translation of any word or letter. Gemini AI will provide a detailed text explanation and an image of the Braille translation, making learning both comprehensive and visually accessible.

How we built it

How we built it

How we built it

Our team of seven was composed of five developers, a graphic designer, and a UX/UI designer. Our first week began with project planning, which included a conversation with UX/UI where developers provided a complete ERD and write up of expectations for our app. From there, the UX/UI team created an interactable figma wireframe alongside the creation of a branding plan. The developers then split into groups, working on front-end with React and back-end with Django. Despite being newcomers within the field of AI, a few of us dedicated 2.5 weeks to comprehend AI modeling and training. Through this process, we explored tools such as the Gemini AI studio, delved into Google AI documentation and experimented with TensorFlow and Jupyter Notebook to enhance and streamline our AI usage.

Our team of seven was composed of five developers, a graphic designer, and my role as the lead UX/UI designer. I was brought on during the early stages of the project and helped to shape shape the concept of the app. We had an early white boarding session where we ideated potential layouts. I took those and created a low fidelity mock ups


Our first week began with project planning,


which included a conversation with UX/UI where developers provided a complete ERD and write up of expectations for our app. From there, the UX/UI team created an interactable figma wireframe alongside the creation of a branding plan. The developers then split into groups, working on front-end with React and back-end with Django. Despite being newcomers within the field of AI, a few of us dedicated 2.5 weeks to comprehend AI modeling and training. Through this process, we explored tools such as the Gemini AI studio, delved into Google AI documentation and experimented with TensorFlow and Jupyter Notebook to enhance and streamline our AI usage.

Landing

Landing

Alphabet

Alphabet

"A" Card

"A" Card

What we learned

What we learned

What we learned

Developing an app to better serve a specific community such as individuals experiencing vision loss or those eager to learn Braille, presented a significant learning curve. This was our first time diving into the world of accessibility so it required extensive research into Braille’s intricate details.

Additionally, we developed skills in training Verte and Gemini models and became adept at navigating Google’s documentation. Working closely with the UX/UI team was crucial, as constant communication ensured alignment and cohesive progress.

This process not only advanced our technical capabilities but also deepened our appreciation for teamwork and the critical role of tailored accessibility in technology.

What's next for TouchStone

What's next for TouchStone

What's next for TouchStone

Ensuring that TouchStone is fully accessible remains our primary focus for the app’s next development phase. We plan to support connectivity with portable Braille devices, enabling tactile learners to engage directly throught their smartphones.

Future enhancements for TouchStone include refining our Gemini AI with tuned models and structured prompts for more accurate responses. We are committed to incorporating Responsible AI to ensure these responses are reliable and appropriate.

Additionally, we aim to introduce a camera feature that utilizes OCR technology powered by Google AI. This will allow users to take a picture of a text document and receive instant BRaille translations. We also plan to implement image detection technology so sers can photograph objects and receive their Braille descriptions.

Our ongoing efforts will also focus on improving the accuracy of our Braille translations, ensuring TouchStone continues to meet the needs of our users effectively.

Interested in working together or looking to chat over coffee?
Shoot me a message and let's chat!

© 2024. Designed by Xuemei

Interested in working together or looking to chat over coffee?
Shoot me a message and let's chat!

© 2024. Designed by Xuemei