BrailleWatch
Braille input method using Smartwatches.
Overview
Our goal was to get an understanding of what current technologies were being used by BVI users and whether or not our design solution are favorable to potential users.
We also wanted to determine what type of solution would be feasible based off of the feedback gathered from the users we want to design a prototype that would increase typing speed, comfortability, accuracy.
Duration: 4 months
Team: 3 members
Skills: Contextual Inquiry, Prototyping
Tools: Raspberry Pi, Qualtrics.
Motivation.
Smartphones are becoming thinner and the increasing dependence on touch screens makes it harder for the blind and visually impaired (BVI) to interact with the interfaces quickly and efficiently. The need for smartphone accessibility features and assistive technologies to help BVI people navigate through their smartphone independently is rising in relation to new smartphone features which place less importance on tactile interaction components. This study investigates previous Braille keyboards that allow BVI users to type on the cellphones with minimal to no visual cues necessary. According to Science Daily in 2019 , the average typing speed on a traditional QWERTY keyboard on the cell phone was about 38 words per minute (wpm), compared to the wpm rate that has been reported with a Perkins Brailler of 14.2 wpm, and about 4 wpm for BVI users on smartphone QWERTY keyboard with the use of screen readers.
Research Questions.
Question 1
If BVI cell phone users could design a technology that would improve their typing experience on your cell phone, how would they describe it?
Question 2
What do BVI cell phone users like and dislike about our proposed solutions?
terlocking Button System
Smartwatch + Smartphone
Tempered-glass Covering
Proposed Solutions.
Interlocking Button System
An additional component would attach to the back of a cell phone while holding the device in a portrait (horizontal) position. The buttons in the first column (right side of the device) from the top down are cell 1, cell 2, and cell 3; the buttons in the middle column are backspace and space; and the buttons in the last column are cell 4, cell 5, and cell 6. The buttons are laid out in a manner that would require interlocking the fingers across the phone’s back. The left-hand fingers would be on the right column of buttons with the pointer finger (cell 1), middle finger (cell 2), and ring finger (cell 3), and the right-hand fingers would be placed on the left column of buttons with the pointer finger (cell 4), middle finger (cell 5), and ring finger (cell 6). The pointer finger on the left hand would be used as both cell 1 of the Braille input as well as the backspace with the button in the middle column. The ring finger on the right hand would be used as both the cell 6 of the Braille input as well as the space key with the button in the middle column. Users would support the phone with the pinkie underneath the bottom of the phone and the thumbs on the device’s screen.
Allows the users to support bottom of the phone bottom with their pinkies
Buttons would be clicked with their pointer, middle, and ring fingers
The screen can be supported with their thumbs
Concern: effort to hit buttons
Smartwatch Design
This solution involves a smartwatch application that allows for tap entry Braille input. The smartwatch would be used as an input device, allowing users to tap on the screen to input letters to the cell phone. The input of the Braille letter would be done row by row, starting with the first row (cells 1 and 4) and moving down. The screen of the smartwatch would be divided in half, a tap on the left side of the screen would activate the left cell, a tap on the right would trigger the right cell, and a simultaneous tap of both sides of the screen would activate both the left and right cells in the row. For example, to type the letter “N,” a user would tap both sides of the screen, then the right side only, and finally the left side only. The input system would also allow for gesture inputs such as swipe from left to the right side of the screen would input a space, and swiping from the right side of the screen to the left would backspace. If the user is on the second row and there are no activated cells for the Braille letter, a swipe from left to right would skip that row and go to the final row.
Tap application on a smartwatch
Allows the user to hold their phone how they would like
Concern: three steps for one letter
Tempered glass Covering
A cover with six raised buttons is placed on top of the touchscreen of the cell phone device. Users would hold the phone horizontally with the screen facing away from the body. The software would have the buttons located at the side of the screen (the short end of the phone). The cover has six holes located where the software buttons are on the screen, and the holes allow a user to feel where the Braille buttons are. The buttons are to be tapped simultaneously in the pattern of the Braille letter that is desired. The input system would also allow for gesture inputs such as a swipe using the left-hand middle finger towards the center of the screen to input space and a swipe using the right-hand middle finger towards the center of the screen to backspace.
Works with “Braille Screen Input” which is used with VoiceOver or TalkBack
Cell phone is to be hold facing away from the user
Allow the users to feel parts of the screen with tactile feedback
Concern: would need to be design for specific phones
Survey.
The survey aimed to understand how BVI smartphone users input text on their touchscreen devices, and receive their opinions and levels of satisfaction regarding their text-input methods. The survey was also used to gain insight into how they would design their own design solution if they could, and to evaluate our proposed solutions.
Section 1: Demographic Questions.
The first section of the survey consisted of basic demographic questions such as age, gender, occupation, and education level, as well as identifying questions about their vision status, including if they identified as blind, low vision, or visually impaired, how they would best describe their vision status, assistive technologies they use, and if they knew Braille. If the participant indicated that they did know Braille, they were then asked in what situations they used it, if they had experience with Braille keyboards/input devices, and prompted to list which Braille keyboards/input devices they had experience with. If the participant indicated that they did not know Braille, the survey then directed the participants to a question asking how interested they were in learning it (5-point Likert Scale: Extremely Interested to Not at all interested).
Section 2: User-Generated Text-input Approaches.
The second section of the survey asked questions about how the participants currently enter text on their smartphones and what they would like to see if they could develop their text input solution. The survey provided an introduction that reassured the participants that they did not need to know and use Braille. Their solutions could use any form of text input methods, including QWERTY keyboard or short-hand Braille, to name a few. The questions in this section began by asking about current methods to input text on cell phones and their satisfaction levels regarding that method. Then the section asks participants’ preferences about using multiple devices to type (e.g., smartwatches and keyboards) and solely application-based solutions downloaded onto the phone, along with their reasoning behind each response. The section concluded by asking the participants if they had any ideas, suggestions, or thoughts on improving their current typing experience and describing their design approach to improve their typing experience on their cell phones.
Section 3: Proposed Solutions.
The third and final section of the survey presented the participants with the three design solutions proposed by the researchers. The participants were given a detailed description of the proposed design and how it would be interacted with. These descriptions are found within the proposed design section. After each of the descriptions, the participants were asked if the solution made sense to them (if not why), what they liked and disliked about the solution, how likely they would be to use a solution similar to what was described, and the reasoning why, and later if they were to change anything about the proposed solution what would they adjust. After completing section 3 of the survey, the participants were asked if they would be willing to participate in research studies related to this type of research.
Results.
Interviews.
We conducted an interview with five participants
Semi-structured interviews on Zoom for 30 minutes each
Totally blind and lost vision later in life.
Fluent in Grade 1 Braille
Use iPhone with VoiceOver (iOS devices in general)
Working.
Prototype Information.
The application was coded on Kotlin on Android Studio
Sony Smartwatch 3 used for testing the prototype
No speakers on the smartwatch prohibited the use of audio feedback
Code.
Demo.
User Testing.
3 participants
Verbal instructions on the procedure
Training 1: Type all the alphabets twice
Training 2: Type as many phrases (from a predetermined set) as possible in 10 minutes
Testing: 5 additional phrases from the same list
Future Work.
Development on a smartwatch with a speaker
Connection to the keyboard on the connected phone
Audio (word and/or letter based)
Type of Braille: Standard, Contractions, Indicators, Emojis