TipTap
Prototyping
Product Design
UX Design
Overview
This is an exploratory project developing an interactive frontend for hardware. Rather than proposing an exciting solution, its purpose is to provide a methodology that enables those with limited mobility to operate devices without using their hands.
Time
2025 Summer
Categories
Prototyping
Wearable Interaction
Assistive Technology
Tools
ESP 32
MicroPython
Rhino
Bambu Studio
Figma
Role
Interaction Designer
Creative Technologist
Problem Definition
Most digital interaction systems rely on hand-based input, which limits accessibility for people with motor impairments or for situations where hands are occupied. Alternative input methods often lack precision, comfort, or intuitive control.
How might we explore new forms of subtle, body-based input for interacting with digital systems?
Inital Research
Pain point
Many people with disabilities (individuals with high-level spinal cord injuries) cann't use traditional mouse and keyboard.
Existing solutions
Eye trackers, voice input, and head-controlled devices exist, but they have drawbacks such as high cost, significant latency, and user fatigue.
Breakthrough
The tongue muscles are highly flexible and well-protected by nerves. Even if other parts of the body are paralyzed, the tongue can still move independently.
Interview
Market Research
Material Research
I explored four approaches to micro-wearable technology, highlighting the strengths and key features of each method as well as areas for potential development.
Initial Design
Other than the movement of tongue, the structure and oral environment are also important for the implementation and usage of this wearable auxiliary tool. Wearing comfort, foreign body sensation, ease of wearing manner, affect to the oral environment, and other factors are taken into consideration during the process of design. To better study the tongue and mouth, I started with a 3d oral model and researches about oral environment.
Feasibility Study
Implementation
Physical structure
The physical structure is fabricated through 3D printing, designed to replicate the underlying logic of tongue movements. The outer frame restricts the pulling forces applied to the “tongue” component, resulting in motion patterns that more closely resemble real behavior. The “tongue” component is a matrix which simulate the traction and extrusion between different muscle in reality
Control System
The control system is built on an ESP32-WROOM development board, programmed to drive eight 360 degree servomotors to produce a range of tongue motions such as curling, retraction, and elevation. In addition, an IMU module is integrated to collect motion data and transmit it back to a computer, enabling assistive interaction functions such as mouse control
On the software side, I implemented programming based on Micropython for motor control and sensor processing, while on the computer the data received is converted into corresponding mouse control commands, enabling assistive interaction functions
Inital Test Run
To get the shape of the ideal version of the Tip-tap, I first create a 3d model of it in rhino to match with the shape of tongue and palate. Then I use 3d printing to create a test version to physically verfy the model in the simulation of the mouth environment
PRototype
Esp32 Circuit
Due to the presence of lingual frenulum, the wearable devices need to leave enough space at the bottom for it to move
Data Collection & Analysis
In the firmware coding, the acceleration data is collected, calculated, and assigned as ‘roll’ and ‘pitch’ instead of directly sending raw data to the software program to reduce data traffic in serial channel
Program Coding
Feasibility Testing
Feedback
After implementing on the 3d palate model, I tried the device on my own body and went through a couple tasks to test the feasibility and comfort of my prototype. The tasks include basic actions such as moving the mouse, left click, scroll the webpage. Eventually, I came up with following result
When I use dental glue to stick the device onto my tongue and palate, there is a relatively strong feeling of foreign body. Although I am able to perform the tasks through my prototype, wearing it for a long time will still cause discomfort in the mouth, especially from tongue. The reason behind I believe is that the material and the prototype itself is not flexible enough to adapt to the deformation caused by the movement of different muscle
Through completing different tasks, the sensitivity of the prototype does not reach to my expectation. For the movement of tongue, its agility is only about 30% compare to using hand, far from the 50% I expected. Also, by directly changing the sensitivity control in the program does not improve or even reduce the experience of using it
Final Rendering
Product Image
Exploded View
