MUSI-G
Touch-Free Tunes for Busy Moments🎵.
What & Why
Can we control music using gestures and how? Musi-G is an interactive experiment I did for answering it.
In an era dominated by touchscreens, I wanted to explore a more embodied way of connecting with music—through movement and physical gestures."
Time Frame: 1.5 month
Exploring potential commercially viable and design possibility
Bringing focus back to bodily expression
Tools: Figma | Adobe Illustrator
OUTCOME OVERVIEW
Onboarding
As the user opens MusiG for the first time, they're greeted by a clean, inviting login screen — like arriving at the door of a cosy music studio. After a simple login, they’re gently introduced to a quick visual tour that promises creativity and ease. Then it will asks for camera access, enabling the app to sense their gestures and bring the experience to life.
It’s not just onboarding — it’s stepping into a new way of interacting with music.
Scroll Down & Scroll Up: A Conversation with the Air
Here, the user is standing in front of their device — no touch, just motion.
The screen subtly invites them to make a gesture. A downward swipe in the air scrolls through playlists, while an upward gesture brings previous options back.
The system pauses, watching closely, waiting patiently. As soon as it detects the movement, a gentle animation confirms success — as if the app is saying, “Got it, let’s go.” It’s a fluid, quiet conversation between user and screen — intuitive, responsive, and hands-free.
Choose Down & Confirm: The Moment of Decision
Now the user wants to select something — perhaps a genre or a visualised option. A downward pointing gesture helps them browse.
Once the perfect choice appears, they hold a confirm gesture. The system observes, confirms, and locks in the choice with a subtle visual cue — a soft glow, a ripple, a reassuring checkmark. It’s the rhythm of decision-making — choose, pause, confirm — designed to feel natural, like picking a song from a shelf with a nod.
Choose Playlist & Target Song: Crafting a Soundtrack
In the final scene, the user explores their curated playlists. With simple gestures, they navigate to the perfect one — maybe it’s “Chill Vibes” or “Workout Boost.”
Within it, songs appear like shimmering tiles. The user hovers briefly, selects, and confirms. Each action is met with subtle, thoughtful feedback. It’s not just interaction — it’s like composing their mood in real time.
CHALLENGES：Busy hands make it difficult with touch-based devices.
In many daily scenarios—such as cooking, showering, working out, or driving—our hands are often occupied, making it difficult or even unsafe to interact with touch-based devices. Traditional music apps rely heavily on manual input, requiring users to stop what they’re doing to skip a song, adjust volume, or manage their playlists.
EARLY IDEATION
Constrains
Initially, I explored the idea of mapping emotions to gestures for controlling music. But through prototyping and testing, we found that emotional recognition through gestures was too subjective and lacked consistent feedback loops.
With a consideration under the strategic design, I prioritize the users’ immediate task goals over expressive capabilities. As a result, I decided to move away from expression-driven design and instead focus on optimising control efficiency and natural feedback mechanisms.
I narrowed the focus to a more fundamental question: How can we use simple gestures to perform basic music control functions—like play, pause, skip, or volume adjustment?"
GESTURE STUDY
What gestures people are already familiar with?
Instead of building a new gesture system, choosing a mature gesture system that has been widely used can help users onboard more quickly and intuitively
Thus, I looked into Vision Pro and smart home gestures — not because they relate to music specifically, but because they represent popularised and learnable gesture systems in current digital products.
Vision Pro Gesture System
Methodology
Luanna Gesture System
broke down existing gesture systems into functional categories
observed how intuitive these gestures felt to users through informal testing
questionnaire survey: what gestures usually being used? which ones being use most often? why are they easy to remember?
GESTURE OBSERVATION IN REAL LIFE
*observation tip: Gestures’ meanings varies among cultures and countries, when choosing and designing gestures for app, there’s necessity to avoid controversial gestures.
MODERN ICONIC GESTURE DESIGNED
USER FLOW DIAGRAM
These diagrams ensure that every interaction aligns with user needs, guiding the design process toward creating a seamless, intuitive experience. I used the following user flow diagrams to define 3 different action to be done in my app.
TASK 1 Set up gesture control
TASK 2 Create or select playlists
Task 3 Use gesture commands to play music
WIREFRAME FOR DESKTOP
WIREFRAME FOR MOBILE
BRANDING
REFLECTION
Through this project, I learned how crucial it is to design intuitive and culturally considerate gestures to ensure usability in gesture-based interfaces. I also realised the importance of testing gestures with users, as initial ideas aren’t always as clear or functional as anticipated. This experience highlighted the value of iterative design, where testing and refining based on feedback lead to a more effective product.
I believe gesture is a valid and useful interaction form, especially in contexts that benefit from hands-free or intuitive control, such as in virtual reality, automotive interfaces, and public installations where physical buttons may be impractical. Gestures can also enhance accessibility for users with certain physical limitations, providing more interaction options.