Figma: Audio Prototyping. — Product + Interaction Design — Kleiner Perkins Design Challenge
Audio Prototyping in Figma
Introducing a new modality of experience design & prototyping.
The Opportunity Space
With today’s myriad of comprehensive design tools, there is no shortage of experience design software that can empower designers of any skill level to help bring their concepts to tangible reality. However, I realized that experience design—In the context of today’s complex technology ecosystems—Is often multimodal. Multimodal designs like voice interactions and mixed reality are becoming more and more relevant.
One aspect that is lacking in most experience design software is the audio modality. Although often unspoken of, sound design actually plays a very important role in the way we interact with our devices. One step to empowering designers to be able to design with the consideration of the audio modality could be integrating audio capabilities into popular design tools.
Scope
Given the timeframe of this project, I opted to first support importing audio along with the ability to prototype it rather than implementing a digital audio workstation directly within Figma. This would also segment the roadmap and create a phased approach toward the release of the feature which can allow for design research to be conducted, provide time for usability metrics to mature, and gauge overall interest and engagement in the audio paradigm.
Team Members
Me
Myself
& I
Timeframe
1 day
Research
Ethnographic Survey
I surveyed and interviewed 12 experience designers of varying skill levels to understand their workflow and also uncover what kind of elements their finished designs consist of. Participant responses to the latter include: refined visual design, comprehensive microinteractions, complete documentation and specifications, code annotations, redlines, interactive prototypes, etc. Some participants also discussed the varying nature of deliverables given the context of the scenario, but there was no mention of the audio modality in these discussions.
Competitive Analysis
I also looked into competing experience design tools and what potential audio capabilities they may have. Out of the six competitors I looked at (Adobe XD, Figma, Framer, Invision, Protopie, Sketch), only Adobe XD as of recently has any kind of designated interface for audio integration.
Insights
1
Sound design is not a paradigm that has been deeply explored by the leading design tool companies.
2
Most experience design software to date does not support audio capabilities to a large capacity.
3
Designers often overlook sound as a possible element of what they consider a "completed prototype or design".
How might we empower designers to consider sound as a possibility for design within their workflow?
Design Response
Audio Prototyping
My design response is the implementation of a lightweight audio import addition to the interactions modal within Figma's prototyping space. This simple change can allow incredible flexibility in the way audio can be used while prototyping screens. I considered several other different explorations of how this feature would visually and conceptually work, but ultimately decided on this response because it was the simplest change with the greatest impact. I also was designing with the consideration of not significantly altering or disrupting the current functionalities of the editor.
Building on top of existing
interaction paradigms
This audio prototyping addition could be kept quite simple by utilizing the already existing frameworks of how Figma notates interaction. In this solution, the audio playback behaves under the same parent interaction commands as the animation feature and therefore doesn't need its own set of parameters.
.png)
Click the dropdown to see
imported audio files and/or
import new audio files.
Selecting a pre-imported audio file.
Aside from this addition, the prototyping user journey continues on as expected with minimal disruption to the expected user flow.
Next Steps
I would like to conduct some design research on the relationship between sound and interaction design to uncover potential new ways of integrating sound into a designer's workflow. Additionally, insights from this study can also serve as valuable information to help guide the design direction of this particular feature down the road.
THANK YOU