Want a challenge? Try to play back interface sounds on the show floor at CES. [Intel Booth, CES 2012.]
For those that might not know, for the last decade I earned my living designing digital installations: Multi-touch interactive walls, interactive projection mapping, gestural interfaces for museum exhibits, that sort of thing. Sometimes these things have sound, other times they don’t. When these digital experiences are sonified, regardless of whether they are imparting information in a corporate lobby or being entertaining inside of a museum, clients always want something musical over something abstract, something tonal over something mechanical or atonal.
In my experience, there are several reasons for this. [All photos in this post are projects that I creative-directed and created sound for while I was the Design Director of Stimulant.]
Expectations and Existing Devices
It’s what people expect out of computing devices. The computing devices that surround me almost all use musical tones for feedback or information, from the Roomba to the XBox to Windows to my microwave. It could be synthesized waveforms or audio-file playback, depending on the device, but the “language” of computing interfaces in the real world have been primarily musical, or at least tonal/chromatic. This winds up being a client expectation, even though the things I design tend not to look like any computer one uses at home or work.
Yes, I strapped wireless lavs to my Roomba. The things I do for science.
Devices all around us also use musical tropes for positive and negative message conveyance. From Roombas to Samsung dishwashers, tones rising in pitch within a major key or resolving to a 3rd, 5th, or full octave are used to convey positive status or a message of success. Falling tones within a minor key or resolving to odd intervals are used to convey negative status or a message of failure. These cues, of course, are entirely culture-specific, but they’re used with great frequency.
The only times I’ve ever heard non-musical, actually annoying sound is very much on purpose and always to indicate extremely dire situations. The home fire alarm is maybe the ultimate example, as are klaxons at military or utility installations. Trying to save lives is when you need people’s attention above all else. However, even excessive use of such techniques can lead to change blindness, which is deep topic for another day. Do you really want a nuclear engineer to turn a warning sound off because it triggers too often?
The Problem with Science Fiction
Science fiction interface sounds often don’t translate well into real world usage.
This prototype “factory of the future” had to have its sound design elevated over the sounds of compressors and feeders to ensure zero defects…and had to not annoy machine operators, day in and day out. [GlaxoSmithKline, London, England]
My day job has been inexorably linked to science fiction: The visions of computing devices and interfaces that are shown in films like Blade Runner, The Matrix, Minority Report, Oblivion, Â Iron Man, and even televisions shows like CSI set the stage for what our culture (i.e., my client) sees as future-thinking interface design. (There’s even a book about this topic.) People think transparent screens look cool, when in reality they’re a cinematic conceit so that we can see more of the actors, their emotions, and their movement. These are not real devices – they, and the sounds they make, are props to support a story.
Audio for these cinematic interfaces – what Mark Coleran termed FUI, or Fantasy User Interfaces – may be atonal or abstract so that it doesn’t fight with the musical soundtrack of the film. If such designs are musical, they’re more about timbres than pitch, more Autechre than Arvo Part. This just isn’t a consideration in most real-world scenarios.
Listener Fatigue
Digital installations are not always destinations unto themselves. They are often located in places of transition, like lobbies or hallways.
I’ve designed several digital experiences for lobbies, and there’s always one group of stakeholders that I need to be aware of, but my own clients don’t bring to the table: The front desk and/or security staff. They’re the only people who need to live with this thing all day, every day, unlike visitors or other employees who’ll be with a lobby touchwall for only a few moments during the day. Make these lobby workers annoyed and you’ll be guaranteed that all sound will be turned off. They’ll unplug the audio interface from the PC powering the installation, or turn the PC volume to zero.
This lobby installation started with abstract chirps, bloops, and blurps, but became quite musical after the client felt the sci-fi sounds were far too alienating. Many randomized variations of sounds were created to lessen listener fatigue. There was also one sound channel per screen, across five screens. [Quintiles corporate lobby, Raleigh NC]
Music tends to be less fatiguing than atonal sound effects, in my experience, and triggers parts of the brain that evoke emotions rather than instinctual reactions (in ways that neuroscience is still struggling to understand). But more specifically, sounds with without harsh transients and with relatively slow attacks are more calming.
Randomized and parameterized/procedural sounds really help with listener fatigue as well. If you’re in game audio, the tools used in first- and third-person games to vary footsteps and gunshots are incredibly important to creating everyday sounds that don’t get stale and annoying.
The Environment
Another reality is that our digital experiences are often installed in acoustically bright spaces, and technical sounding effects with sharp transients can really bounce around untreated spaces…especially since many corporate lobbies are multi-story interior atriums! A grab bag of ideas have evolved from years of designing sounds for such environments.
This installation had no sound at all, despite our best attempts and deepest desires. The environment was too tall, too acoustically bright, and too loud. Sometimes it just doesn’t work. [Genentech, South San Francisco, CA]
Many clients ask for directional speakers, which comes with three big caveats. First, they are never as directional as the specification indicate. A few work well, but many don’t, so caveat emptor (they also come with mounting challenges). Second, their frequency response graphs look like broken combs, partially a factor of how they work, and so you can’t expect smooth reproduction of all sound. Finally, most are tuned to the human voice, so of course musical sound reproduction is not only compromised sonically, but anything lower than 1 kHz starts to bleed out of the specified sound cone. That’s physics, anyway – not much will stop low-frequency sound waves except large air gaps with insulation on both sides.
The only consistently effective trick I’ve found for creating sounds that punch through significant background noise is rising or falling pitch, which lends itself nicely to musical tones that ascend or descend. Most background noise tends to be pretty steady-state, so this can help a sound punch through the environmental “mix.”
One cool trick is to sample the room tone and make the sounds in the same key as the ambient fundamental – it might not be a formal scale, but the intervals will literally be in harmony with one another.
Broadband background noise can often mask other sounds, making them harder to hear. In fact, having the audio masked by background noise if you’re not right in front of the installation itself might be a really good idea. I did a corporate lobby project where there was an always-running water feature right behind the installation we created; since it was basically a white noise generator, it completely masked the interface’s audio for passersby, keeping the security desk staff much happier and not being intrusive into the sonic landscape for the casual visitor or the everyday employee.
Music, Music Everywhere
Of course, sometimes an installation is meant to actually create music! This was the first interactive multi-user instrument for Microsoft Surface, a grid sequencer that let up to four people play music.
These considerations require equal parts composition and sound design, and a pinch of human-centered design and empathy. It’s a fun challenge, different than sound design for traditional linear media, which usually focuses on being strictly representative or on re-contextualized sounds recorded from the real world. Listen to devices around you in real life and see if you notice the frequency (pun intended) with which musical interface sounds are commonplace. If you have experiences and lessons from doing this type of work yourself, please share in the comments below.
Sometimes sound design requires thinking inside of multiple boxes.
I’ve developed a small collection of handmade and boutique electronic effects and instruments over the years, like the Grendel Drone Commander, Lite2 Sound PX, and many more (perhaps the subject of another post). Longtime readers may recall that I just love supporting independent makers and small cottage industries: That’s where all the weird, truly innovative stuff happens, and I (like many of you, dear readers) am more interested in cool sound design possibilities than straight-up distorted guitarrrrrrrr sounds.
Mr. Heater, the oddest and loudest camping stove ever.
One of the guys at work loves camping gadgets (as do I), and he shared a video of his odd little Mr. Heater camp stove making some weird unholy racket. Naturally, I asked to borrow it and did some recording sessions with it over the holidays.
A metal reflector lets the unit be used as either a heater or a camp stove. This ring of steel doesn’t make much sound when it’s running (all you hear is the hiss of gas emission, much like this recording), but it sure resonates when the stove fires up, starting as one tone and diverging into two separate tones, creating a harmony. Very effective or driving away bears, or as a means for summoning the dead.
The only processing applied to this sound is some noise reduction to minimize the gas regulator’s hiss, to pull the resonance forward. Recorded at 192kHz, a clip like this is ripe for pitch shifting for even scarier tones! (Mic placement was tricky; placing the mics right in front would melt them instantly.)
[soundcloud url=”http://api.soundcloud.com/tracks/9154382″ params=”show_comments=true&auto_play=false&color=ee0000″ width=”100%” height=”81″ ] [Sennheiser MKH 50/30 pair, rigged for mid-side stereo, into a Sound Devices 702 recorder]
I'm now the proud owner of an NJ-1 Heavy Lifting Utility Unit!
I’m very excited to share a new audio recording: The first steps of my new NJ-1 Heavy Lifting Utility Unit (HLUU). The HLUU is a do-everything kind of robot, and I’m hoping to use it for landscaping and home improvement projects. It was a big purchase in a pretty down economy, but my significant other and I think it’s a solid long-term investment.
I was so excited that I had to grab my field recorder and document its first steps. The manual says to let it charge overnight and then calibrate its voice command recognition system, but I just couldn’t wait to just let ‘er rip. Unfortunately, that meant that it only took a few steps before losing power and automatically shutting down, so that’s why this clip is so short.
This puts my Roomba to shame. Check out the recording below and say hello to HLUU!
These razors were locked in a battle to the death for the sake of sound!
I think it was Ben Burtt who described coming up with a sound effect by putting an electric razor in a trashcan and letting it vibrate while recording it…sadly, I’ve forgotten where I saw or read this… (if you do, note it in the comments!)
This naturally made me wonder what would happen if you put multiple electric razors in a resonant space like a trash can. I have both a small beard trimmer and a larger hair razor, so I knew they’d create two very distinct sets of harmonics.
What else is there to say, really? I put those two razors in a trash can, turned them both on, and then let the ol’ 702 rip with a large condenser mic. I tried it with the trash can lid open and closed. It was when I placed both of them on the lid of the metal trash can that the magic started to happen. The trash can acted as a resonator, like the body of a guitar.
Here’s the bizarrely awesome moment (unprocessed) when they started harmonizing and turning into a rich, thick chorus of motorized drone-y goodness!
All my posts to date have featured what’s newest to me: sound gathering in the field and only slight manipulations to said sounds. But synthesis is a longtime love of mine. In my studio, hundreds of small snippets of synthesized sounds exist scattered across terabytes of hard disk space. I usually have no clue what the source material was, or how I created them.
Luckily, I (re)discovered several unusually well-documented synthesized sounds for this post: a collection of samples that were oriented towards making impactful, short sci fi sounds, but created using virtual synthesizers in software rather than real recordings. These sounds all wound up evoking lasers, blasters, and other sci-fi energy weapons, or discrete layer elements for the same.
Ben Burtt defined this sound for generations with his struck-guy-wire laser blasts in Star Wars, and I (like most) tend to agree that these real sound sources make a big difference in the complexity and character of the final sound. But synthesizing these sounds from scratch is a fun exercise, as well: deconstructing what works about that classic sound (amplitudes of high and low frequencies offset in time), figuring out how to execute it, and then modifying the sound for different emotional effects.
(I’ve found other real-world objects that also make Burtt-style blaster sounds, which will be featured in an upcoming post!)
[soundcloud url=”http://api.soundcloud.com/tracks/5764060″ params=”show_comments=true&auto_play=false&color=ee0000″ width=”100%” height=”81″ ] [EFM1 and ES2 virtual synths, Apple Logic Pro 8]
![Want a challenge? Try to play back interface sounds on the show floor at CES. [Intel Booth, CES 2012.]](https://noisejockey.net/blog/wp-content/uploads/2018/03/musicalSDI-03.jpg)
![This prototype "factory of the future" had to have its sound design elevated over the sounds of compressors and feeders to ensure zero defects. [GlaxoSmithKline, London, England]](https://noisejockey.net/blog/wp-content/uploads/2018/03/musicalSDI-05.jpg)
![This lobby installation started with abstract chirps, bloops, and blurps, but became quite musical after the client felt the sci-fi sounds were far too alienating. [Quintiles corporate lobby, Raleigh NC]](https://noisejockey.net/blog/wp-content/uploads/2018/03/musicalSDI-01.jpg)
![This installation had no sound at all, despite our best attempts and deepest desires. The environment was too tall, too acoustically bright, and too loud. Sometimes it just doesn't work. [Genentech, South San Francisco, CA]](https://noisejockey.net/blog/wp-content/uploads/2018/03/musicalSDI-04.jpg)
