Interactive Sonic Spaces
research project

Objective

My objective was to create a sensation of falling through the space. I was interested in making a sonic environment which would make one experience the same as a falling object. What we perceive is largely related to what we hear, so the idea was to simulate the downward falling motion by creating an environment of sounds that are moving upwards.

Method

I decided to use the binaural recording technique because it gives a very natural representation of the surrounding sound. In this method, sounds are recorded with two small microphones that are placed at the entrance to the ear canals. The ear shell filters sounds differently depending on the direction they come from. Since the binaural recording already contains the outer ear filtering, it allows to hear sounds with an amazing sense of location. The perception of space becomes very realistic, the listener can clearly sense the sounds appearing around, below and above his head.

For the recording, I used a DAT recorder and two omni-directional mini condenser mikes that I dissembled in order to be able to place them in my ears.

Implementation

(1) City sounds

The first idea was to record something that is already moving by itself. I started out by recording street sounds and traffic on a highway. Since I needed vertical rather than horizontal motion, I bent my head forward in a way that I was facing the road and the movement of cars became vertical relatively to the position of my head.

The following recordings were made in this way:

Street

Highway

Although I liked the sounds, it was not possible to record straight vertical movement while standing on the side of the road. To get it really vertical, I would have to stand exactly in the middle of the traffic stream. Otherwise the sound curves either to the right or to the left. I also tried to record the traffic below me from the bridge. However, that made the sound diffused and the pressure changes were too vague to perceive any movement.

Bridge

(2) Synthesized sounds

After that, I switched to experiments with synthesized sounds. I made some Max patches with noise and frequency modulation. Then I played back the sounds with a loudspeaker, moving it up and down in front of myself at different speeds.

Here are some examples of my experiments:

I tried to keep the speaker as straight as possible, but still it was quite difficult to get completely vertical movement because the ears are sensitive even to the smallest deviations to the right or to the left. At the same time, the localization is least accurate for sounds at the same distance from each ear. That is why most of the time it seems that the sound bends slightly to either direction.

In another experiment, I moved the speaker around myself and made it follow some chaotic patterns.

In the end, I combined some parts of the recordings into a single piece.

Conclusions

The hearing provides only part of the information which the senses interpret as movement in space. The rest comes from other sensory systems. Vestibular system is the key mechanism responsible for the feeling of balance, so it has to be taken into account in order to create a real simulation of falling. I would like to do more research on that system and find possible ways to relate its work to sound.

In a way, the chaotic patterns of speaker movement affect the vestibular system, creating a kind of an "out-of-balance" feeling. The sounds that are supposed to go upwards and downwards can also contribute to that sensation. When I tried to simulate the vertical sound movement, I found out that people perceive it differently, also because of the individual ear filtering. In the same recording, some hear a movement from front to back, others from top to bottom. So the movement that is vertical to me may seem right-to-left movement to somebody else.

After this research, I would like to continue experimenting with binaural recording. I am interested in creating various sound movements that could subsequently be used to build responsive sonic environments.

2004, Julia Casianova