Sonification at the BBC

It must be time for the occasional appearance of sound<-data on the airwaves. It happens every now and then, like another swing of the pendulum on whether red wine or coffee is good or bad for you. Sorry if I sound<-weary, but it is generally rather primitive stuff. Here’s a link to a brief feature on the Today programme (aren’t they all brief, with the notable exception of business and sport?) this morning. If you can’t hear it, this is my own transcript made while eating my lunch:

#####################################

Sarah Montague: Can you turn information into music, so that instead of looking at a spreadsheet or a graph, you could listen to what it sounds like? Would that help you to spot patterns that are otherwise very hard to discern? Domenico Vicinanza is from DANTE, the Delivery of Advanced Network Technology to Europe, and before we hear from him, here’s a clip that could be described as the sound of space discovery.

<Clip>

SM: Domenico Vicinanza, what are we listening to there?

DV: Good morning Sarah, we are listening to the sonification of 300,000 data taken by Voyager 1 and 2 during their 36-year existence in space.

SM: Hold on a second, that sounded just like music. You’re saying that …it obviously was… but it didn’t start with someone composing the piece.

DV: That’s right, sonification is basically the representation of information or data by means of sounds and melodies instead of using points and lines. When Voyager 1 and 2 were in space, they took measurements. Measurements  are numbers and, what we do, we just match the numbers to musical sounds. Basically, the larger the number the higher the pitch of the sound we use. The smaller the number, the lower the pitch. So the music, if you like, is following exactly the same behaviour as the data. If the data is increasing, the pitch is going up, if the data is decreasing, the pitch is going down, and so we can say that this process is actually creating a sound fingerprint of the spacecraft.

SM: So are you saying that you took information about Voyager 1 and 2 – their positions – and you did not manipulate it in any way when you turned it into music?

DV: So, we used one particular measurement from the Voyager data, it was the cosmic ray count, the number of particles arriving every second at the detectors on Voyager 1 and 2. The one thing I did was mapping this number to a diatonic scale, using the C major scale. In this way, the set of notes we were having were easy to play or easy to harmonise. I didn’t add any single note manually.

SM: OK, we’ve got very little time so can I just ask you, how do you think this process could be used?

DV: There are many applications of sound and sonification for other applications, and the main reason is the fact that sound can bundle more information than any graph or picture that we can imagine. Because vision is limited to three dimension, but sounds can stack together many more layers. You can listen to an orchestra of eight, nine, ten players and actually listen to each of them.

##################################################

So why am I weary? Well, I can’t really justify being weary until I get off my backside and actually investigate what mappings and explanations work in sonic communication, but if I had to make an educated guess now about best practice in sonification it would be:

  1. Pitch is not the be-all and end-all. In fact, it is probably less clearly perceived than rhythmic and timbral changes (unless you are a trained musician, but we don’t make graphs for trained statisticians only…). Every time I see / hear pitch=f(x) I die a little. When pitch is an element of {C,D,E,F,G,A,B} I die a little more. But later, I recover.
  2. Time dominates all other dimensions in our perception. If you map time to time, you are making it the main focus of interest. Is it really of interest in this application?
  3. Like graphs, you need an explanatory prolegomena (which should not include words like prolegomena). Vicenanza blew it by talking about the mapping over all the short time he was given. I may not have done any better under fire at 6:50 am, but knowing I would get about 30 seconds to explain it, I would give one example. Oh, and that means just featureless noodling is probably not the best bit to use for the prolegomenous exemplarium. Let’s have gravity fields as they slingshot past Jupiter and Saturn – features! patterns! and a story! all in one, like kosmic Kinder eggs – please, not cosmic rays: possibly the most truly “random” (i.e. boring) phenomenon you could find anywhere!
  4. C major. Really? You’ve just crushed any chromatic features out of existence, and as harmonic ones will only be perceived in relation to their neighbours in recognised patterns and cadences, which you won’t get because it’s random, you’ll be left with very little of note. Trust me, I’ve read Walter Piston from cover to cover. Hidden depths, you see…
  5. I like his dimensionality point at the end, but it’s not entirely true. He seems to have forgotten shapes, colour, sizes and time, but what the hell, let’s give the guy a break. If I had to cite an example here it would be neurosurgery.
  6. Before anyone attempts any sound<-data, they should go and read Nouritza Matossian’s book on Iannis Xenakis. Listen to a lot of his music, reflect on what was done in the 50s and 60s, then check out what little we know about Pierre Boulez’s secretive matrix manipulations. Think before you convert, and then stand on the shoulders of giants when you do so.
Advertisements

1 Comment

Filed under Uncategorized

One response to “Sonification at the BBC

  1. Pingback: Kosara on stories vs worlds | Robert Grant's stats blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s