Data Sonification: Acclaimed Musician Transforms Ocean Data into Music

Håkon Kornstad is an internationally renowned, award-winning Norwegian musician, who is known for his unique blend of opera and jazz. Recently his career took an unexpected swing: first he enrolled on a bachelor program in Robotics and Intelligent Systems and then did an internship at HUB Ocean. The reason? To turn ocean data into music.

The ocean is changing

Håkon Kornstad:

“I grew up by the ocean, spending summers in a small cabin on the tip of Norway’s southern coast. The ocean was the first thing we connected to in the morning. And it looked as though it was here to stay. But I noticed that it has changed. When I was a kid, we used to catch a lot of fish – now it’s scarce. There are fewer and fewer sea mussels in the water. There are almost no birds – they have moved to the city. Instead, plastic is floating around.”

The ocean view from Håkon’s summer cabin.

The change happening in the ocean is not only visible to the human eye. It’s in the data. Variables that represent sea temperature, the amount of sea ice, fish abundance, the volume of oxygen etc. are changing at a rapid pace. While scientists are aware of this change, it can be challenging to convey the true alarming meaning of these dry figures to everybody. But to revert this decay and heal the ocean, we have to spread the message and promote what UNESCO calls ocean literacy – an understanding of the ocean’s influence on you and your influence on the ocean.

Music, which does a good job of igniting emotions, can help on this mission, Håkon believes.

“What if the essential ocean variables could play their own melodies? Could this be a tool to raise awareness? Could this music spark the emotions in politicians, urging them to take action? Could it even make seasoned ocean scientists see new correlations in complex ocean data sets?”

Sonification – the music of science

Sonification literally means “turning into sound”. Similar to visualization, which helps us see numbers and makes the relations between them more obvious, sonification allows us to hear data, and sometimes even feelsomething about it.

“Music is numbers. A piano keyboard with its 88 keys is just an array of 88 numbers, where middle C would be number 39. The notes on the musical score give us the values in an array of numbers. So when you play the piano, you sonify that array. And if you’re really good at it, you will make people emotionally engaged,” says Håkon.

Piano keyboard represents an array of 88 numbers.

The concept is not new – the idea that a dataset could generate sounds to help us understand it better dates back to the 1990s. Håkon mentions the most interesting projects: for example, Leah Barkley’s “Acidification: Requiem for the Reef” that combines ocean data with real hydrophone recordings, documenting the destruction of the Great Barrier Reef’s ecosystem.

Another approach is taken by data scientist Judy Twedt, who used climate data to express the dramatic impacts of climate change through piano music. Håkon chose this route for his project:

“Music lives in a different space of the brain than numbers or texts do; it sparks emotions. We at HUB Ocean can use it, because what we know is numbers and what we want is to wake people up.”

Working with datasets on the Ocean Data Platform

The very first step of the project was to log in to the Ocean Data Connector and look for the most suitable dataset. Eventually, he chose “the global ocean physics reanalysis monthly mean” by Copernicus Marine Service. It provides a good overview of historical data, ranging from 1993 to 2022. The variables that Håkon needed are water temperature, salinity, sea currents, and sea ice concentration.

 The Barents Sea is located off the northern coasts of Norway and Russia.

In the next step, Håkon made a slice of the map to look at one specific region. He chose, the Barents Sea. Then, still in the Ocean Data Connector, he visualized the data to create animations for the music.

Water temperature in the region month by month. Warmer colours represent higher temperatures.

Salinity in the region month by month. Warmer colours represent higher salinity.

Here comes the sound

Now it’s time to download the necessary data from the Ocean Data Platform and sonify it. Let’s look at this graph showing the rise of sea temperature from 1993 to 2022. The change happening here is easy to see – both peaks and troughs are getting higher over time.

The graph shows minimum and maximum temperatures in the region from 1993 to 2022.

But how will this graph sound? Will we be able to spot the difference? Let’s create a raw sonification of this graph.

The same graph, but sonified.

This sonification is good enough to convey the message – we notice that notes are becoming higher closer to the end of the audio. This approach has already been used to illustrate graphs to the visually impaired. But it can hardly provoke any emotion. Nor can anyone call it music. But how to turn it into music?

Choosing scales and instruments

To do this, we need a scale. A scale is a set, or an array, of tones. For example, natural major is a scale consisting of 7 notes: do, re, mi, fa, sol, la, ti.

“Scales evoke meaning in us, but it’s also a cultural thing,” explains Håkon. “Some scales are more popular in one country than in another. The Barents Sea is close to Russia, so I googled what the most common Russian scale is, and it turned out to be melodic minor. That’s how I picked a scale to set the mood in the project.”

Finally, Håkon picked instruments representing each of the four variables: temperature, salinity, sea currents, and sea ice concentration. As the composition unfolds, each of the instruments is moving along the scale, representing the real change that happened between 1993 and 2022.

The top part of the video shows temperatures, the lower part – sea ice concentration, and the ripples – sea currents.

“This composition is not mine, but the ocean’s,” says Håkon. He adds that the combination of data and music not only brings emotions to science, but also democratizes it, making accessible and understandable to more people.

“Imagine one day you could play a dataset on Spotify or hear it in a Beyoncé song,” he says.

Håkon believes that this approach has endless applications, for example, creating ready-to-play datasets so that anyone can play them.

We at HUB Ocean share Håkon’s vision and aspirations and hope to witness more initiatives like this one, where datasets are both democratized and emotionalized. At the same time, we are committed to doing our part of the job – unlocking ocean data, ensuring its accessibility to all.

Photo in the header: Enas Albadri