musicube blog image

How does it sound?

March 3rd, 2022

In the last weeks we have been talking about disruptive technologies and basic history of artificial intelligence. In the following 4 posts, we will be talking in particular about musicube’s AI. We want to share with you the type of tasks and concepts our engine is being trained for.

No, the AI does not experience music as a human

It is important to say that artificial intelligence should not be understood as an autonomous entity, making decisions based on smart, artificial “minds”. They are highly dependent on their programmers. Their activity and performance should be considered as a prolongation of the way their programmers understand and solve a specific problem they are exposed to. AI, thus, is “no more” than an advanced tool able to automatize and quickly perform certain processes that, otherwise, a person would need to do. In order to do that, the AI has to be fed with great amounts of data. This data is prepared and curated by humans. After a long period of training, the AI can perform the tasks it has been trained for, with great accuracy.

We want to highlight this concept because a lot of people fail to give credit to AI engines of tasks deeply related to human intelligence, such as music and other forms of art. This occurs because they don’t understand how deeply the AI relies on the human minds behind it. Thus, when, in the following lines, we start to talk about how musicube’s engine es able to analyze a music track and extract information of its moods, engagement, pleasantness, etc… don´t think about it as a robotic mind understanding music on its own terms. All that the AI does is based and driven by human experience.

Let's get to the music!

Music is a very complex art. There are endless nuances on it, that, altogether, create the full listening experience. When we listen to a track, objective particularities of the sound, blend with our own memories, cultural backgrounds, and even genetic imprints. For this reason, there are so many aspects to account for when it comes to properly defining how a track sounds.    Nonetheless, when you ask others why do they like music, the most extended answer has to do with very subjective aspects of the musical experience: the triggering of memories and/or capacity to alter or highlight certain emotional states. Does this talk to you too? Refrain to turn on the radio! We will have plenty of music to listen to today!   In musicube, we have spent the last two years training our AI to be able to identify a lot of particularities that have to do with the emotions conveyed by the music, and we will be talking about some of them today.

1. Valence

In the famous Pixar’s movie “Inside Out”, the first two basic emotions filling the protagonist’s mind were two: Joy and Sadness. Experts agree with the idea that they are the most basic to human beings. Thus, it is smart to consider them when classifying musical pieces. For this purpose, in musicube we incorporated the concept of “Valence”. But these emotions aren’t absolute values. In between “Very sad” and “Very Positive”, there are intermediate stages. In order to do this, we assigned the purest sadness to the value -1 and the pure positivity to the value +1. For training the AI, our team spent a long time listening to tracks and tagging them accordingly. And now, musicube’s AI can do it by itself. Do you want to check what is “Sad”, “Neutral“ (aka, Moderate Valence), and “Positive” in terms of music? Here we go! We’ve tried to keep them in a similar style in order to make it easier to compare them.

  1. Sad: To June by Ganglians
  2. Neutral: Bob Dylan's Dream by Bryan Ferry
  3. Positive: Forever and a Day by Buddy Miler
2. Primary Mood

The mood is related to the emotional estate the music conveys. It is a fundamental part of the listening experience, and it is the main reason why most people love listening to music. We wanted our AI to excel in identifying musical moods. With the help of our team of musicians and musicologists, we defined 12 mood values and linked hundreds of tracks to those moods. Nowadays, musicube’s AI is able to accurately identify those moods in tracks it has been not exposed to before and able to add them as part of the metadata. This helps to specifically retrieve those tracks in the music catalogs, and make them available when we need them the most. Do you want to listen to some mood examples?

  1. Positive: Papa Was A Rolling Stone by Marcus Miller
  2. Sad:Heather by Conan Gray
  3. Dreamy: Everything I Wanted by Billie Eilish
  4. Scary: Ayers Rock by Cornelus Bolten
  5. Anxious: Lullaby by Broken Flash
  6. Calm: For What It's Worth by Buffalo Springfield
  7. Aggressive: Ten Thousand Fists by Disturbed
  8. Amusing: The Devil Went Down to Georgia by Charlie Daniels
  9. Enigmatic: Magic by Coldplay
  10. Epic: One Day by Hans Zimmer
  11. Erotic: Hast Du Jemals by Vanessa Mai
  12. Energetic: Centuries by Fall Out Boy
3. Engagement

Engagement is another aspect we account for this first group of categories, that is important for the basic listening experience. This is a measure of how engaging the song is to average listeners. It does not have to be how long do they listen to that song, or how many times do they listen to it. For our AI, if a song sounds similar all the time, it is normally not engaging, but if it surprises you on the way, it improves the engagement. Nowadays we link “engagement” to something to look forward to, but not being engaging is not necessarily a negative quality. For example, if we like to study or work with music, or we want to relax, an “unengaging” track would be great to avoid distractions. If we are driving in the middle of the night, maybe we need an engaging playlist to keep the sleepness out of our way. These are some examples:

  1. Engaging: Mister Blue by John Phillips
  2. Neutral Engagement: That’s what I need by SWV
  3. Not Engaging: The Long Walk by Mark Isham
4. Pleasantness

As the word indicates, this refers to how pleasant is the song for average listeners. Certain sounds and distortions are unpleasant to the ear, whilst others are very easy to listen to and enjoy. Similarly, to the previous category, being unpleasant isn´t necessarily bad. In certain cases an unpleasant song is desired, as background in a movie, to convey the desire of the main character to get out of a certain place. Unpleasant songs also help to convey negative emotions such as anger and violence. Check these examples to get a better idea of how the “pleasantness” sounds.

  1. Unpleasant: Unsainted by Slipknot
  2. Neutral Pleasant: Slow Dancing in the Dark by Joji
  3. Pleasant: Blinding Lights by The Weeknd

This is just a small group of the categories and tags we are automatizing in musicube. Next week we will be talking about another fundamental part of the musical experience, which is the rhythm and the groove. There is so much to account on this line that it deserves an entire article just for itself. You will be surprised.

What do you think about our first group of categories? Are we missing something that should be put in there? Let us know by emailing us at info@musicu.be.