# Sony files a patent: Dynamic Music Creation in Gaming



## rhizomusicosmos (May 28, 2020)

"A method and system for dynamic music creation is disclosed. An emotion is assigned to one or more musical motifs and a game vector is associated with the emotion. The one or more musical motifs are mapped to the game vector based on the emotion. A musical composition is generated based on the game vector and desired emotions."




__





WO2020102005 DYNAMIC MUSIC CREATION IN GAMING


A method and system for dynamic music creation is disclosed. An emotion is assigned to one or more musical motifs and a game vector is associated with the emotion. The one or more musical motifs are mapped to the game vector based on the emotion. A musical composition is generated based on the...



patentscope.wipo.int





It appears to be a system that generates emotional and/or personality metadata in-game and uses that to generate an appropriate soundtrack. The flowchart includes a component "train the model" that links to "write motifs for characters, scenes, etc."

The inventor is listed as Albhy Galuten who is a technology executive at Sony and was a record producer for the Bee Gees and others. He has a number of patents under his name:








Albhy Galuten - Wikipedia







en.wikipedia.org





There's no mention of AI but there is good possibility that the patent encompasses both the use of a library of pre-composed motifs and using generative algorithms.

For those working in the gaming industry, do you see this as the way things are evolving? Or is it simply patent privateering by Sony/Galuten?


----------



## d.healey (May 28, 2020)

I'm glad it's a US patent :D no software patents in the UK... yet


----------



## Patryk Scelina (May 29, 2020)

That is very interesting. Imagine exporting music in MIDI to game engine like in the old days and game engine would play it through some sample player. I guess it would take years to make it possible and have powerful enough machines which could handle playing orchestral samples in realtime while generating graphics, calculating physics and so on.


----------



## dannthr (Jun 13, 2020)

I think you're asking two separate questions and unfortunately, I do not think I'm allowed to read the patent details.

My thoughts are:
1) Is this the way things are going?

The short answer is yes.

In general, the future of certain games and software entertainment experiences are going to demand more and more procedural content generation. This is not relegated solely to music and sound. From a practical standpoint, curating an open-world experience lays a huge burden on artists and designers to create content for every nook and cranny (along with overhead expenses to push and maintain all of that data).

If you need a room in a building, do you actually need an artist to build the room? Maybe all you need to know is how big the room is and what is inside it. Since most normal rooms are laid out in very similar ways. In this way, you just need artists to create the modular content that would go inside. 

How deep this goes is part of the balance. Do you procedurally generate an individual table? Maybe you just make a tile-able top and legs and let an algorithm do the rest.

So systems take the place of individual curation for specific cases where you need some background element to feel endless but still logical and aesthetically cohesive.

Does this take the job away from the designer? Not necessarily. It just transforms the job of the designer. Instead of curating every single space, the designer is creating the logic for which the space is filled.

Games are already made this way.

There are different levels at which music and sound may become procedurally generated. How much and where depends greatly on the needs of your project and your preferences as an artist.

For example, if you use Sample and Hold on some noise for picking a cutoff frequency on a filter in your modular synth. But you play the keys on your keyboard to control your oscillator pitch. How many leaps does it take to go from using an algorithm to pick notes, to an algorithm that picks chords, to an algorithm to chains chord sequences? 

Or coming from the top down direction, picking a music track based on the mood of a place, to picking music transitions based on moving from one mood to another mood, to segmenting out every piece to allow smooth and fluid emotional transitions, all the way to selecting and layering instruments based on the intensity of individual moods or the combination of several mood "vectors."

So yeah, this is the way things are going.

2) Is this a profiteering grab by Sony?

Short answer is yes, patents are always a profiteering grab.

It's very frustrating to watch executives at large companies act like they came up with some great idea or whose research team comes up with a great idea and whose executives attach their names to the patents--and this happens ALL THE TIME.

Dead Space had a Fear vector that was used to control game systems including interactive music and sound systems.

It's not special by your description and maybe even contestable, but almost all technology patents are because patent offices are bereft of people with domain insight into these kinds of design spaces.


----------



## MartinH. (Jun 14, 2020)

dannthr said:


> 2) Is this a profiteering grab by Sony?
> 
> Short answer is yes, patents are always a profiteering grab.



Are they known to grab these to enforce them, or just as ammunition for the inevitable legal battle when they are being sued for infringing on a patent that some other corp already grabbed? 

I'm appalled by stuff like this being granted patents in the first place.


----------



## d.healey (Jun 14, 2020)

MartinH. said:


> Are they known to grab these to enforce them, or just as ammunition for the inevitable legal battle when they are being sued for infringing on a patent that some other corp already grabbed?
> 
> I'm appalled by stuff like this being granted patents in the first place.


A lot of big companies gather a portfolio of software patents so they can bully smaller patent holders into giving up their licensing rights and so they can cross license with other big company's that they can't bully. And some do it just to make money from licensing fees.


----------



## C.R. Rivera (Jun 14, 2020)

And, Albhy Galuten did some work on Layla, to boot!


----------



## Dex (Jun 15, 2020)

Has this not already been done in several games?


----------



## gsilbers (Jun 16, 2020)

Isn’t that what gameboys and old games use? 
the only difference is triggering samples instead of synth/digital sound.

Id have to read the patent.
but as a record producer and no background in software tech I think it’s just a bs way to do midi to samples triggering and and say it’s a patent.


----------



## Dietz (Jun 16, 2020)

I can't believe that this hasn't been implemented already one way or the other _long_ time ago. As a matter of fact I've been member of a development board for some kind of subterranean adventure and game park which included POI-, mood- and context-dependent algorithmic composition in the late 90ies.

8-/


----------



## David Kudell (Jun 16, 2020)

Now Sony just has to digitize the music from all 10,000 of the Westworld scenes and they have enough music for decades!


----------



## d.healey (Jun 16, 2020)

Sure this has been done before, but that's not a problem with software patents. If you have a thing that does A, and you have a thing that does B, you can still get a patent for a thing that does both A and B.


----------



## rudi (Jun 16, 2020)

Just do a search for "adaptive music in video games" in the search engine of your choice to see how novel (not!) that idea is...

e.g:

https://www.designingmusicnow.com/2016/06/13/advantages-disadvantages-common-interactive-music-techniques-used-video-games/


----------



## MartinH. (Jun 18, 2020)

@Glen Brown: Have you heard anything internally regarding the plans for this patent, that you can share with us? I understand if the answer is no, as far as I know sony is quite strict with NDAs. 
Also welcome to the forum!


----------



## Glen Brown (Jun 18, 2020)

MartinH. said:


> @Glen Brown: Have you heard anything internally regarding the plans for this patent, that you can share with us? I understand if the answer is no, as far as I know sony is quite strict with NDAs.
> Also welcome to the forum!



Unfortunately you guessed right! Nothing I can comment on now. What I can say is they have an amazing R&D team that are constantly trying to push the boundaries. I'm sure there will be lots of exciting releases in the future we can all talk about!

And thanks for the welcome :D


----------



## MartinH. (Jun 18, 2020)

Oh well, it was worth a shot :D. 



Glen Brown said:


> What I can say is they have an amazing R&D team that are constantly trying to push the boundaries.



Of that I have no doubt! I might even get a PS5 for myself. I'm more concerned with the legal team that might want to keep all the amazing stuff Sony-exclusive.
If I remember correctly the first "Thief: The Dark Project" game had some of the best 3D spatial audio of games in its time and the decade to follow, but the tech was patented and was barely - if ever - used again. 
I recently heard interpolating between different convolution IRs to emulate changing reverb with changing player position was patented too. Not sure if that's accurate or the patent might have run out yet.


This kind of stuff bothers me. I know a guy who had his first released book taken off the shelves because another company had some kind of trademark protection on square books of 8 to 12 cm height... Personally I think this is bullshit and harms the markets.


----------

