Monday, March 20, 2017

Interactive Music

Occasionally, interactive music is referred to adaptive music or dynamic music in the games industry. "An interactive score is defined as music that can change dynamically based on some type of control input" (Sweet, 2015, pg. 36). Interactive music is applied when the player has direct control of the music. On the other hand, adaptive music is applied when the player has indirect control over the music. There are five types of interactive music used in video games. These are improvisational, realtime composition and arranging, performance-based, experimental composition techniques and art installation performance.

Guitar Hero (RedOctane, 2015) is an example of interactive music. The direct input (guitar) influenced the music outcome (gameplay).

Jazz is the most common form of improvised music. "Improvisation within music is defined as the ability to create or perform spontaneously, or without preparation" (Sweet, 2015, pg. 37). A jazz composer decides the harmonic structure and initial ideas and the performers in the group execute the melody, enhancing and changing the notes as their own artistic representation. In interactive improvisational games music, the composers create the framework and compositional structure for the music, but the player's control input determines the final outcome of the music. Variation helps to lengthen a piece of music by creating different variations of incremental changes in the melody and harmony. Variation is an important aspect for improvisational games music.

"Real-time composition is when the musician or ensemble makes musical choices while watching the events unfold the story (Sweet, 2015, pg. 39). The difference between real-time composition and improvisational technique is the manipulator/s of the control input. Real-time composition technique has control input that consists of the story, scene, or narrative being shown; whereas improvisation's control input is normally determined by the musicians themselves or the audience. A composer writing in the style of real-time composition generates the computer processors, which sends a message of what the player is doing to the music engine. Hence, it would have an approximate transition to the next musical cue or emotion.

Performance-based games music reacts to the events or stories that unfold. An example would be the opera, the speed with which the action on the stage unravels dictates how the conductor speeds up or slows down the musical tempo.  "Composers for video games can think about using control mechanisms in the game to change the dynamics of the music or increase the tempo in real time to escalate the tension in a scene (Sweet, 2015, pg. 40).


O2Jam (O2 Media, 2005) is an example of a performance-based music game, the player has to trigger the computer keyboard to play the music notes in the gameplay.

Experimental composition techniques usually includes chance music, aleatoric music, and indeterminate music. The roots of interactive music began with experimental music dated many centuries ago.

Spore (Electronic Arts, 2008), composed by Brian Eno, is an experimental interactive game music.

Many arts installations build virtual or acoustic instruments that allows users to interact with and play the instruments. These instruments usually incorporate interactivity into part of their design to construct a dynamical music experience. "Video games that use music creation as a core mechanic can be inspired by real-world instrument installations and their design an interaction (Sweet, 2015, pg. 42).

The arts installation above shows the interactive with the sound art to create a music piece.

The video above shows a Max/MSP and Flash arts installation that used volume signal to play the game Pong.

Hachibun Onpu (八分音符) (Freedomcrow, 2017) has a relatively similar concept to the volume signal that triggered Pong.

The output of interactive music is usually determined by the control inputs. There are two types of control inputs, the direct and indirect. The direct control inputs include: game controllers, visual and motion sensors (Kinect and EyeToy), touch and pen controllers, computer keyboards and mice, steering wheels, guns and microphones. Specialised inputs are also available for some games, an example would be the bongo and shakers in Samba de Amigo (1999). The indirect control inputs comprises of the character location, enemy proximity or location, environmental effects (e.g. weather or time), level of suspense in the scene, the player's emotion and health in the gameplay, the interaction between the non-player characters (NPCs), the player's current action and the number of puzzles the player had solved.

In the process of composing interactive music, the composer decides which game parameters will dynamically affect the music. Sweet introduced seven responses the music interacts with the control inputs: (1) cue switching and musical form adaption, (2) dynamic mixing, (3) tempo and rhythmic manipulation, (4) DSP and effect application, (5) stinger and musical flourish additions, (6) instrument and arrangement alteration, lastly (7) harmonic approach, melodic adaptation, and note manipulation (Sweet, 2015, pg. 51).


Buffoni, L.X. Making Interactive Music For Video Games [Internet]. Available from: [Accessed 19 March 2017]. 

A. (2009) Fly Magpie (02jam music) [Internet]. Available from: [Accessed 19 March 2017]. 

D.B. (2016) GRIDI: An Interactive Music Installation [Internet]. Available from: [Accessed 19 March 2017]. 

Electronic Arts (2008) Spore, computer game, Microsoft Windows, Electronic Arts United States.

Freedomcrow (2017) Hachibun Onpu, mobile games, Android and iOS, Freedomcrow Japan.

O2 Media (2005) O2Jam, computer games, Microsoft Windows, O2 Media South Korea.

RedOctane (2005) Guitar Hero, PlayStation 2, Activision California, United States.

S.S. (2010) Queen - Bohemian Rhapsody 100% Expert FC Guitar Hero: Warriors of Rock [Internet]. Available from: [Accessed 19 March 2017]. 

S. (2008) Spore - Creature Stage Part 1 [Internet]. Available from: [Accessed 19 March 2017]. 

Sweet, M. (2015) Writing interactive music for video games: a composer's guide. Upper Saddle River, NJ, Addison-Wesley. 

T.T.S. (2017) 什麼∑(Д)竟然用A Cappella挑戰「八分音符」！！合作頻道【VOX 玩聲樂團】 [Internet]. Available from: [Accessed 19 March 2017].