Based on: Wassily Kandinsky and the beautiful curse of Synaesthesia.
Article by Sam Kalman | January 20, 2008
Creation, rewinding blips
Tapping my fingers to the beat
This begins my journey to another world
Harsh cold static
Full round blossoms of budding beauty
State of art, or state of the art?
Droplets fly from a sprinkler
Shattering and reforming glass panes of ice
Drilling a swirl into the Earth
Seesawing on light
(image of fully-lit layer cube, just before activation)
Rez was the first game of its kind. As the brainchild of game designer Tetsuya Mizuguchi and United Game Artists (UGA), Rez was created to invoke a feeling of Synaesthesia in those who play it.
Synaesthesia is a neurological condition in which incoming sensory pathways can cross and tangle together. Experiential blending of one's senses is the result. Some with the condition will taste words or smell colors. For some, each different note on a piano will appear to be a different color. Even others will taste different flavors upon hearing different words.
Games are a primarily audio/visual experience. Because of this, Rez could only possibly succeed at invoking Synaesthesia by working within this boundary, blending sounds with visions. It does succeed in blending these senses, as the experience of playing Rez is nothing less than an incredible audiovisual journey.
(insert colorful gameplay image)
Rez was originally designed for the Dreamcast, though it was never released on this console in the US. The Dreamcast offered a rumble pak controller accessory, but it could not be assumed that all players would own it. Nevertheless, UGA implemented rumble features into Rez, adding the sense of touch to the list of blended senses.
In the US, Rez only saw release for PS2. Many will argue about which is the superior version: the DC version with its full-screen anti-aliasing, of the PS2 version with its 60 fps. If we look beyond these cosmetic differences, we'll find that the significance of the PS2 version lies in its Dual Shock 2 controller. Every person who plays Rez on PS2 will experience the sensation of their fingers intertwining with their eyes and ears.
(insert another gameplay image)
In this gamer's opinion, the inclusion of touch senses into the game is absolutely critical for receiving the full experience. Playing the DC version without a rumble pak is a terribly static experience. The game remains beautiful to see and hear, but without rumble it's very hard to forget that you're staring at a television screen. When rumble is added, it's easy and natural to fall through the screen and into the world. Playing in the dark is especially helpful to becoming immersed in the lights and sound.
(insert image of trance vibrator box)
Also worth noting for the PS2 version is the addition of the Trance Vibrator. Mizuguchi has openly stated his desire to integrate tactile sensations into the game as much as possible, with as much variety as possible. Dual Shock 2 is functional but not incredible for this purpose. As a result, Mizuguchi thought up the Trance Vibrator. It's a computer mouse-sized plastic rectangle that plugs into the PS2's USB port. It only works with Rez, but it adds an incredible amount to the gaming experience.
(insert image of the Trance Vibrator)
One of the best parts of going to a rock concert is feeling the sound waves pounding so hard they vibrate your gonads. The Trance Vibrator is about as close to simulating this experience as you can get without a several-thousand dollar speaker system and some very pissed off neighbors. The Vibrator actually works differently than very loud speakers would. It responds to in-game actions that are independent (but closely connected) to the music. The Vibrator might be turned up to 11 during a part of the song without quite as much bass. It serves its purpose to make the player part of the gaming experience rather than part of the musical experience.
Chances are likely that if you're reading this article, you already know about other documented uses for the Trance Vibrator. While Mizuguchi prefers to put the Trance Vibrator under his feet or against his back, obviously others are happy to experience emotions never intended to be evoked by the game. This unintentional added bonus alone makes the Trance Vibrator a brilliant peripheral. Pulling ourselves out of the gutter for a moment, simple reflection about the emotional impact that the sensation of touch can bring to a game is incredible. In this respect, Rez is a successful experiment.
(insert another image of gameplay)
Before we can journey into the details of mechanics used by Rez to invoke synaesthesia, we should understand a bit more about what it is and how it affects people.
The first Oxford University Press definition of synaesthesia is "A sensory experience elicited by a stimulus in a different sensory modality, as when particular sounds evoke sensations of colour." To further understand the physiological condition of synaethesia, I'll direct you to the Scientific American article "Hearing Colors, Tasting Shapes" by Ramachandran and Hubbard.
Instead of focusing on definitions, let's take a look at one famous synaethete: Wassily Kandinsky, the inspiration behind Rez.
(insert image of Wassily Kandinsky)
Wassily Kandinsky was a brilliantly gifted Russian painter. His synaesthesia allowed him to see music appear before his eyes. He would then paint what he saw.
(insert image of Kandinsky's Concert)
Kandinsky must have been studied to great length by Mizuguchi. Simply looking at Kandinsky's paintings alongside screens from Rez will evoke recognition almost too obviously.
(insert image of Kandinsky's Fugue beside Rez gameplay)
When the final level of Rez is completed, and the credits are done rolling, the dedication to Kandinsky is plainly revealed with the following words: "(get the exact quote...) Dedicated to the everlasting creative soul of Kandinsky". If UGA wanted everyone who play Rez to experience music the way Kandinsky did, who can say whether or not they succeeded? However, if the experiences are at all similar, Kandinsky truly lived an amazing, experiential life.
(insert image of Kandinsky's White II)
If Kandinsky could see colors and shapes when he heard music, and Rez is meant to emulate that experience, then we have established the critical foundation upon which Rez is built: Music. Listening to music is a passive experience, and we all know games are far from passive. Rez is built to not only evoke synaesthesia through listening and viewing of music, but to evoke it through interaction with visuals and by association, the music itself.
The implication of this design is that the audio experience of Rez is much different from one play to the next, depending on how you choose to play the game. The game mechanics of Rez involve pressing and holding the X button to lock onto enemies, then releasing the button to fire shots at them. Every single one of these events evokes an aural response. Pressing, locking on, releasing, firing, and ultimately destroying an enemy. The rhythm of different players as they perform these actions will rarely be identical. There are too many subtleties in the way our brains work for two people to manipulate the controller the exact same way with the exact same timing. For this reason alone, every single play of Rez is unique in its own special way.
(insert image of something)
Creation is an additional layer beyond the interactive elements of the music. As the player, you not only alter the music, but you build it through your actions. Each level has 10 different "layers". Each of these layers is unlocked by attacking a special glowing cube. When the cube has been activated, and light pours forth from its opaque walls, the player transcends to the next layer, and the music grows and changes along with them. Failure to activate each layer is not rewarded with extreme punishment, which reinforces the malleable nature of the game mechanics. However, activating all the layers is the expected thing to do, and the rewards for doing so are extremely rewarding (especially if you're using the Trance Vibrator).
From a production perspective, creating all this audio interaction and creation logic must have been quite a challenge. The first step was to start with great source material. Mizuguchi hand-picked five artists and approached them about creating an original song for Rez. Those artists went off and did their thing for a while, and came back with some truly amazing music. The development team then took their source material and rearranged it to fit within the context of the gameplay. Listening to the Rez soundtrack immediately after playing the game will clarify how the songs were changed to fit the gameplay. All the songs are immediately recognizable of course, but there are some obvious differences in arrangement and organization.
(Insert image of Level 1 here)
Music: Buggie Running Beeps 01
Artist: Keiichi Sugiyama
(Insert image of Level 2 here)
Music: Protocol Rain
(Insert image of Level 3 here)
Music: Creation The State of Art
Artist: Ken Ishii
(Insert image of Level 4 here)
Music: Rock Is Sponge
(Insert image of Level 5 here)
Culture: Evolutionary Earth
Artist: Adam Freeland
It is highly unlikely that Kandinsky would have heard music like this in his time. If he had, I wonder what kind of visions he would have had.
Secret outline (to be removed before publishing):
-Rez the game
--Rumble and the Sense of touch
--Trance Vibrator brilliance
-Synaesthesia the disease
--Malleable audio experience
--Gameplay theory of musical creation
--CD soundtrack reveals heart of the experience
-The levels (I'm not quite sure what to say for each of the levels)
--Keiichi Sugiyama - Buggie Running Beeps - Egypt
--Mist - Protocol Rain - India
--Ken Ishii - Creation The State of Art - Mesopotamia
--Joujouka - Rock Is Sponge - China
--Adam Freeland - Fear - Earth
-Themes (I'm not as confident about this section when I think about trying to write it...)
--Creation and destruction
--Life, death, and rebirth
-Post thought... discuss Rez HD?