GMD - German National Research
Center for Information Technology
[Camera Musica Sketch - 54.6 Mb QuickTime movie]
The idea of using virtual architecture as a medium for musical exploration arose from my interest in open musical forms. By openness I understand conceiving form not as a line with a clearly marked starting point and an inevitable end, but as a field of possibilities merely laid out in a composition without anticipating their realization. Striving for open forms means aiming at the creation of ambiguous music characterized by a network of interrelations combining all its elements - music that does not know any final form in time and that opposes repetition.
It has been tried to create open forms by arranging musical texts in a way that enables the performers to choose among different possible readings during the performance of a piece. Pieces of this type sound different every time they are performed and therefore show a certain degree of openness. But this openness is in contradiction to the uniqueness of presenting music in a concert. The audience cannot comprehend the open form since it is listening to one variation of an open composition which is closed by the performance itself. The aspects of openness could only be experienced by comparing several of these variations. There are, however, not many opportunities to do so, since this would require performing one and the same piece for several times during one concert or releasing different versions of it on CD.
The high demands on formal openness can therefore hardly be met in the concert context and as a consequence, many composers gave up any serious consideration of the problem. In a way this is also due to the fact that the concept of the open form calls into question the concert itself as a form of musical presentation, which still causes a lot of hesitation among composers. Thus, the utopia of the open form, which can look back on a long tradition in the history of 20th century composition, may for the time being be regarded as a failure. Another reason for this might be found in the contradiction between formal openness and one of the basic qualities of music: its linear extension in time. It is, however, this apparent contradiction that makes the problem of the open form so interesting for me because it calls into question the nature of music as we know it.
In my opinion, musical installations offer an important alternative to the presentation of music in concerts. Because of its nature, an installation seems to be by far more adequate form of presentation for a music where aesthetic concepts such as openness, vagueness and ambiguity play an essential role. By way of example, I should like to briefly introduce my musical installation en face.
en face is a type of endless composition realized as a computer program creating, once it has been started, a series of new variations of music. En face has neither beginning nor end and never repeats itself. Still, the music can always be identified as one and the same composition.
Strictly speaking, these characteristics are the only possible formal answer to the situation given by presenting music in an installation, since the audience can enter or leave the installation any time, spend as much time in it as they wish and maybe return after a day or a week.
With en face, I was first of all interested in this type of freedom - the freedom of the audience to move about the room, to be able to experience the music from different angles, to explore actively instead of having to follow from a fixed viewpoint a musical performance produced on stage, as is the case in a concert. My experience with en face showed that part of the openness laid out in the composition could actually be experienced. If one stays in the installation long enough and the program happened to explore similar musical situations (the main formal elements used), one can imagine the possibilities laid out in the composition. Once again, this is done merely by way of comparison, but still one gains the very clear impression that in detail, things might also have come differently. Another factor is that the program makes decisions on the structural level of the composition (i.e. regarding the music's deep structure) and not on the music's surface, on the level of a structure set in time, on the text level, as is the case with musical texts which allow different reading varieties.
The comparison with the musical text points to another important characteristic I should like to point out when discussing en face; as I said, en face was not conceived as a musical text, but as a program representing a composition model - a kind of meta-composition. This model represents, for me, a new type of musical artifact.
In this context, I consider a musical artifact the concrete result of a composition process, in order words, the method chosen to represent the music. Normally, this would be the musical text, the score, which is basically characterized as being a text that requires interpretation in order to be received. In electronic music, the musical artifact is the tape - classical electronic music does know musical texts as such. The new type of musical artifact appearing in en face (but also in pieces by Karlheinz Essl, such as e.g. in his Lexikon-Sonate) redefines the relation between composer and audience as well as the circumstances and possible ways of composing, presenting, distributing and receiving music. In leaving out the interpretation process (which still plays a central role in the general understanding of music), this new type of musical artifact opens a range of highly interesting possibilities of composing. We are probably only at the beginning of a new development in music when using a model as means of representation. This open description offers a far higher potential for the realization of formal openness than the musical text.
The compositional limit I reached with en face may become clear when examining the composition process for a second: to compose, here, means to model, to program. By interacting with the model, it is itself being refined again and again. Simultaneously, the music produced by the model can be listened to while working on it. We can imagine the model as a kind of machine, its construction as part of the composition. The definition of levers attached to the machine in order to later on influence its function is also part of the compositional process. After finishing the construction of such a machine, one can start experimenting on it, i.e. using the different levers to control the effects the modification of parameters has on the model. This means exploring the possibilities inherent to the model, which has not only a representative but also a simulative function.
The compositional process is characterized in turn by a description of new possibilities or their modification and by a direct examination of the effects these changes have on the musical structure. If you know the model well enough, it can be explored in an interactive way - as for en face, I am, however, the only person to know it well enough, since the program was not written to be used by anyone else. This is why in en face, a second level was placed above the composition model. It explores the composition automatically by going through its inherent possibilities via a complicated hierarchy of directed random operations. Thus, a program takes over the exploration, automating the very level of decision on which the openness of a composition could be experienced directly - just like during the compositional process.
In Camera Musica, the musical installation I am working on at the moment, I am trying to make this level accessible to the audience, to let them directly experience and apprehend the openness of the music. And this is exactly what I mean when I am talking about the exploration of music. The music in Camera Musica is basically be conceived in a similar way as in en face, even if it sounds entirely different. As I have already mentioned, in en face I use a basic formal element I refer to as situation. This situation will be used in an extended form also in Camera Musica.
The music in Camera Musica is conceived as a family of various, interrelated musical situations composing in their interplay what we may call a musical space. And it is this space I wish to make accessible in my installation. So the audience should be able to move from one situation to another within this space and to slowly explore its special features through the relations between the individual situations.
Each situation is characterized by certain possibilities of choosing the musical material and arranging it, thereby determining the particularity of the situation, its mood, atmosphere, form and air. In a concrete situation these possibilities describe what can "happen" in the music, i.e. which sounds can be related in what way. Each situation has a static and a dynamic aspect. The static aspect results from the fact that any situation disposes only of certain possibilities. So after listening for a while one is able to predict what can happen and what cannot. The dynamic aspect is responsible for sounding out the local field of possibilities. Again and again, new combinations and variations are played through, thereby conjuring up the situation itself. From a global perspective, the music is making no headway, but within the local possibilities laid out in the situation it remains unpredictable, open.
What does the concept of the musical situation achieve on its way to approach the utopia of the open form? The situation as a basic element of musical representation holds in itself features of openness, undetermination. And since the composition does not lay down the sequence of situations but only establishes a wide variety of possible combinations on the structural level, it becomes an open composition. The situation concept can therefore be regarded as a solution to one of the key problems of interactive musical installations, that is the problem of articulating the music-inherent temporality by letting the audience participate. The movement the music itself performs, so to say, in order to appear can normally not be influenced by the audience. This is originally the basic situation of a concert which is mainly characterized by the pursuit of this movement. An interactive installation, however, should make it possible to influence the temporal form of music in order to make the moment of openness accessible. This other movement which is contributed to the music by the audience must be articulated with the movement inherent to the music. This is exactly what can be achieved by using the situation concept.
The movement taking us from one situation to the other is a movement on the level of the musical structure. It does not articulate individual musical elements but changes the more global qualities of music such as its harmonic color, rhythmic form, dynamic air or the relation of tone colors. These are the aspects of musical situations that are to be mediated to the audience and made comprehensible. This is the essential demand on a vehicle intended for the exploration of music which I believe to have discovered in virtual architecture.
I got the idea of using virtual architecture when reflecting a detail of en face. An extrapolation of the en face concept made me consider a version of the installation designed for several rooms with something like a musical situation placed into each of them. The audience would be able to move within the musical space while walking from room to room. In such an installation the specific architectural features of a building (e.g. the order of the rooms and the way they are connected) could be interwoven with certain aspects of the musical structure to a far greater extent than in the original version of en face (which was installed in a single room). And this is just what is supposed to happen in Camera Musica, by means of virtual and not real architecture though.
But this approach was only the project's starting point - the possibilities of combining architecture and music, which I am trying to track down, are continually changing the project as a whole, a fact that by making the project more and more interesting on the one hand, on the other hand makes its first realization recede further and further into the future. During my work on Camera Musica it became obvious quite soon that space can only be explored when it is carefully organized. What do I, therefore, expect from an architectonic structuring of space? First of all, I expect a certain readability of spatial relations. For the audience of Camera Musica the architectonic space will be a kind of text (not comparable, however, to notation), a kind of incomplete representation of the music or of possible ramifications of an open composition.
To achieve this readability, I use the language of architecture, elements of the architectonic vocabulary of structural design, well known to all of us from everyday experience, since it is architectonic symbols that help us find our way in urban spaces and decode the wide variety of meanings and programs inscribed in architectonic structures. In my reflections and experiments on the connection between architecture and music I seek inspiration in e.g. the functions of certain elements of architectonic structuring and the meanings attributed to them. I am thinking for example of a house's function as a protective shield against outside influences, as a definition of private territory, as a space for daydreams, or of the functions of openings in the wall such as a door functioning as threshold, as obstacle, as possible access, or of the fascinating concept of the window permitting us a privileged view of the world, that is a view from within our own four walls - a concept which seems to be closely connected to the development of the subject in the western world. Only if I see myself as a subject with my own perspective of the world, it will become important for me to not only conceive windows as sources of light but also in order to look at the world through them.
Aside from this very concrete approach I also consider it extremely exciting to think about possible ways of connecting the dialectics of inside and outside, of being protected and being exposed, of closeness and distance, of being open and being closed to the aspects of musical structure. In the musical discourse, spatial terms are used fairly often and also in our imagination we are constantly making use of various spaces. They serve as a means of ordering things. We arrange things in space so that they are not all in the same spot, to make relations between them become clear or allow them to develop in the first place. The concept of space is of essential importance to our thinking and feeling (we say e.g. that we need more space, we are constantly operating with spaces and of course within spaces). When I am talking of musical space, I am not always referring to one and the same thing, but I do have a clear picture of what I mean in each concrete case. It always has got something to do with the relation between different elements, with the distances between them, with the possibilities of getting from one element to the other or with the forces working between them.
As mentioned above, when talking about architecture in the context of my work I refer to a kind of virtual architecture which is not understood as a model for a real architectonic project. To a certain extent, my way of using architecture is very close to architectonic sculpture, a form of sculpture incorporating architectonic elements in its formal repertory without becoming architecture itself, that is without serving a function in the conventional sense. Virtual architecture has no other function in Camera Musica than to stimulate and structure the exploration. Still it is closely linked to the traditional functions of architecture since it refers to a use of forms which has its roots in real architecture (e.g. by using openings like windows or doors and meanings attributed to them).
In virtual architecture, function as well as material constraints and gravity do not apply, but may exert their influence via the use of forms. The virtuality of this architecture opens up interesting opportunities of playing with the relationships of tensions between slightly modified formal elements and their counterparts in real architecture (think of e.g. a wall built of floating bricks). Vision and movement (i.e. the change of perspective) are the only ways of approaching virtual architecture. When exploring it, smell, temperature, tactile stimuli and the sense of gravity and balance are no longer important. These conditions mark out the frame of what is possible and what can be reached in virtual architecture within Camera Musica. The only sensible approach will be by being aware of these conditions and making active use of them.
Camera Musica is a work in progress. One of the sketches realized with GMD's CyberStage system shall now be discussed here to illustrate what has been developed above. In the sketch a simple visual scene is linked to a set of related musical situations. Using the classical walk-through metaphor, i.e. limiting user interaction to spatial navigation, the audience is immersed in a very concrete visual scene acting as interface to a more abstract musical scene. The relationships to be discovered between these two worlds let the music appear in the imagination and memory of the audience.
The CyberStage is GMD's CAVE-like audio-visual display system which integrates a 4-side stereo visual display with an 8-channel spatial auditory display and 6 vibration emitters built into the floor. The CyberStage is a highly immersive display ideally suited for creating virtual environments. Viewer centered stereoscopic imaging and spatial sound rendering provide for an advanced degree of presence in virtual space. The sketch described here was developed with GMD's Avango VR toolkit and a sound server based on IRCAM's Max/FTS system.
[CyberStage front and rear view]
The visual scene used in the sketch consists of a building-like structure composed of free-floating walls of various dimensions and colors. A free-floating ceiling unites the walls and forms an interior space. Some of the walls reach out into open space thus mediating between inside and outside. The different heights of the walls create permeable sections with varying degrees of spatial continuity. Invisible spot lights are used to articulate the spatial structure and mark points of attraction. The light passing through the gaps between the walls and the ceiling enhances the impression of weightlessness and permeability. Global illumination techniques are used to create a strong sense of spatiality. The combination of daylight and artificial light (only visible on the videos) was inspired by Magritte's famous painting "L'empire des lumières" which shows a night scene with artificial lighting under a daylight sky.
[three views of the scene without sky]
The musical scene is composed of different situation which are mapped out in the visual scene. The following three examples show some basic possibilities of relating the visual with musical scene. The examples were recorded in real-time and directly from one of the video outputs of the CyberStage. The first clip illustrates a sound texture linked to a red beam floating above ground. Whereas the beam (which is actually a very low wall) only suggests a division of space (maximum visual continuity), the sound texture articulates the section to a much stronger degree. This shows that the visual and the musical space are thought as complementary components of the installation.
The second clip illustrates the transition from exterior to interior space. Visual outer and inner space are clearly linked to different musical situations. In this example the relationships between the musical and architectural structure are very explicit and direct.
The third clip is more complex as it combines several types of spatio-musical articulations. The relationships between the situations defining the musical space become apparent.
In order to illustrate the user interaction with the installation, a 5 min. video was shot with a fixed camera located behind the CyberStage. The video shows the images projected on the three walls and the floor as well as the user's silhouette as he or she is navigating through virtual space. Since the image projection in the CyberStage is calculated for the current position and orientation of the user's head, the images appear distorted to camera positioned behind the display. The user always sees undistorted images. The 8-channel spatial sound output of the sound server has been mixed down to a stereo signal preserving some of the spatial information.