Gerhard Eckel, Francisco
Iovino, René Caussé
IRCAM
Paris, France
Abstract: In the context of more general considerations concerning
the use of sound synthesis in contemporary music composition, the latest
version of the physical modelling synthesis system Modalys (previously called
Mosaïc) is presented. In Modalys synthesis is based on the modal representation
of vibrating objects. Modalys has been extended to allow modifications of
the modal representation before and during the synthesis process. This extension
is discussed in the light of the musical motivations of using sound synthesis
in general and physical modelling in particular. The general trend towards
a convergence of signal and physical models is discussed, and future extensions
of Modalys following this tendency are presented.
1. Introduction
In order to describe the current features of Modalys and to discuss its
future development we first have to introduce the context for which this
sound synthesis system was conceived: the domain of music composition including
the constitution of sound into its realm.
1.1 Sound technology
The way composers may think and imagine sound changed radically with the
development of sound technology in our century. Sound transducers (microphones
and loudspeakers) allow to detach the sound phenomenon from its mechanical
and acoustical production mechanism (instrument) by the means of analogue
representation of sound as electric current. Sound transmission and storage
techniques (radio and tape recorder) use this and other analogue representations
to delocalize sound in time and space. Analogue sound generation and processing
devices (analogue studio) permit the direct production and manipulation
of the analogue sound representation. This enables composers to create what
could be called abstract sound - sound whose structure is not bound
to the constraints of mechanical and acoustical systems (except for the
loudspeakers and room acoustics).
The digital representation of sound signals used in today's computer systems
increases once more the flexibility with which sound may be manipulated.
Digital sound production and treatment does not any longer depend on the
ephemeral nature of the electric current which represents sound "in
time". In computers, sound is represented "out of time" (as
a sequence of samples) and is converted to the analogue representation only
when needed (sound input and output). Since digital sound processing is
not constrained by the causal nature of the analogue studio it allows to
conceive synthesis and transformation techniques that are impossible to
realize or control with analogue technology. Moreover, digital sound representation
is not restricted to mere sequences of samples: More abstract mathematical
models representing sound allow for a direct manipulation of particular
sound features.
Generally speaking, modern sound technology made sound at the same time
more immaterial (its production is not any longer bound to instruments
and instrumentalists) and more object-like (it can be manipulated
with tools acting on its representations).
1.2 Sound synthesis
By sound synthesis we understand here all mathematical techniques used to
create and transform sound. Each technique is based on a certain model capable
of representing and generating a class of sounds. We can distinguish two
types of models: On the one hand, there are models that describe the sound
phenomenon (the sound signal) and they are sometimes called signal models
[1] (e.g. additive synthesis). On the other hand, models
are in use that describe the production mechanism of sound, which is usually
a physical process and therefore these models are generally referred to
as physical models (e.g. modal synthesis). For the composer the main difference
between the two types lies in the way the models are parametrized: Physical
models use physical parameters (e.g. geometric information, masses, forces,
gestures) whereas the parameters of signal models are usually more closely
related to sound perception (e.g. frequency and amplitude trajectories).
The kind of model used and the types of its parameters are of great relevance
with respect to any creative use of sound synthesis: They define the space
exploitable by the imagination of the composer.
1.3 Musical motivations
There are various motivations for using sound synthesis in contemporary
composition. Among them we find the desire to extend the repertoire of sounds
that can be produced with reasonable effort by traditional instruments.
Already long before sound could be generated synthetically composers where
longing for this possibility. With the increasing availability of sound
synthesis tools during the second half of our century this desire could
be satisfied. But at the same time yet another potential of sound synthesis
became apparent: the possibility to consider the design of sound as an integral
part of composition, or as González-Arroyo puts it: "We want
to make music with sound, and not sounds to make music with" [2].
Such approach implies a total assimilation of the idiosyncrasies of sound
synthesis in the domain of musical imagination and composition. Sound synthesis
is not any longer only a means to extend the instrumental sound repertoire
but a specific way of conceiving music.
The choice of the synthesis models (usually several at a time) used in a
concrete compositional situation is thus of fundamental musical importance.
Given that the same sound can usually be synthesized by several models,
this choice is not only dependent on the models ability to represent a particular
real or imaginary sound but also on the perspectives or the potential of
musical development [3] a certain model offers in a
concrete compositional situation. Since each model describes a whole family
of related sounds (structured sound material), it is the kind of these relations
which is of major interest for the composer. This implies that synthesis
tools should allow for easy definition of new models and seamless combination
of existing ones. With respect to the second requirement Modalys offers
interesting possibilities.
2. Modalys
2.1 History
Modalys is the successor of Mosaïc, IRCAM's modal synthesis system
designed by Jean-Marie Adrien [4] [5]
and implemented by Joseph Morrison in 1991 [6]. A detailed
introduction to the features of the first version of this modular sound
synthesis system based on modal representation can be found in [7].
Research and development work on the system was picked up again at IRCAM
in 1994 by the Instrument Acoustics research team headed by René
Caussé and the Interfaces and Sound Representations development team
directed by Gerhard Eckel. A new version of the program (now called Modalys
to avoid name conflict with the widespread Internet World-Wide-Web navigator
Mosaic) has been ported to various computer platforms and is now available
for DECStation, DECAlpha, NeXT, SGI, Macintosh and PowerMacintosh computers
through IRCAM's user group service.
2.2 Synthesis model
Modalys' synthesis model is based on four types of elements (objects, accesses,
connections, and controllers) which can be assembled by the composer in
order to build and play a particular virtual instrument. Modalys objects
describe vibrating structures defined by their geometrical characteristics
(e.g. strings, plates, or membranes) and the physical parameters of the
(homogeneous) material they are made from. Based on this description, Modalys
calculates the (intermediate) modal representation used in the synthesis
process (frequency, damping factor, and mode shape for each mode of vibration).
Instead of synthesizing the modal representation, modal data stemming from
modal analysis or other modal synthesis systems can also be imported. Modalys
objects (representing all linear aspects of the synthesis model) can be
put into relation by Modalys connections (representing all non-linear aspects)
which describe the mode of interaction between objects (e.g. strike, pluck,
or bow). Modalys accesses are used to specify the (variable) locations on
objects at which they interact with other ones. Modalys controllers are
used to specify the trajectories of all time-varying synthesis parameters
(e.g. coordinates of accesses, connection parameters, forces, and controller
inputs).
2.3 Extensions
The modular nature of Modalys' synthesis model guarantees a large amount
of freedom when constructing instruments. This feature, which is due to
the uniform representation of all vibrating objects as modal matrices, distinguishes
Modalys from many other physical modelling synthesis techniques. But the
major advantage of modal synthesis (besides efficiency of calculation) is
that the frequencies and the damping coefficients of the modes of vibration
are represented explicitly in the model [5]. This is
essential for musical applications of sound synthesis where a direct manipulation
of the spectral structure of a sound is desirable. Any compositional approach
aiming at an articulation of timbre and harmony by means of sound synthesis
may serve as an example. There the control over the frequencies and amplitudes
of spectral components is often constrained by both the timbral and harmonic
organization. Since in the first version of Modalys (Mosaïc) direct
access of the modal data was not supported very efficiently the new version
includes a set of control modules for static and dynamic modal data manipulation.
This allows composers to adapt the modal data according to their needs.
Since arbitrary modification of the modal representation may render the
model incoherent with respect to any physical reality, direct manipulation
of modal data loses its value at the point where the original interest of
using a physical model is lost. When exactly this point is reached depends
entirely on the concrete musical application and thus should not be defined
by the synthesis system (it is rather decided by the composer's ear).
2.4 Why using physical models?
The extensions described above seem contradictory to the very interest of
using physical models in sound synthesis: the introduction of the constraints
of causality [8]. But why is the maintenance
of causality in a synthesis model of any interest if one of the motivations
of using synthesis in musical composition is to liberate sound creation
from the constraints of physical systems (musical instruments)? Certainly,
a composer would never use sound synthesis (physical models) to simulate
or replace traditional instruments when they can be used in the first place.
Thus is left the interest in sounds ranging between the concrete sounds
produceable by musical instruments or other vibrating systems (including
physical models) and abstract sounds easily generated by signal models.
This interest in (perceptual) ambiguity is shared by many composers and
appears as the most plausible motivation of using synthesis when excluding
the extreme case of searching for sounds completely unrelated to our listening
experience.
When using signal models, the problem of controlling the synthesis process
consists mainly in creating a system of constraints (a control model) that
forces the synthesis parameters to evolve in a way that gives rise to a
certain identity of the sound phenomenon such that the latter can be used
in a musical discourse (i.e. is recognizable in a certain context, can be
ordered in scales, etc.). With physical models such system of constraints
(causality, spatial representation) is included in the synthesis model.
This is why it is so easy to produce so-called realistic sounds with physical
models and abstract sounds with signal models. Realistic means in this context
that our perception is easily able to link a given sound phenomenon to its
production mechanism (source) and abstract means that it is hard for our
perception to imagine a source for a given sound. This does not imply that
there are no natural abstract sounds - also natural sounds can become abstract
to our perception, but they are usually very hard to produce and control
(e.g. multiphonics). Physical models can be used in this case to allow for
a better control (given that they are refined enough to produce the sounds
in question). But the more important aspect of physical models for musical
composition is that the set of constraints introduced by causal systems
is an especially interesting one because our perception is specialized to
recognize sounds produced under these constraints. And even if we are interested
to produce abstract sounds only, these constraints (which will always be
present in our perception as reference) can teach us how to avoid realistic
sounds. Thus if we are interested in articulating abstract and realistic
sound material we should search for possibilities to modify the systems
of constraints of physical models. This is what the discussed extension
of Modalys is all about: The modal data calculated by Modalys respecting
the physical description of vibrating objects can be modified arbitrarily.
In its current form this new feature is not yet very powerful and we consider
it rather experimental but the experiences composers will make with it are
vital to guide any future development in this direction.
2.5 Future directions
The general goal in the development of synthesis systems can be seen in
a convergence between signal models and physical models [1].
Modalys' synthesis model offers several interesting possibilities to approach
this goal. The clear distinction between linear and non-linear components
of the model allows to combine modal synthesis easily with signal models.
Already the first version of Modalys allowed to produce control signals
with additive or frequency modulation models. In the future we plan to integrate
Modalys more closely with other synthesis models, aiming at a homogeneous
synthesis platform comprising all models of interest.
Concerning more directly the physical modelling aspects, it is planned to
make available a data base of analysed modal data usable with Modalys and
to allow access to the data base of IRCAM's resonance models [9].
These data sets stemming from analysis of percussive instruments are compatible
with the modal representation with the exception that they do not contain
spatial information. It is planned to introduce into Modalys an object type
that simulates "out-of-space vibrations of a single point" [5].
Such an object could directly use the resonance model description and thus
make the resonance models available in Modalys. Again, from a physical modelling
point of view such an extension seems contradictory since the essential
difference between modal synthesis and other synthesis techniques "is
that the spatial properties of the causal devices are included in the modelling"
[5]. But from an integrative viewpoint this extension
seems only logical.
Further extensions are planned on the level of different interaction modes
such as the introduction of more general non-linear devices usable to compose
complex interactions in a modular manner from basic elements. This will
also compensate for the currently rather poor repertoire of interaction
types available in Modalys.
References
1. Depalle, Ph., Rodet, X., "De la voix aux instruments",
Cahiers de l'IRCAM, No. 2, pp. 121-139 (1993).
2. Eckel, G., González-Arroyo, R., "Musically
Salient Control Abstractions for Sound Synthesis," Proceedings
of the 1994 International Computer Music Conference, pp. 256-259 (1994).
3. Cohen-Levinas, D., "Entretien avec Marco Stroppa",
Cahiers de l'IRCAM, No. 3, pp. 99-117 (1993).
4. Adrien, J.M., "Etude de Structures Complexes
Vibrantes, Applications à la Synthèse par Modèles Physiques",
Ph.D. thesis, Université de Paris VI, Paris, 1988.
5. Adrien, J.M., "The Missing Link: Modal Synthesis",
in: G. De Poli, A. Picalli, and C. Roads, eds. Representations of Musical
Signals. MIT Press, Cambridge, Massachusetts, 1991.
6. Morrison, J., Adrien, J.M., "Control Mechanisms
in the MOSAIC Synthesis Program", Proceedings of the 1991 International
Computer Music Conference, pp. 19-22 (1991).
7. Morrison, J., Adrien, J.M., "MOSAIC: A Framework
for Modal Synthesis", Computer Music Journal, Vol. 17, No. 1, pp. 45-56
(1993).
8. Cadoz, C., "Timbre et causalité",
in: J.B. Barrière, ed. Le timbre, métaphore pour la composition,
Christian Bourgois Éditeur / IRCAM, Paris, 1991.
9. Potard, Y., Baisnée, P.F., Barrière,
J.-B., "Méthodologie de synthèse du timbre: l'example
des modèles de résonance", in: J.B. Barrière,
ed. Le timbre, métaphore pour la composition, Christian Bourgois
Éditeur / IRCAM, Paris, 1991.