Science by Ear 4

SBE4 Workshop. May 3-5, 2018.

About

The SBE workshop series

Science By Ear is a quasi-periodic, hands-on workshop series on sonification at IEM Graz (previous workshops: SBE1 (2006), SBE2 (2010), SBE3 (2014)). Small groups composed of sonification experts and scientists of other disciplines work together on specified tasks. The resulting sketches/prototypes are discussed in plenum.

SBE4

In SBE4, we explored the method of Auditory Augmentation (Bovermann et al., 2010) within our Sonic Interaction Design laboratory (SIDlab). Three interaction scenarios (different hardware platforms and data sets) have been provide,d allowing to sonify patterns of electric power consumption at various levels of time and feedback. SBE4 was co-funded by the Österreichische Forschungsgemeinschaft (ÖFG).

Group photo of the SBE4 participants

Group photo of the SBE4 participants


Prototypes

During the three days of the workshop, four parallel prototyping sessions took place; one for each of the three data sets and an open session on the last day where chosen prototypes were refined.


Outline

The workshop provides a set-up to explore Auditory Augmentation (AA) in/on three different “platforms” within interdisciplinary teams. With auditory augmentation, we mean to augment an object and/or its sound by sound which conveys additional information.

  • We start from objects in a physical environment: a table, a room, or any sensor-equipped object. These objects may produce a sound; or not; we might interact physically with them; or not. The sound may be a result of the interaction but does not have to be. Either of these inputs has to be there: live sound input or live data input from the interaction.
  • We sonify the data, which can be live data or recorded data files of electrical power consumption. The sonification may use sound input (live or recorded files); or it is based on virtual sound synthesis. Further input for parameterizing the sonification can stem from interaction data.
  • The sonification is played back in the physical environment, auditorily augmenting the physical object we started from.

In short, for auditory augmentation we need an object, the sonification of data, and live input from either the object’s sound or the interaction. Our working definition is meant to be a starting point for discussion to grab the idea of auditory augmentation (or whatever it shall be called).

Interaction Platforms and Data

The various possibilities of auditory augmentation in the sense described above shall be explored in and on three different platforms, we call ROOM, TABLE, and BRIX, and sonifying three types of data, we call REAL-TIME, PLUG-WISE, and IRISH.

Platforms

  • ROOM: The Cube is the institute’s main performance and lecture hall, equipped with a 24 loudspeaker array on a half-sphere for Ambisonics sound. Furthermore, there are five microphones mounted permanently to allow for a virtual room acoustics. For SBE4 we prepared to work with both live sound input from the microphones of the virtual acoustics’ system and additionally added ambient sounds.
  • TABLE: The TABLE platform is an experimental platform in an ongoing PhD project (Weger et al., 2018) Technically, it incorporates a wooden board or table (depending on its orientation in space) equipped with hidden contact microphones and exciters or additional loudspeakers; an optical tracking system locates the position of any object or hand interacting with the surface. Any sound produced on the TABLE is recorded and can be augmented in real time through a filter-based modal synthesis approach. The prepared setting for the workshop allows to change the perceived materiality in a plausible way while, e.g., knocking or writing on it. Through this auditory augmentation, the auditorily perceived material of the table can be changed to, e.g., metal or porcelaine at certain definable positions.
  • BRIX: our co-organizers, provided their BRIX system. This system has been developed at Bielefeld university to allow for simple prototyping of interactive objects. In the prototyping sessions with the BRIX, the team could choose an interaction scenario with any object, equipping it with BRIX sensors and/or with microphones.

Data

  • REAL-TIME: The real-time data set comes from a measurement of 5 wallplugs of kitchen appliances (dish washer, coffee machine, water kettle, microwave, fridge) or the group can plug any other appliances during the workshop. The sampling interval is one second, sent over UDP, with data in the power range of 0 to 3000 Watt.
  • PLUG-WISE: The data stems from an anonymous private household, where 9 appliances’ loads have been measured for one year with a sampling interval of 10 minutes.
  • IRISH: The data set stems from a large survey of smart meter data in Ireland, collecting data of 12.000 Irish households over 1.5 years with a sampling interval of 0.5 hours. About 4200 of these have additionally completed a pre-trial questionnaire as well, thus we could group the data sets in different categories. We differentiate
    • three types of household: single household, double household - two persons over the age of 15, and “standard family” - two adults, two kids under the age of 15,
    • two types of housing: appartement or semi-detached house from detached house, terraced house or bungalow), and
    • two types of education of the chief income earner: secondary or tertiary education.

Schedule

Thursday, May 3

10-12am

Welcome & Introduction

Lunch

parallel

demo

session

1-4pm

Prototyping sessions I a,b,c

Coffee Break

4.30-6pm

Presentation of prototypes and discussion in plenum

7pm

Reception at the Town Hall Graz, Hauptplatz 1, 1st floor

8pm

Concert Minoritensaal "CrossTalks"/ ÖGZM

Friday, May 4

9am

Introduction II

9.30-12.30am

Prototyping sessions II a,b,c

parallel

demo

session

Lunch

1.30-4.30pm

Prototyping sessions III a,b,c

Coffee Break

5-6.30pm

Presentation of prototypes and discussion in plenum

Saturday, May 5

9-11am

Open-space prototyping sessions

Coffee Break

11.30-1pm

Final discussion and closing

Lunch