Monday, September 27, 2010

dispositif VIII: The behavior of technical beings


In the following section, we shall try to return to the questions raised near the end of section VI, in order to gain a clearer methodological grip on the daunting challenges that underlie a sociological as well as pragmatist/materialist analysis of "interfacial installations." What is at stake, after all, is an advance in knowledge about how we assess or value attributes or affordances of "technical beings," of programmed responsive environments or hybrid media spaces (augmented realities), which behave – i.e. behave with and towards the visitor-participant – as if becoming living, moving, animate matter, confirming a certain vitality and a range of symptoms in their materiality (motion, agency, autonomy, aura, protocol behavior, etc.).

If these "symptoms" are describable – and surely they are – in technical terms, then we'd need to resort to code, to patch-making, the use of object-oriented coding or "actor" coding (in Isadora), to the workings of physics engines and other programming mechanics for 3d virtual spaces and for 2d projected video output and/or sound. The terms used, for the "symptoms," of course can easily cross over to the symbolic realm and the repertoires Goffman applies to interaction rituals. Other repertoires would clearly derive from the field of HCI / interaction design.

To repeat my suggestions from Section VI – they proposed to give more attention to how a particular dispositif enables the interface relations technically while observing how the human performers respond to responsive environment or experience its sensate articulations. Thus, we want to parse the distinctions between the twelve installations (our examples from the final presentations), and how these installations perform, or are performed. My distinctions are meant to serve as a methodological guide to (1) understanding the interface design features and interpreting/describing the content-expression of the projected worlds characteristic of the dispositifs described here; (2) drawing attention to the interaction behaviors that are performed or performable in the dispositif; and (3) locating some of the dramaturgical decisions or strategies that may have guided the artists’ compositions and their intent on involving audiences in machining architectures.

("Grid," interactional performance installation by Sarah Dahnke, with James Cunningham, Suzon Fuks, Julia Alsarraf, Wendy Chu).

(“Dotted Landscape,” interactional performance installation by Wendy Chu, with James Cunningham, Julia Alsarraf, Sarah Kraft, Sarah Dahnke, Victor Zappi)


(1) Interface design features

The "Tiling Performance" dispositif seemed constructed for a 'solo'performer engaging the technical environment and the physical object (water bowl) on the lit square - a square area specifically overlooked by an HD digital camera on a tripod, sending its signal to the Isadora programming environment. Visible/sensorially experienceable to the performer (who could be standing in or modelling any visitor to the installation who picks up the technical environment's behavior) was the projected image motion, the two or three bands of light containing the serials, the "tiles" of processed images (see below).

(The participating performer holds his hand in front of the camera lens, the tiled images appear in the white bands, arranged almost like a film strip of frames)

The dispositif is a clear arrangement, as we can see in the performance photo: the participant knows the placement of the camera, and can control what the camera lens sees by observing the real-time processing of the images that are appearing around him projected from above onto the floor and spatial architecture. In the Isadora patch, the programmer can design the projector-output and precisely configure the "geographical" locations of the bands of tiled images processed by the software actors.

In the Isadora environment, the programmer prepares "scenes" for the "stage," deployable as projected output in any given performance or installation. Isadora is a graphic programming environment for Macinstosh and Windows providing interactive control over digital media, with special emphasis on the real-time manipulation of digital video. Each scene can have multiple "actors." The titles and values of every inout and output are visible to the programmer and are quickly editable, which means the state of each module (referred to as "actors" in Isadora) is instantly clear and can be improvisationally changed by the user.

In regard to the technical description of the "technical object" (I am using a term that is well established but owed to the magnificent work of French philosopher Gilbert Simondon's 1957 thesis Du Mode d'Existence des Objets Techniques), it might be of interest here to quote from the Isadora manual and give you the specific explanation of the "image tile" actor:


This actor, the Manual tells us, displays the incoming video stream in the form of 'tiles' of varying brightness that are created by from a second video stream.


Whenever a new frame of video arrives at the 'tiles in' input, it is broken into a series of tiles whose height and width are specified by the tile cols and tile rows inputs. Then, the brightness of each tile is analyzed and stored. When a new frame

of video arrives at the video in input, it is reconstituted by creating a mosaic of tiles of the appropriate brightness.


For instance, consider the following tiles in input which is the ASCII character set. Each character occupies 6 pixels across, and 8 pixels down. Because there are 67 characters across, the total size of this image is 67 x 6 = 402 pixels across by 8 pixels down.


Using the Picture Player to supply this image as an input to the tiles in input, you would set tile cols to 67 and the tile rows to 1. Upon receiving this input, the Image Tile actor would analyze the each tile, determining its brightness.


Finally, as frames of video arrive at the video in input, they would be broken down into tiles of matching size (6 across, 8 down). Each of the tiles in the video in stream is replaced by the tile from the tile in input whose brightness most closely

matches the original. Using the ASCII character set example above, note the transformation of the image. (Because the image is small, you may need to blur your eyes a bit to appreciate the result – higher resolution images look better.)


While this example was made using a still image, you can just as easily supply a moving video to the tiles in input. The results in this case depend greatly on the content of that video stream, but it can lead to interesting effects.


Input Properties

video in: The video input stream to be tiled.

tiles in: The video input that will be used to create the tiles. The height and width of each tile are given by the 'tile cols' and 'tile rows' inputs.

tiles across: The number of tiles across in the 'tile in' video stream.

tiles down: The number of tiles down in the 'tile in' video stream.

steps: The number of brightness steps to use when creating the final image.

Lower numbers produce a coarser resolution of brightness, higher numbers give finer resolution.

color: "When off, the color of the tiles used to create the final output is the same as received at the 'tile in' input. When on, imposes the color of the source image on the tiles. Turning this setting on may produce unusual colorization effects

when the 'tile in' image is not black and white.

bypass: When turned off, this effect functions normally. When turned on, the effect is disabled and the video input is passed directly to the video output.


Output Properties

video out: The tiled video output stream. [pp.209-10]


(2) Interaction behaviors

It was mentioned earlier that in "The Table," a physical performance interaction was presented to us in the dialogue between two real actors, Jennifer and Tommy, who manipulated the physical tea cups while seated at a table across from each other, with Jennifer speaking in a low-toned voice. In the interactive disposition of the scene, the camera captured the manipulation of the real objects on the table top, and this input was used to effect a different video output projected onto the adjacent table, thus creating a virtual double scenario of the scene enacted in front of the audience. The virtual scene, however, does not have any human agent in it: only the technical objects are manipulated in the computational space-medium. The objects are like sprites that are moved in a 2-D plane, like in the old computer games. The projected objects are also “mixed” with real objects (two dinner plates) that are placed on the projection surface of the adjacent table.

In "Bubble Playground," the projected video was not taken from a camera source, but the camera vision informed the software (via the "Blob" and "Eyes++" actors) of the positions and movements of the spectator-participants, and this information drove the percolating, colorful bubbles that were flowing across the floor. In this installation, the real actors are the participant players in the playground. But of course they are playing with the software actors, the colorful circles created in Isadora. These bubbles exhibit a fluid motion behavior, and so d the participants, and much of the participants’ playfulness resided in their hop-scotching the graphic images that floated on the floor. Compared to “Bubble Playground,” both “Table” and “Memory Table” include a narrative dimension which marks the time in spatial dispositions involving associations with memory.

Ian Winters’ second piece, “Memory Table,” also offers an invitation to play in its scenario, an invitation for the visitors to sit down at the table and engage with the objects that are lying there – small rocks, glasses, cups, books, a microphone, toys. When the installation was opened to the visitors, it didn’t take long and first one, then a second person sat down on the two empty chairs. Across from the table top, a mid-size projected image (circa 80 cm x 60 cm) showed a black and white image of the same seating arrangement, chairs and table with objects, and the players could see themselves as if in closed circuit. But the projection revealed layers, and as the participants engaged in playful banter, ghostly images of previous visitors appeared in the projection, as if the technical system were waiting to display, at certain times, the prior visitors it “remembered,” so that past, present and future started to mingle and convey an accumulative experience. Some of visitors played with the sound they could generate with the objects, others engaged their co-players in a conversation, either through gesture and mime, or through spoken conversation, and in one case a third visitor entered, seemingly trying to distract the couple at the table by crawling underneath, reaching out with one hand and removing an object, teasing the “dialogue” of the visitors further along as we begin to hear echoes and reverberations of sound stored and re-played. Again, as with “Tiling Performance,” the programming seems technically preoccupied with the idea of serializing action-images, but also possibly reflecting on the nature of the distinctions Deleuze made between “movement-image” and “time image.”

(It might take us into a different direction to expand here on the Deleuzian terms, so I would just want to mention that Deleuze offers fascinating insights into the “interval” – the gap within and between frames – as a constitutive condition of the cinematic medium and the manner in which duration/time of visual experience is configured as narration or instead of narration. Pertinent to Winters’ playing with series is Deleuze’s significant theoretical distinction between movement-image and time-image. The animated character of much of the image work produced in the lab would seem to warrant a closer analysis. For Deleuze, the role of the interval has evolved across cinematic history: in the classical period, it tended to operate as a rational connector between images, for example when subdividing a bodily movement over several consecutive frames to record action, as it was already suggested in Muybridge’s and Marey’s chronophotography where photographic frames are linked as progressive exposures, spatializing the movement of a body in time. Calling it “movement-image,” Deleuze posits that these images subordinate time to movement. Since the 1950s and 1960s, Deleuze claims that with Resnais and Godard we begin to see how shots are increasingly released from any logical connection and tend to become irrational, the intervals between frames generating ambiguity and dreamlike (dis)connections or inexplicable interruptions. Movement is now subordinated to temporality in the “time-image.” While the use of projected images in interactive installations may not follow any cinematic or narrative logic, what is surely needed in future analyses of interactional performance is a closer attention to the particular seriality or flow, stasis or permutation in the layering and digital modulation effects of processed images or animated technical objects. One fundamental question underlying my commentary here is the question: what is an interactive image? And how is it performed?)

A methodological step in this direction was made on the second day of the lab when we asked everyone to create s short scene with direct (closed-circuit) camera-projection performances, during which each member of the group tried to explore the relationships between subjects and objects and objects and space. Without editing and coding, the image object does not have any autonomy at all, it is entirely dependent on the performer. Even in such a basic study, it is possible, however, to play with the superimposition of images onto a surface, and thus evoke animation effects between object and background,

It became apparent during the week that artists with a sculptural background (Jennifer and Emily) developed a particular interest in the virtual manipulation of objects through the computational software.

Distinct from “Bubble Playground” and also from Chafer’s “Lying Bodies/Outside In”, the technical organism of “Memory Table” is programmed to remember input and re-call recorded actions in sequences that become animated/layered with co-presences that are enacted on screen. The live feed is stored and processed to return, a proper symbolic action in the theatrical sense in which Hamlet’s ghostly father appears to re-appear in the Shakespearean play (having prompted theatre theorist Herbert Blau, in Take Up the Bodies, to refer to acting as a process of “ghosting” and ghosting techniques) – an agent of pressure on the protagonist’s consciousness/conscience, a reminder of the (Goffmanian) dilemma of choice, actions having fateful consequences, and non-action just as well. Action reveals character, and actions can be fateful or inconsequential. Hamlet’s famous soliloquy, which includes the question “To act or not to act,” surely brings back to us memories of dramatic conflict, underlying the social rituals Goffman intends to analyze, implying tragic consequences as we know them from real life, if not from games or entertaining spectacles.

Performative installations are not known to involve such tragic plots, and they have not been discussed in the framework of dramatic plots in theatrical representation. The history of theatre evidently evokes literary contexts, fictions and myths, stories and complex characterizations, dialogue and psychological/emotional crises. The examples of interactional works discussed here cannot reach these levels of content, but they engage fragments of symbolic action, ritual behavior and modest narrative association. Provoking interaction rituals, the installations under review here begin to question, even if this may not have been the intentions of the programmers, the representational conditions and material grounds at the intersections of which we can trace complications of behavior, identity, reflexivity, and inter-faciality (with its ethical implications). Moving from “The Table” to “Memory Table,” after all, meant that we had witnessed a troubled, elliptical dialogue between a couple whose relationships seemed fractured, only to find ourselves on the other side of the studio, enjoying the wildly improvisational meeting of “strangers” who participate in an intimate tête-à-tête tinkering with objects that had passed through the hands of others, now re-appearing again on the screen to open up a window into past time, into protracted duration of time that extends the illusion of presence. This was also beautifully noticeable in the fact that while the whole group had moved on to prepare a new setting for the next performance, during an occasional silence in the preparations one could still hear the “Memory Table” audio from the far corner, the tapping of a glass, a murmured voice drifting in from somewhere.

But while the visitors were standing behind the participant-actors at the table, they laughed and appeared to cheer on, encourage and stimulate the actors at the table. This context created a very different social ensemble, compared to “Lying Bodies” and “The Table” – situation in which humorous playfulness and acting out was embedded and safe.

The question about enabling proactive behaviors of playfulness was raised afterwards. How do installations allow and insinuate intuitive behavior, making it very clear to the visitors that they are encouraged to step inside and explore, to enjoy themselves, to experience the sensorial/sensational dimensions of an environment, to trust their re/cognition of the responsive organism? Are there technical environments created for interaction which are too complex or insensitive to the tacit knowledge or reliance on etiquette on part of the visitors. Does interactive behavior emerge inevitably once the participants are comfortable in their roles or “learn” the rules of the game or the states of the environment, recognizing causal patterns or consequences?


[continued in the next section]

No comments:

Post a Comment