留学生艺术类毕业dissertation定制：美学与材料的表达
Augmented Reality in Art: Aesthetics and Material for Expression
Geoffrey Alan Rhodes, July, 2008
York University Department of Communication & Culture, Toronto
Stephen Heath begins his introduction to the collection of essays, The Cinematic Apparatus,
广告学留学生硕士dissertation定制with an observation about the early descriptions and advertisements of cinema. He writes, “Inthe fi rst moments of the history of cinema, it is the technology which provides the immediateinterest: what is promoted and sold is the experience of the machine, the apparatus. The Grand
Café programme is headed with the announcement of ‘Le Cinématographe’ and continues
with its description: ‘this apparatus, invented by MM. Auguste and Loius Lumiére, permits therecording, by series of photographs, of all the movements which have succeeded one anotherover a given period of time in front of the camera and the subsequent reproduction of these
movements by the projection of their images, life size, on a screen before an entire audience’;only after that description is there mention of the titles of the fi les to be shown, the ‘sujetsactuels’, relegated to the bottom of the programme sheet.” The context here is that, aftersixty years of critical analysis of fi lm on the basis of narrative text, expressive composition,psychology, and a range of other methods, Apparatus Theory desired to re-approach cinemathrough an a priori look at the structure of technology (in the larger sense) involved. New Media
Art is continually in the situation of early cinema; so much of our descriptions of its functionswith each new technological iteration are based on the apparatus present: markers, circuits, tags,
live cameras, streaming data, processing, screen technologies—descriptions of these things oftencome fi rst in order to identify the territory of a piece of media art. This is a banal observation,but, similar to Stephen Heath’s citation of this as a starting point for an apparatus theory ofcinema—standing in for both a criticism of our focus on technology and our glossing over of
its ideological complexities—I start with this as a point of both jumping over and jumpingin to a way of understanding media art through its technology. Marshall McLuhan writes anappropriate graphic aphorism of theory in Understanding Media: “Just before an airplane breaks
留学生dissertation网the sound barrier, sound waves become visible on the wings of the plane. The sudden visibilityof sound just as sound ends is an apt instance of that great pattern of being that reveals new andopposite forms just as the earlier forms reach their peak performance. Mechanization was never
so vividly fragmented or sequential as in the birth of the movies, the moment that translated usbeyond mechanism into the world of growth and organic interrelation. The movie, by sheerspeeding up the mechanical, carried us from the world of sequence and connections into theworld of creative confi guration and structure.” What I want to begin to look at is a similarjumping forward to look back... if cinema was the epitome of sequence and connections in aworld where time had become connection, then where are we now, and what can we now seebehind us?#p#分页标题#e#
Though the mode of thought I’ll put forward here can be applied to analysis of new media artbroadly, I’d like to focus here on areas of new media art most closely related to cinema throughapparatus—media art containing cameras and screens, where, though reconfi gured, there remainsan orientation between viewer, maker, camera, objective reality, capture, screen, and viewing
space. I’ll use the term ‘Augmented Reality’, but we could call it ‘Mixed Reality’, or someother indexical subsection of new media; there is a good bit of slippage and overlay in theseterms. Looking to Wikipedia to locate emerging defi nitions we fi nd under Augmented Realitythat the critical components of this medium are the juxtaposition of the virtual or computationaland the real, an ‘invisible’ interface based on a somehow immersive environment not an iconicor semantic interface, and, it notes, especially referring to machine vision and tracking (there
are our technological apparatus indicators). Ronald Azuma, in his early survey of AR (1997),describes its goal in contradistinction to VR, as a seamless integration of virtual and real, but
I think this is too narrow, and too closely related to the absolute cinema dreams of VR. ARfrom its earliest iterations in the Boeing projects to provide wiring diagrams for technicianscrammed into the belly of airplanes, sought instead multiple confi gurations of the virtual andreal, frequently, as in the case of the diagrams, desiring to in fact retain the distinction betweeneach. What is elemental, though, is the process resulting in the juxtaposition—some sort ofrepresentation (even if it is ‘see through’) of objective reality, combined through computationwith an actively corresponding computational creation... and, as Wikipedia notes, especially inthe visual sense. Azuma’s diagrams of what AR is, in fact, reminds me of Jean-Luis Baudry’sapparatus diagrams of cinema. Jean-Louis Baudry’s 1970 essay Ideological Effects of theBasic Cinematographic Apparatus, seeks in technological analysis a new understanding of howexpression and reception function in cinema, what is repressed and what emphasized, and whatare the limits of subversion. He draws this diagram, which I’ve redrawn here. Note that the
infl uence of the director or artist can be inferred in a number of the processes, both throughdirect intention and through technology... editing, scenario/scripting, and so on... but insteadof making specifi c distinctions, Baudry takes as the basis of analysis the components of theapparatus, both appareil and dispositif... technical and social structures. Baudry notes, with
Azuma, more lovingly, draws this:
the dotted line, the inherent ideology in the suppression of some of the components... where the
perception of the audience is of objective reality represented within the screen, and the goal of
the artist is to suppress perception of the intervening processes.#p#分页标题#e#
My project here is to reconcile these two drawings... to try to begin to understand this changingapparatus of AR in the way Baudry has applied to cinema, and attempt to draw some of the sameconclusions about what is suppressed, ideological, expressive, limited, emphasized, and unseen
objective reality/
lightscenario/
decoupage
camera/
fi lmstock
sound recording
montage
projector/
fi lm/
light
screen/
projection/
refl ection
spectator
from: Jean-Louis Baudry,
Idological Effects of the Basic
Cinematographic Apparatus
. 1970 Cinéthique... translation
1974 by the Regents of the
University of California... Film
Quarterly 1974
Cinema Apparatus
in AR
Let me take several examples to outline some territory in augmented or mixed reality within
the art gallery. David Rokeby is an artist whose interactive work made an early entrance to
the art gallery. His famous 1988 piece, Very Nervous System used motion tracking to create
an invisible interface to computer sound creation. His subsequent gallery work has been
mainly visual, relying on live camera, and frequently incorporating this live-camera image
into the gallery space. His 2002 installation, Seen, installed at the Canadian Pavilion of the
Venice Biennale, used what could be termed a screen-as-window arrangement. The intended
留学生dissertationconstruction was four live-feed video cameras whose footage would be altered in real time with
temporal, spatial, and chroma fi lters, and then projected into four augmented-image screens.
In actual fact, he was not able to arrange the hookup for the live cameras, and used instead 30
minutes of pre-recorded footage. But so important to Rokeby—and, in fact, to the apparatus
defi nition of AR—is the live nature of this work that he still describes the apparatus as livecamera
based in his description on his website, and only notes in parenthesis that this was not,
in fact, true. These ‘live camera feeds’ are then presented in four augmented screens, each
deconstructing the temporal image in different ways: the middle two screens display only the
elements of the image that are in motion or static, and the other two screens provide algorythmic
changes of the image, based on combining multiple frames in time, tracing the paths of motion
that have taken place in the image. A second work by Rokeby, the 2002 Cheap Imitation, is#p#分页标题#e#
confi gured as a ‘magic mirror’—a technological constellation that is fairly common within AR in
the gallery as well as performance. The monitoring camera is pointed out at the viewer from the
screen, and instead of projecting the analogous mediated image of the viewer, the screen presents
an augmented image... in this case the viewer/ fi gure of the art work is presented, through motion
tracking, as distorted segments of Marcel Duchamp’s Nude Descending a Staircase... or, more
acurately, the subject’s observed motion triggers the revealing of segments of the Duchamp
Seen, David Rokeby, 2002
Cheap Imitation (screen view), David Rokeby, 2002.
Be Careful, Fragile, Clara Boj and Diego Diaz, 2006
Level Head, Julian Oliver, 2008
image directly in front of their motion, so that if they move their hand, directly in front of this
movement that part of the image is revealed.... if they move their whole body, a larger section of
the image is revealed, corresponding to their body.
In a totally different aesthetic attitude, there is a screen-as-mirror piece by Clara Boj and Diego
Diaz, Be Careful, Fragile which was created for the 2006 ARCO exhibition in Madrid, Spain.
This magic mirror does not provide an affected mediated image, as Cheap Imitation does, instead
it uses a more functional aesthetic of AR and motion tracking, overlaying a 3 dimensional virtual
object, in this case a vase, on top of the mediated image using a fi ducial marker tracking system.
The gallery visitor is presented with an empty pedestal, identical to many of the pedestals at the
exhibition holding real artwork, behind which is a screen that shows the gallery space, as if in
a mirror, except the pedestal holds a virtual-looking vase, it’s contours and color synthetically
regular. When the view passes their hand over the top of the pedestal, crossing the visual path
between the live camera and a black-and-white printed marker on top of the pedestal, the screen
shows the animated image of the vase falling to the fl oor and breaking. Then, in the screen, the
viewer sees the two artists arrive and place another identical vase on top of the pedestal. Boj and
Diaz, in their description, say the work “questions its own presence, as digital artwork, in the art
market... showing that there are no “unique digital artworks.” Unlike much of Rokeby’s work,
Be Careful, Fragile could be described as sheerly conceptual, lacking any effort for an aesthetic#p#分页标题#e#
sublimity, but within the context of conceptual art, it succeeds, successfully representing a
concept within a gallery art frame. A fi nal AR piece I want to note, is Julian Oliver’s Level Head
(which he, in fact, terms a “spatial memory game,” but at the same time lists the art galleries and
festivals it has been exhibited within). Like Be Careful, Fragile, Level Head uses fi ducial marker
systems—tracking embedded in an object—but here the object is also the site of what will be
replaced with the virtual, so it becomes a type of magic object within the mirroring screen.
Unlike Be Careful, Fragile, the object embedded with the AR markers is the art object itself.
The cube is meant to be handled and shown to the camera, which gives back an image, or partial
image, of the user, along with an augmented image of the cube, where each side is replaced
with relative, relating animations: a digitally drawn human character that walks labrynthine
paths through a confusing, Escher-like cubic world. Like Rokeby’s Seen, it’s interesting to
note the language of his description, where he states the only interface is the marker cube...
because the projection as mediated reality is so common, it is suppressed in the description of the
apparatus—not considered as part of the interface.
If we take these three orientations of cameras and screens in the art gallery, what can we say
about the elements of the apparatus? How do they differ from those of cinema in the structuring
of circuits between objective reality, camera, projector, and spectator, and those things that come
in between, and can we fi nd within this process of creating a mixed reality what is suppressed
in the circuit, and what is presented as the representation or art. If we take the basic AR circuit,
and combine the apparatus diagrams of Baudry with the diagrams of Azuma, we might have
components like those drawn here, containing objective reality, subject/ spectator, camera/
capture, mediated reality, computation/ tracking, process / scene generator, render/ video
compositor, and screen. I have also included in brackets within these chains aknowledgement
of the input that comes from outside the electrical apparatus; such things as the framing of
participation, framing of reception, and the produced content added as augmentation. We can
make some generalizations about what is suppressed and what used to express within these
different types of AR artworks.
All areas of AR seek to suppress the computation/tracking and render... for all AR a system
crash, rasterizing issues, registration errors are considered against the goal. The interface
should be “invisible,” and the computer should not reveal its inhuman character, displaying the
sharp edges of computation, fundamental errors of representation, or, most repressed of all, the#p#分页标题#e#
digital code that lies underneath (such as a ‘screen of death’ when not only a program, but the
underlying operating system kernel crashes). In fact many artists, such as Rokeby, emphasize
their desire to subvert the “digital” nature of the machine, and produce, instead, image that is
evocative of the human or organic. But within this wide generalization of suppression that is in
almost all of new media art, there is a diversity of how the perception of objective reality and
computation, mediation (or ‘virtuality’... a bad term) should co-exit. We can see a spectrum,
where at one end there is Virtual Reality, and at the other end, sheerly industrial applications.
In Virtual Reality the goal is to completely suppress Objective Reality and Mediated Reality
(that is, suppress any perception of mediation within the media) and achieve an absolute cinema
of fi rst person perspective. At the other end are the AR proto-typical Boeing wiring diagram
interfaces, which used see-through displays over which were overlayed tracking diagrams, with
the specifi c goal to neither suppress objective reality, nor to convolve the virtual and the real. In
mediated reality objective reality
capture/
surveillance/
camera
Processor/
scene generator/
fi lter
Difference /
Augmentation
video compositer/
render
screen
spectator
Augmented Reality Apparatus: Window
tracking/
computing
[framing]
objective reality/
Installation
Subject/
Figure
mediated reality
capture/
surveillance/
Processor/
scene generator/
fi lter
video compositer/
render
screen
Augmented Reality Apparatus: Mirror/ POV
tracking/
computing
Artist/
Figure/
Objective Reality
screen/
mediated reality
Video Art Apparatus: Acconci
spectator
camera
frame/
mis-en-scen
Video Art Apparatus: Campus
objective reality/
Installation
Subject/
Figure
mediated reality
capture/
surveillance/
camera
Processor/
scene generator/
fi lter
Difference/ Augmentation
video compositer/
render
screen
Augmented Reality Apparatus: expanded with dispositif
tracking/
computing
limits of allowed
performance/ activity
framing
objective reality/
Installation
Subject/
Figure
capture limitations/
camera
screen limitations/
installation THE
WALL/
mediated reality
projector
between these two are the projects of the art gallery which we have been looking at, which seek
different degrees of suppression and convolvement between virtual and real.
In the Magic Mirror type display, such as Rokeby’s Cheap Imitation, the object of representation#p#分页标题#e#
is the mediated subject or viewer and his or her surrounding objective reality. In a site-specifi c
mode, objective reality is not suppressed, but instead the area of expression is within the
juxtaposition of the real, the live mediation, and the illusion. So that the experience of viewing
Cheap Imitation is one of checking and inspection... the viewer presents themselves to the live
camera and checks for their own representation in the mirroring display. The difference between
the imagined refl ection/mediation of a live mirroring camera, and the observed augmentation
displaying fragments of Duchamp—the lack and excess that occurs between ‘expected’ and
found—is the space of expression. In the case of the Window displays, such as Rokeby’s Seen,
the mediated is presented as real (thus Rokeby’s mendacious insistence that the pre-recorded
footage was live)... the mediated piazza San Marco is to be perceived as the real object being
effected, and, as in fi lm, to be taken as objective reality—though the suppression is even greater
than in fi lm, because it is to be taken as a circuit from objective reality... the live broadcast...
the analogue of the thing itself. So again, the area of expression is the difference between the
image of the piazza that the viewer imagines the camera to be seeing, and the augmented images
playing with duration on the screens. But here there is no checking, as in the mirror, instead
perhaps a backwards analysis... by searching the augmented image of the piazza and drawing on
their own experience and knowledge, the viewer imagines what the non-augmented image might
be, and thereby locates at least the area and nature of the augmentation. Similarly we can see
different goals in the computational processing of the different AR art works. Rokeby’s fi lter
based augmented projections I would call ‘Affected Reality’. The mediated is not suppressed,
instead everything is rendered to this fi nal product. It is like what Brenda Laurel has described
as a stage, where all the behind the scenes capture, mediation,and computation, are rendered to
a single digital representation, with no other artifact [Laurel 2001, p110]. In this type of work,
the computational nature of the processing is frequently extremely suppressed. Rokeby is an
exemplar in this respect. In a lecture for Info Art, he describes his aesthetic goals in computation:
“the computer as a medium is strongly biased and so my impulse while using the computer
was to work solidly against these biases. Because the computer is purely logical, the language
of interaction should strive to be intuitive. Because the computer removes you from your
body, the body should be strongly engaged. Because the computer’s activity takes place on the
microscopic scale of silicon wafers, the encounter with the computer should take place in humanscaled#p#分页标题#e#
physical space. And because the computer is objective and disinterested, the experience
should be intimate.” But, at the same time that the nature of computation is suppressed, the fact
of augmentation—the fact of the mediation being manipulated and affected—is emphasized as
exactly the site of representation and expression. The discontinuity between real mediation and
illusionistic augmentation is not suppressed—as in cinema when the CG dinosaurs are made
to combine realistically—but played upon. We as receivers are looking within the art work for
what is affected in the image and what remains in order to interpret the representation. With the
the two examples of object-based AR in the gallery—Be Careful, Fragile and Level Head—the
role of the mediated is different. The mediated is to a larger extent suppressed. Instead the
mediated image is to be taken as a valid reality, and it is within the mediation that the borders
between virtual and real are presented. The juxtaposition between illusion and objective (albeit
mediated) reality is emphasized as metonymy and addition within the screen... an augmentation
of space. In a sense, within these types of augmented reality, the point of entrance to the work
is in the edges between real and illusion—these sharp edges of addition. It is not an affected
mediation. This is why, at least for me, the fi ducial marker based work carries a special
fascination: the marker as a material object becomes a frame for this edge... a materiality of the
digital code, similar to the aesthetic preciousness of the fi lm sprockets for structuralist fi lm.
In all these works I think you could argue that the ground of representation is the circuit itself...
the mediation taking place—like Rokeby’s insistence on the live nature of the cameras. The
basis of representation is the live nature of the transmission—the mediated as a component...
a live analogue of reality, and exactly this circuit which is being augmented. In many ways
the body—either the body as subject-object as is the case with mirror displays, or the body
as depicted and showing presence and scale as in the Piazza San Marco—is the delineating
edge of augmentation. In the case of the magic mirror, part of the game is that our bodies
dually inhabit the non-augmented space and an augmented space. In a sort of Christian Metz,
cinematic identifi cation schema, it is like a tertiary stage of identifi cation (beyond his secondary
narcissism) where we must have embodied mediation itself in our perception to identify. This
circuit as a material for expression is not essentially digital, but instead, electric... based off of
the live nature of the electric image, but taken to its height in digital augmentation. We can, in
fact, see its relationship to the criticism of video art by Rosalind Krauss in the 70’s.#p#分页标题#e#
In her 1976 essay, Video and the Aesthetic of Narcissism, Krauss defi nes her critical search in
the new medium of video art as for “an object-state, separate from the artist’s own being, through
which his intentions must pass” ...like the pigment bearing substances of painting, and the
matter through space of sculpture. She defi nes the crucial element of the electric image in this
regard as the instantaneity of the communication from notion to message: “This is why it seems
inappropriate to speak of a physical medium in relation to video. For the object (the electronic
equipment and its capabilities) has become merely an appurtenance. And instead, video’s real
medium is a psychological situation, the very terms of which are to withdraw attention from
an external object—an Other—and invest it in the Self.” The object is bracketed out, she says,
and instead the artist is creating within a psychological state created by the mapping of the
mind onto this network, in a very McLuhanesque the-medium-becomes-the-nervous-system
way. In her analysis, this self-gazing video art is, for the viewer in the art gallery, like the
viewing of an electronic loop of the artist, camera, and screen. AR, instead, is this loop itself
installed in the gallery, now including the apparatus within the gallery. The actual apparatus of
the electric image medium are installed. Lars Qvortrub, in an essay on digital poetics, proposes
that all of new media art could be described as ready-mades, where instead of R. Mutt’s urinal,
there is now a computer, a projector, a screen, a camera... “digital materials have realized the
intentions of the avant-garde” he says [p241]. But AR art has complicated the ready-made; it
is an art of apparatus, but not of the objects themselves, but the objects plugged in to eachother,
broadcasting to eachother and constantly looping in real live time... art of circuits. The apparatus
is released from the linear narrative of cinema because it does not have to be linearly productive
(in the sense of Leotard’s acinema), because it is constantly in movement. If we take this
electric circuit as the fundamental material of expression within AR art, then I think the next
step is to come to grapple with how this circuit augments the viewer. Like Krauss, we can try to
留学生dissertationunderstand what this means for a psychological state. If video art of the 70’s was appropriately
placed to deal with issues of surveillance, celebrity, and mediation coming out the tradition of
pop art, then what is AR art poised to express? We could begin by saying it is a state of postmediation.
We are no longer seeking to grapple with mediation, but have embodied it, and are,
instead, grappling with our relationship to the interface and machine. And, if we have succeeded#p#分页标题#e#
in creating an acinema—breaking narrative but retaining movement—then what are the new
ideological limits of the new apparatus.
