November 2009 - I passed my PhD critique successfully!
My committee: Hiroshi Ishii, Edith Ackermann and Cynthia Breazeal. I will now focus on few more studies and building few more projects as much as I can before graduating (in 9 months). A little bit on my presentation …
Gesture Objects: Play it by Eye - Frame it by Hand!
I started with my master thesis Dolltalk, where I establish the ability to access perspective as part of gesture analysis built into new play environments. I then, move into a significant transition phase, where I research the cross-modal interface elements that contribute to various perspective taking behaviors. I also present new technologies I implemented to conduct automatic film assembly.
The structure of my presentation
At each step, I present the studies that allow me to establish principles which I use to build the final project, the centerpiece of my third phase of research, Picture This. At its final point, Picture This is a fluid interface, with seamless integration of gesture, object, audio and video interaction in open-ended play.
With Picture This! children make a movie from their toys views, using their natural gestures with toys to animate the character and command the video making assembly. I developed a filtering algorithm for gesture recognition through which angles of motions are detected and interpreted!
Finally, I developed a framework that I call “gesture objects” synthesizing the research as it relates to the field of tangible user interfaces.
Gesture Objects Framework: In a gesture object interface, the interface recognizes gestures while the user is holding objects and the gesture control of those object in the physical space influences the digital world.
A .pdf of my slides!
June 2009 - My journal paper Play-it-by-eye! Collect movies and improvise perspectives with tangible video objects is now published at Cambridge University Press!
The paper in .pdf ->here<-
We present an alternative video-making framework for children with tools that integrate video capture with movie production. We propose different forms of interaction with physical artifacts to capture storytelling. Play interactions as input to video editing systems assuage the interface complexities of film construction in commercial software. We aim to motivate young users in telling their stories, extracting meaning from their experiences by capturing supporting video to accompany their stories, and driving reflection on the outcomes of their movies. We report on our design process over the course of four research projects that span from a graphical user interface to a physical instantiation of video. We interface the digital and physical realms using tangible metaphors for digital data, providing a spontaneous and collaborative approach to video composition. We evaluate our systems during observations with 4- to 14-year-old users and analyze their different approaches to capturing, collecting, editing, and performing visual and sound clips.
April 13th 2009 - Artist talk & panel.
On the WOW Pod: A Design for Extimacy and Fantasy-Fulfillment for the World of Warcraft Addict
A discussion about the inducement of pleasure, fantasy fulfillment, and the mediation of intimacy in a socially-networked gaming paradigm such as World of Warcraft (WOW). This event is co-sponsored with the MIT Museum in conjunction with the exhibition SHADA/JAHN/VAUCELLE, “Hollowed,” which includes the WOW Pod a collaborative project by Cati Vaucelle & Shada/Jahn.
Panelists include Jean-Baptiste Labrune, Postdoctoral Associate at the Tangible Media Group, MIT Media Lab; Raimundas Malasauskas, Curator, Artists Space (NYC); Henry Jenkins, Co-Director of the MIT Comparative Media Studies Program; Marisa Jahn, Artist in Residence, MIT Media Lab; Steve Shada, artist collaborator; Cati Vaucelle, artist collaborator & HCI researcher, MIT Media Lab; and Laura Knott, Curatorial Associate, MIT Museum.
March 19 - September 13, 2009
Hollowed, exhibition at the MIT Museum!
The exhibition SHADA/JAHN/VAUCELLE reflects MIT’s ongoing commitment to working with contemporary artists whose work often examines the relationships between science, art, technology and humanity. Shada/Jahn’s work — and in their collaboration with Media Artist Cati Vaucelle — “explores, constructs, and intervenes natural and social systems. Ranging in practice from deeply personal to highly participatory, their work often relies on the collaborative authorship and distributive intelligence of surrounding people and situations.”
March 2009 -
The WOW Pod not only received another grant from Eyebeam in New York but will be exhibited at the Mixer event!
March 6, 7, 9 pm 2 am at Eyebeam’s Mixer event (540 W. 21 b/w 10th and 11th Ave, NYC): ‘WOW Pod’ a collaboration by Cati Vaucelle & Shada/Jahn
Participating Artists: Taeyoon Choi + Cheon pyo Lee, Angela Co + Aeolab, Anakin Koenig, Chris Jordan, Caspar Stracke, The Institute for Faith-Based Technology, The Loud Objects, Di Mainstone, Not An Alternative, Mark Shepard with G. Doulas Barrett, Craig Shepard, Daniel Perlin, Al Laufeld and Jon Cohrs, Cati Vaucelle, Steve Shada, Marisa Jahn
Event Summary: Two nights of performance and installations that will turn Eyebeam’s gallery into a temporary village of creatively engineered pavilions themed around the idea of utopia.
The WOW Pod is an immersive architectural solution for the advanced WOW (World of Warcraft) player that provides and anticipates all life needs. Inside, the gamer finds him/herself comfortable seated in front of the computer screen with easy-to-reach water, pre-packaged food, and a toilet conveniently placed underneath his/her custom-built throne. When hungry, the gamer selects a food item (‘Crunchy Spider Surprise’, ‘Beer Basted Ribs’, etc.) and a seasoning pack. By scanning in the food items, the video game physically adjusts a hot plate to cook the item for the correct amount of time. The virtual character then jubilantly announces the status of the meal to both the gamer and the other individuals playing online: “Vorcon’s meal is about to be done!” “Better eat the ribs while they’re hot!” etc. When the food is ready, the system automatically puts the character in AFK (‘Away From Keyboard’) mode to provide the gamer a moment to eat. When the player resumes playing, he/she might just discover his/her character’s behavior is affected by the food consumed in real life — sluggish from overeating or alternately exuberant and energetic. The exterior of the WoW Pod mimics the look of authentic WOW architectural structures, whose swaths of flat, pixellated surfaces digitally recreate the built environment of an imagined past. But upon crossing the threshold and entering into the WOW Pod’s interior, the player finds the digitized look actually becomes the real life experience that World of Warcraft simulates.
February 2009 -
The two papers I wrote for CHI 2009 were accepted this year! One paper is entitled Design of Haptic Interfaces for Therapy the second one, a work in progress, is called Cost-effective Wearable Sensor to Detect EMF . More info ->here<-
January 2009 -
I’ve been awarded a nice grant by SHASS’s Peter de Florez Fund for Humor also for the AFK cookset and the WOW Pod!
December 2008 -
I’ve been awarded a grant by the Council for the Arts at MIT . This grant is to help the design of the AFK cookset and the WOW Pod, projects that will be exhibited from April 2009 until September 2009 at the MIT Museum.
More info ->here<-
November 2008 -
I have successfully passed my qualifying MIT general exam, I am officially PhD candidate! The general exam is composed of three written requirements and one oral exam. More here …
September 2008 - Electromagnetic Field Detector Bracelet -
The “Electromagnetic Field Detector Bracelet” has been accepted as a video for Ubicomp 2008, Seoul South Korea!Abstract
We present the design of a cost-effective wearable sensor to detect and indicate the strength and other characteristics of the electric field emanating from a laptop display. Our bracelet can provide an immediate awareness of electric fields radiated from an object used frequently. Our technology thus supports awareness of ambient background emanation beyond human perception. We discuss how detection of such radiation might help to “fingerprint” devices and aid in applications that require determination of indoor location.
Press on Engadget and Make Magazine.
Jabberstamp earned an honorable mention as one of the 23 finalists from over 2,500 entries to I.D. magazine’s 2008 Student Design Review! Working on Jabberstamp with Hayes Raffle and Ruibing Wang was exceptionally fun and inspiring, I am glad it won an award!!-> The I.D. review <-
A child playing with Jabberstamp and me in the background blurred by the magical photoshop touch!
My paper Picture This! Film assembly using toy gestures has been accepted as a full paper for the technical conference on ubiquitous computing: UbiComp 2008. With an acceptance rate of less than 19% for technical papers in the field, it is very encouraging!
We present Picture This! a new input device embedded in children’s toys for video composition. It consists of a new form of interaction for children’s capturing of storytelling with physical artifacts. It functions as a video and storytelling performance system in that children craft videos with and about character toys as the system analyzes their gestures and play patterns. Children’s favorite props alternate between characters and cameramen in a film. As they play with the toys to act out a story, they conduct film assembly. We position our work as ubiquitous computing that supports children’s tangible interaction with digital materials. During user testing, we observed children ages 4 to 10 playing with Picture This!. We assess to what extent gesture interaction with objects for video editing allows children to explore visual perspectives in storytelling. A new genre of Gesture Object Interfaces as exemplified by Picture This relies on the analysis of gestures coupled with objects to represent bits.
We connect to our world using our senses. Every one of our senses is a knowledge shopper that grounds us in our surroundings : with touch, one feels the texture of life, with hearing one perceives even the subtlest murmurs of our existence, with vision one clarifies their instincts. But human senses are not only about perception. We use gesture to apprehend, comprehend and communicate. We speak to ultimately translate and exchange with others. We visualize, record, and playback events using our memory to reflect on our history and to be immersed in experience. We as children and adults are engaged in everyday pretense and symbolic play. We embed and later withdraw from the world, using imagination to project ourselves into situations . Our mental constructs are necessary to reach a deeper understanding of our relationship with our environment . Children are offered stories by adults and are driven into fantasy play. They use toys to externalize and elaborate their mental constructions . With character toys they create interrelationships and plots, a means to expose their social knowledge: knowing about human beings and social relationships . If the toy has an immediately accessible visual perspective, a new world is opened to the child. The toy brings her into exploring visual and narrative perspectives of character props, expanding the discovery of her environment.
We imagine a world in which people play, create and exchange visual narratives with ease and transparency. Motivated by the playful improvisational environment of child storytelling with toys, we have developed a new category of video editing tools progressing towards the child’s natural expression of play. In Picture This! we combine the activity of play with the video making process. Whereas play emphasizes spontaneity and improvisation, video making necessitates structure and composition. We were inspired by the theater play of Goethe’s childhood , investigating what technology could add to the narrative and play experience. We use technology to offer visual feedback regarding how the scene looks like from the point of view of an imaginary audience. The child storyteller enters the world of the movie maker. Cameras become part of a toy system showing how things look from a toy’s point of view. They can be integrated in Lego people, car drivers, and even coffee mugs! The video process, supported by gesture induced editing, benefits children in practicing social interrelationships and visual perspective taking.
More about the system -> here<-
July 2008 I have authored a chapter in the book Fashionable Technology, edited by Sabine Seymour!
I present my work on fashion garments designed in the context of technology -including the Touch Sensitive apparel developed with Yasmine Abbas. The book just came out and is available for pre-order on Amazon! Pages of my contribution: Abstract: The interplay of electronic textiles and wearable technology, wearables for short, and fashion, design and science is a highly promising and topical subject. Offered here is a compact survey of the theory involved and an explanation of the role technology plays in a fabric or article of clothing. The practical application is explained in detail and numerous illustrations serve as clarification. Over 50 well-known designers, research institutes, companies and artists, among them Philips, Burton, MIT Media Lab, XS Labs, New York University, Hussein Chalayan, Cute Circuit or International Fashion Machines are introduced by means of their latest, often still unpublished, project, and a survey of their work to date. Given for the first time is a list of all the relevant information on research institutes, materials, publications etc. A must for all those wishing to know everything about fashionable technology. ->Buy the book<-