Project 2: Shadowboxing

The goal of this project is to gain familiarity with prototyping body-driven interfaces. We will begin with shadowboxing: interactive systems that draw on digital projections of the human body. Check out some of Scott Snibbe's work for examples. This is an introduction to the kind of input that you will get from a depth map on a Kinect sensor. The input is noisy, and you have no joint information like the Kinect will provide, but even shadows provide many opportunities for interaction.

You will be creating a shadowboxing application designed for public engagement. The interactive prototype should be displayable in a public place. The only input to the application is peoples' bodies. Your challenge is to pick a venue where a shadowboxing prototype could be deployed, and to design an application for that venue. We will provide the basic scaffolding to isolate shadows onto an HTML5 canvas using a webcam — the rest is up to you!

You and your team should choose a public engagement space and brainstorm as many different ideas as you can. Try bodystorming. Carry at least three designs out to a mature set of sketches that would convey the interaction clearly to another designer or implementer. Choose your ideas to be interestingly different!

We will provide scaffolding code that performs background subtraction and writes shadows to the 2D context of an HTML5 canvas element. Click the button (or call the function) to save the background pixels, and everything that changes after that will be detected as a shadow by the application code. There are many resources online for HTML5, Javascript and canvas. Here are a few good ones: [basic 1], [basic 2], [advanced 1], [advanced 2].

Some webcams, for example the Macbook's built-in iSight, will adjust their exposure continuously as new people enter and leave the frame. This can interfere with the background subtraction. If this is a problem for you, you will need to find a way to lock the exposure for your webcam. On Macs, you can install a demo of iGlasses and click "lock exposure".

You are also welcome to use your own framework and libraries (e.g., OpenCV) for the application, but remember not to sacrifice interaction ideas for the sake of getting a prototype running at 30fps. You may also choose to use other hardware. We can check out a Kinect to your group if you would like. We will be officially supporting KinectJS for the class. Here are instructions for how to set up KinectJS.

You can download the scaffolding code from:

We recommend hosting your code using a public repository on the CS 247 organization in GitHub. It provides hosting, task management, and source control. GitHub will be required for later assignments, but you are welcome to use it with this one as well.

Deliverables. The first and easiest deliverable is to sign your group up on this form by Tuesday January 15 1:15pm. There are three primary deliverables, all due on Tuesday January 22 by 1:15pm. The project should be submitted in the form of a link to a project web page. Email the URL for your P2 page to cs247@cs.stanford.edu. Your web page should include the names of all project members as well as the following:

  1. Design sketches
    Keep a record of your P2 design process, including brainstorming ideas and design sketches. Turn in sketches that demonstrate a broad exploration of many ideas, followed by thoughtful detail for three ideas. The broad exploration should show evidence that you explored the design space and considered many options. The detailed design sketches should go into enough detail for another designer or implementer to understand what is happening. Make sure that your sketches communicate your envisioned users and their context of use: where, when, and why would people be interested in your public engagement application? Use your project web page to tell the story of how you developed your shadowboxing design and application.

  2. Demonstration video
    Make a short demo video (no more than 2 minutes long) showcasing your working shadowboxing application. Upload the video to YouTube or Vimeo and embed it in your project page.

  3. Project code
    Turn in a zip archive of your development workspace, including all source code. If you are using the CS 247 Github organization, send us the link to your repo. We should be able to simply open the project on a localhost server and run the application. Please provide any special instructions if needed. Your P2 web page should include a hyperlink to your code archive.

You must work in groups of four for this assignment. This does not need to be the same team as your final project, so we encourage you to explore and meet new possible team members.

30%   Design sketches and process log.
60%   Shadowboxing application implementation and fit to application area.
10%   Video demonstration of shadowboxing application in action.
P2 Grading Rubric

Please feel free to e-mail us at cs247@cs.stanford.edu if you have any questions.