A networked art installation in which test-taking robots behave according to how users engage with a website composed of questions from real U.S. standardized tests.

by Christian Croft



"Rarely is the question asked: Is our children learning?"
                                                                                            --President Bush, Florence, S.C., Jan. 11, 2000

OVERVIEW

     The 'Is our machines learning?' machines is a networked art installation in which test-taking robots behave according to how users engage with a website composed of questions from real U.S. standardized tests. The physical installation consists of machines that are mechanically capable of making marks on standardized test forms with a pencil. These machines rest on top of antique school desks, ready to pencil in multiple-choice bubbles on SCANTRON test forms. In a separate online space, visitors coming to a website determine which multiple-choice answers the machines in the installation select to fill in. At this website, users can watch the machines respond to their input via a streaming video feed from the installation.

THE INSTALLATION

     The physical installation for this project consists of a custom designed machine mounted on an antique school desk. This machine is built from lasercut acrylic pieces, aluminum rods, and electric motors. A PIC microcontroller controls the movement and timing of the motors on the machine to drive a pencil to fill in multiple choice bubbles. The machine's microcontroller is connected serially to a PC in the installation to receive commands from the project website about which test bubble to fill in. A network IP camera positioned next to the machine sends video images of its movement to the project website.

Watch an early video of the machine in operation here.

THE WEBSITE

     The project website presents the user with a testing environment in which she can participate by answering multiple-choice questions. Each time a user responds to a question, she can receive dynamic visualizations of data specific to that question along with a streaming video view of a machine in the installation marking the average user-selected multiple-choice option. Before launching into a testing session, the site takes users through a registration page of profiling questions. This login survey begins with standard age and race specification and progresses to more ridiculous questions such as whether a user prefers Coke or Pepsi. After registering, multiple-choice questions are presented one at a time to the user. Once the time for a question expires, the website's backend programming calculates the total answers of the visitors on the website at that instant. On the webpage layout beside the test question, the user sees a graph appear visualizing the number of users who chose each multiple-choice option. Drawing on information gathered from the registration survey, this graph also shows statistics of how different categories of users stack up for each question. Above the graph, the user sees a machine come to life in a streaming video window.

     Multiple-choice questions used on the website come from official standardized tests administered by the National Assessment of Educational Progress, a U.S. governmental agency. The NAEP maintains a website with example questions from past tests that feature statistical breakdowns of achievement scores for each question. For this project, I scraped all example questions and corresponding meta-data about student performance for each question from the NAEP website ( located at http://nces.ed.gov/nationsreportcard/itmrls/ ). Some of the real student performance data will be integrated into the information visualization juxtaposed with new user data at the project website.

IMPLEMENTATION AT THE 2007 ITP WINTER SHOW

Although the physical installation and the web interface to the project are meant to be experienced separately, the two components will be presented side by side at the ITP Winter Show. Due to space constraints, the machine will be set up next to a computer terminal offering access to the project website. Thus, a show visitor will be able to see the physical component of the piece as they interact with the website. Thus, a visitor will be able to see the streaming video on the website simultaneously with the actual machine's movements. Of course, people who are not able to come to the show will be able to have the intended remote experience at the project website.