'Augmented Reality' on Smartphones Brings Teaching Down to Earth

Courtesy David Gagnon

Researchers at the U. of Wisconsin at Madison created software called ARIS, or Augmented Reality and Interactive Storytelling. It allows educators to combine lessons via smartphones with cues from a student's real location.
June 20, 2010

At the University of New Mexico, some students in second-year Spanish classes become detectives. They travel to Los Griegos, an Albuquerque neighborhood 15 minutes northwest of the campus, on a mission: Clear the names of four families accused of conspiring to murder a local resident.

It's a fictional murder mystery, and instead of guns and badges, the students are armed with iPod Touches, provided by the university. When students enter their location into the wireless handheld devices, a clue might turn up: a bloody machete, for example, or a virtual character who may converse with them—in Spanish—about a suspect.

But Los Griegos and the language skills needed to navigate the locale are no fiction. By integrating mobile computing and actual surroundings, the educational game, Mentira—Spanish for "lie" and a reference to the claim of conspiracy the students are assigned to debunk—helps take teaching to a new place outside the classroom: "augmented reality."

Video and computer games are commonly criticized for isolating players from reality, but augmented-reality developers who work in higher education see the technology as a way to accomplish just the opposite.

"Real life is pretty high-res," says David J. Gagnon, a faculty consultant and instructional designer at the University of Wisconsin at Madison. Augmented-reality games, he says, are a way to help people "get out and see that."

Mobile by Design

Mr. Gagnon is the lead developer of a software tool called ARIS, or Augmented Reality and Interactive Storytelling. ARIS lets designers link text, images, video, or audio to a physical location, making the real world into a map of virtual characters and objects that people can navigate with iPhones, iPads, or iPod Touches.

The open-source tool, which is the brainchild of a Madison research group that focuses on games and learning, was built with students and educators in mind. It has not yet been released to the public; developers are aiming for a fall rollout.

Mentira was created by Christopher Holden, an assistant professor in the honors college at New Mexico, and Julie M. Sykes, an assistant professor of Hispanic linguistics at the university. They used a limited-release, early version of ARIS. The software is simple to use, they say, and doesn't call for special programming expertise.

But ARIS isn't the only game in town. Eric D. Klopfer, an associate professor of science education and director of the Teacher Education Program at the Massachusetts Institute of Technology, has created two similar tools and has worked with augmented-reality games for nearly a decade.

His team used to rely on personal digital assistants, which he calls "clunky units," that received GPS readings only in good weather and with an antenna.

But now the technical challenges are "fading away," he says. GPS systems have become more accurate. And they have become phones. Increasingly popular GPS-enabled cellphones come equipped with cameras and other features that open up new avenues for enhancing reality-based games.

Arguments for Augmentation

The researchers and educators in this small, emerging field see clear advantages to using real-world sites as the backdrop for educational games.

A major goal of Mentira is to motivate students "to get their heads out of the textbook" by showing them that language has a vibrant local context, Ms. Sykes says. By setting the story in a nearby neighborhood, she and Mr. Holden took advantage of its historic sites and folklore to integrate learning about its history and culture into the game.

Likewise, Mr. Klopfer calls place-based learning with augmented reality a "great match" for topics at the intersection of science and society, like public health and environmental issues.

In one of his projects, a game about environmental contamination on MIT's campus, students showed a more nuanced understanding of how to handle a toxin's spill after discovering its source. Unlike students who played an entirely virtual version of the game on a computer, those who played the reality-based version appreciated that they were making choices "influenced by the community reality," he says.

In Mentira, too, students learn to weigh the consequences of their choices. For instance, they select what to say in their "conversations" with virtual characters, and their decisions affect the characters' responses. The setup teaches them how to accommodate the conversational styles of the characters—for example, how to speak politely to an older man, Ms. Sykes says.

Games like the New Mexico murder mystery only scratch the surface of what is possible, says Mr. Gagnon. He has seen a variety of other proposals for using ARIS in courses, including ethnographies that tie interviews to specific places and a scavenger hunt that teaches students to identify plant life.

This year's "Horizon Report," which forecasts tech trends in higher education, named augmented reality as one to watch. According to the report, which is produced in a collaboration between the New Media Consortium and Educause, technology that blends the virtual and the real is expected to enter mainstream use in teaching in the next two to three years.

Teaching with augmented reality is not all fun and games, however. Mr. Holden and Ms. Sykes struggled to find an affordable way to make their game a reality. They chose iPod Touches instead of costlier iPhones. As a result, they had to design a game that would work without GPS navigation and persuade the university to sign a contract for a mobile wireless hotspot.

Beyond the obstacles to getting the technology up and running, the duo say, they are still learning how to fit the game to an existing course. "Spanish 202 classes already have their syllabus," Mr. Holden says. Similar challenges are likely to surface as others look to augment their own teaching.

"The idea of it is something people get really excited about, myself included," Mr. Gagnon says. "The practicality of it is still something we really need to work on."