INFORMATION TECHNOLOGY

Multimedia Data Base at Carnegie Mellon Lets You 'Interview' Albert Einstein

By VINCENT KIERNAN

The renowned theoretical physicist has just been asked what advice he could offer to young scientists.

Einstein cocks his head slightly, takes a breath, and launches into a disquisition on the value of living a solitary life in a remote lighthouse.

"Would it not be possible to fill such places with young people who wish to think out scientific problems, especially of a philosophical or mathematical nature?" he asks.

ALSO SEE: Length, 0:52 Real Video (28.8 or better required) A brief introduction and demonstration of the "synthetic interview" of Albert Einstein.

Such a solitary life could lead to scientific advances and great fulfillment for those ensconced in the retreats, he says.

A newspaper reporter may be in no position to challenge the genius who formulated the theory of general relativity, but his answer seems a little off target.

In a brief interview at a multimedia arts studio here, Einstein also explains that he values his unkempt appearance because it discourages people from pestering him. "If you find me so repulsive, then look for a friend who is more appealing to female tastes," he says.

Einstein even calmly discusses his own death, in 1955, and the subsequent study of a sample of material from his brain. It found that he had an unusually high number of glial cells, which provide nourishment to the brain's neurons.

The interview is neither seance nor nightmare. Rather, it is a "synthetic interview" -- an interaction between a human and a sophisticated multimedia data base compiled by researchers at Carnegie Mellon University's Entertainment Technology Center.

The image in the data base is not really that of Albert Einstein, but of an actor named Jerry Mayer who plays the part complete with rumpled sweatshirt and white hair. He was recorded on about 500 video clips as Einstein, presenting the physicist's sayings, quotations from interviews he gave, and excerpts from his writings.

The user poses questions by speaking into a microphone. Speech-recognition software, running on an ordinary P.C. with the Windows 95 operating system, analyzes the question and determines which of the recorded quotations comes closest to offering an answer.

In analyzing questions, the system relies on two vocabularies. One is a set of 5,000 words from general usage.

ALSO SEE: Length, 0:17 Real Video (28.8 or better required) The synthetic Albert Einstein gives a brief description of relativity.

The other is a smaller set of terms that someone interviewing Einstein would be likely to use, such as "relativity" and "quantum."

The two dictionaries are small enough to fit on a personal computer, but together they enable the system to handle most inquiries about Einstein, says Alex Hauptmann, a systems scientist in Carnegie Mellon's school of computer science who developed the speech-recognition technology.

The system relies in part on algorithms from a World-Wide Web search engine called Lycos, which analyzes a user's inquiry to find Web sites that match. Lycos also was developed at Carnegie Mellon.

Once the interviewer's question has been dealt with, the most appropriate snippet of digitized video of Mr. Mayer is played on a video-display device that provides the appearance of a three-dimensional image. Researchers at the university think the technology could be used for new forms of entertainment as well as for educational purposes.

"People feel like they have really spoken with Einstein," says Scott M. Stevens, co-director of the Entertainment Technology Center, who helped to devise the synthetic-interview technology.

Mr. Mayer, the author of a one-man play about Einstein, was selected to play the part because he had mastered the physicist's persona, Dr. Stevens says. "He could be Einstein in a way that another actor could not."

Mr. Mayer plays the role using clear, unaccented English, however. By contrast, Einstein's English was heavily accented. "He was almost unintelligible when he spoke in English, his accent was that thick," Dr. Stevens says.

The Carnegie Mellon researchers compiled the data base by asking people of various ages -- from fourth-graders to physicists -- what questions they would ask Einstein if they had the chance.

Of the 500 or so individual video clips, 90 to 95 per cent re-create what Einstein either said or wrote, Dr. Stevens says. The contents of other clips -- including Einstein's explanation of what happened to his own brain after he died -- were written by the researchers.

The system allows the virtual Einstein to hold forth on subjects as varied as the history of physics, his role in the creation of the atomic bomb, the value of marriage, and his favorite composers, authors, and artists.

The data base also includes other video material. For example, on request, Einstein will play a movie of his arrival in the United States.

Besides recognizing spoken questions, the software permits users to type inquiries on a keyboard. Using translated transcripts of the video clips, the program can even provide written responses to questions typed in German or Italian. "If you ask in Italian, you get an Italian translation," says Dr. Stevens.

Dr. Stevens says Einstein scholars have not assessed the synthetic interview system. Several scholars reached by The Chronicle said they had not heard of the data base or knew too little about it to offer an opinion.

However, Dr. Stevens himself says the data base suffers from some omissions and errors. For example, it offers no comments by Einstein on the topic of Brownian motion, a type of disturbance undergone by small objects suspended in liquid or gas. Some of Einstein's earliest major scientific contributions were in this field. "I'm an ex-physicist," Dr. Stevens says. "I could have kicked myself for forgetting that."

The Carnegie Mellon researchers have also used the synthetic-interview technology to record comments by subjects who are still alive. One of their first projects was an interview with "Lisa," a recent graduate who now is pursuing an acting career on the West Coast. In about an hour, they recorded her answers to about 120 questions on her life and career. The idea was to demonstrate how the technology could be used or an entertainment product that could be sold to a celebrity's fans, Dr. Stevens says.

He thinks synthetic interviews might be a viable alternative to on-line appearances by celebrities in "chat rooms." Such appearances can attract millions of fans, placing great stress on an on-line system and reducing to almost zero the chance that any one individual's question will get an answer from the celebrity.

The researchers have also created

Bill Redic for The Chronicle Scott M. Stevens (left) and Don Marinelli of Carnegie Mellon helped create the technology that allows people to interact with Albert Einstein.

a synthetic interview with Kurt Angle, a wrestler from the Pittsburgh area who won a gold medal in the 1996 Olympics in Atlanta. He fields questions about topics as diverse as his plans for future competition and how heavy the Olympic medal is.

The technology could be put to more serious uses as well, Dr. Stevens says. He envisions a synthetic interview, focusing on surgical techniques, with an experienced neurosurgeon. Less-experienced neurosurgeons preparing for surgery would ask questions of the data base. Any questions that it didn't cover would be forwarded to the live expert -- and the question, along with the answer, would be added to the data base.

In addition, the Carnegie Mellon researchers are planning improvements to their technology. For example, the system now does little to keep track of the overall flow of the interview, says Dr. Hauptmann, the speech-recognition specialist. "That's where there's a lot of work still to be done." With the current technology, a user might pose the same question over and over again, he says, and the synthetic-interview system would never realize that the user probably wasn't getting the answer that he or she was seeking.

He hopes to modify the system so that it would stop regurgitating similar answers after a user had asked repeated questions on the same topic.

In June 1996, Carnegie Mellon formed a spinoff company, Grand Illusion Studios, to market a commercial version of the synthetic-interview technology. All four researchers who created the technology have senior positions in the company. Dr. Stevens is the company's director of technical development; Dr. Hauptmann is the company's director of speech-recognition technology; Donald Marinelli, a professor of drama and arts management at Carnegie Mellon, is the director of creative development; Michael Christel, a researcher in Carnegie Mellon's computer-science school, is the company's director of technology.

Grand Illusion is owned in part by the university and in part by W.R.S. Motion Picture and Video Laboratory, a Pittsburgh-based film-production studio, where The Chronicle's interview with Einstein took place.

Dr. Stevens and his colleagues envisioned several ways to package the technology -- offering it from Web sites, in CD-ROM form, and in kiosks. A science museum could offer patrons the opportunity to interact with a long-dead scientist, and a school might pay to allow its students to do the same via the Internet, says Greg Thomas, president of Grand Illusion and executive vice-president of W.R.S.

A store could use the technology in a kiosk to allow customers to interact with a celebrity spokesperson for a product, and interviews with mass appeal might be marketed as CD-ROMs, he says. Most late-model P.C.'s come equipped with all the hardware and software needed to play such a product, Dr. Stevens says.

The cost of producing a synthetic interview might range from $70,000 to $200,000, depending on the complexity of the interview, says Mr. Thomas. The company is discussing synthetic-interview projects with two museums in Pittsburgh and is forecasting 1998 revenues of $2-million, he adds.

Dr. Stevens thinks that synthetic interviews with major figures of science could be a valuable addition to science museums. If young men and women could "talk" with Einstein or Newton or Galileo, he says, they could be inspired to follow careers in science.

To him, perhaps the most significant market test of the technology came last year, when the Carnegie Mellon researchers exhibited the Einstein interview in March at a conference on the future of computing held by the Association for Computing Machinery in San Diego.

A maintenance worker in the conference center took an immediate liking to the system. He stopped by again and again to pose questions to the virtual Einstein. One day, he brought a friend -- whom he promptly introduced to the simulacrum.

At the same conference, Dr. Stevens recalls, three 12-year-olds were enraptured by the device. "They spent 40 minutes asking questions of Al. Their teacher had to pull them away."