• Tuesday, May 29, 2012

Previous

Next

Technology to Turn Gestures Into Song

February 20, 2012, 8:42 pm

Vancouver, British Columbia—A new technology lets people control a speech synthesizer with gestures, allowing them to speak or sing with their hands. Along with opening new realms of musical expression, research with the speech-generating system may deepen understanding of how the brain drives spoken language and song.

Sidney Fels, a professor of electrical and computer engineering at the University of British Columbia, led the team of researchers that created the device. He talked about it in a session here at the annual meeting of the American Association for the Advancement of Science.

The new technology is mimicking a complicated physiological process. Speech uses 200 muscles between the abdomen and the nose, with the lungs driving air movement and the larynx generating sound.

The speech-generating system (a video demonstration is available for download here) uses two gloves and a foot pedal to control the synthesizer. When the right glove opens it creates vowels, just as the tongue does with its movements in the mouth. Certain gestures create certain consonants, and the left glove makes “stops,” such as the sound for “b.” The foot pedal controls volume, the equivalent of lung pressure. Raising or lowering the right hand also raises or lowers pitch.

With about 100 hours of practice, Mr. Fels says, someone learning how to use the device “can get reasonably good at intelligible speech.” One of the first subjects who used the device said that he had a different personality when he was “talking with the gloves.”

The device makes it possible for singers to have duets with themselves, and there have been seven performances of music written for what is becoming a new instrument. Judging by videos of the performances, it may be a few years yet before the new technology wins any Grammy awards.

The researchers have created a simplified version of the speech technology that can be used on a tablet computer, but the more complex version requires a backpack full of equipment. “You wouldn’t want to go to a restaurant with it to order sushi,” says Mr. Fels.

This entry was posted in Uncategorized and tagged . Bookmark the permalink.

  • Print
  • Comment
  • goxewu

    Because it’s perfectly OK to give legacy admissions to military schools to children of veterans, especially of Medal of Honor winners, then it must also be perfectly OK for colleges to give legacy admissions to the children of rich and/or socially connected alumni.

    Question: Is there a difference between “a proud tradition of military service in this family” and “a proud tradition of silver-spoon and country-club college admissions in this family”?

  • greensubmarine

    J. Robert Oppenheimer’s title in his capacity of scientific head of Los Alamos is my personal favorite: Coordinator of Rapid Rupture.

  • 22185161

    At a former large, multi-facility academic medical center where I previously worked, we had a Wayfinding Committee. I always pictured committee members as having bulging pockets full of bread crumbs which they dropped behind them.

  • 7738373863

    I am particularly fond of the ubiquitous “dean of summer school.”  “Director” would be closer to the truth and need not impact pay grade.  But if one really wants to explore the messy inner recesses of muddled administrative thinking in academe, s/he should take a good look at the titles sported by associate/assistant provosts and deans.  At my institution, one interesting and revealing phenomenon is what happens when one searches for these light-sensitive creatures in the online directory.  They are almost never listed, nor are they found on the units’ web pages.  How about, within a college (at another university), a doctorally prepared but non-faculty “associate dean for student academic life”?  And s/he is only one among eleven associate deans in that college:  too much.

  • esgphd

    I used to be a dean for faculty affairs.  I had to keep explaining to people that the Offfice of Faculty Affairs was, despite the name, not really a dating service.

  • Brian Abel Ragen

    I would love to see a university where the titles reflected the real nature of the job. There would be no provost: instead, their would be a “president’s minder,” “president’s hatchetman,” “pesident’s flatterer,” or “president-in-waiting,” as the case required. And the faculty would give up all professorial pretentions and because simply “fireable teachers” and “malcontents.”

  • missoularedhead

    We often referred to the vice-chancellor and provost simply as ‘the money guy’.

  • 11223435

    I wouldn’t want anyone to jump to the conclusion that any of us might know anything about the porn industry–but how about “the president’s fluffer”?

  • lawrencevillecco

    Yes, apocryphal, but attributed thus….

  • schindy

    Excellent Mark.  Your rumination asks us all to reflect on what has become casual language and questions whether trite terms really serve our students well.

  • 11182967

    This article is a good statement of a very important point.  The concept of the “right fit” reflects the world of Match.com, Christian Mingle, and their ilk (not to mention “Bachelor” and the “Bachelorette”–can “The Widow,” “The Widower,” and “The Gay Divorce[e] be far behind?). 

    What is fundamentally perncious about the concept of the “right fit” are the twin presumptions that 18-year-olds are already so fully formed that they can wisely choose their associations and so finally formed that these associations–of interests, of persons, of ideas–cannot, will not, should not be challenged and changed.  What better place to receive an education than a college which is in many ways not a good fit?–what challenge is there (other than that of boredom) in a place where everyone is the same?  

    When I went to college I assumed that I would be a different person when I graduated and that this would be a good thing (and I was and it was).  Indeed, that was why one went to college.  College was part of one’s “formative” years, a time of formation and frequent re-formation (we were serially Socratic, Platonic, Aristotelian, Augustinan, and so on, until we started to become ourselves), not conformation and affirmation.

    For the individual and for society it is most fitting that college be not a fit, but fitful–and give us fits.  

  • tippens

    Well said! This essay demonstrates that “fit” can be easily be code for “whatever pleases my palate.” Sadly, this plainly consumerist approach to education may eliminate the exquisite pleasure of surprise, the encounter of the unknown. Yvon Chouinard, the environmentalist founder of Patagonia, said, “Adventure is the uncertainty of outcome.” Here’s to adventure.

  • collegeexplorations

    Brilliant! 

  • greatcollegeadvice

    Very well done, Mark. 

    Amadis, Beatrice, the Holy Grail.  Somehow kids–and their well-meaning parents–have romanticized university life to the point that we have to remind them that university life is mostly just “life.” 

    I force a lot of my students to read Robert Frost’s poem, “The Road Not Taken.”  We talk about how one cannot know what is down which path.  We talk about how life is full of choices, and many choices have to be made in conditions of relative uncertainty.  And that what makes “all the difference” is how one takes advantage of whichever path one choose.

    When I talk to parents, I generally compare the selection of a college to a choosing a spouse:  is there only one person to whom you could be happily be married?  Is there one ideal “fit” for you?  After 20 or so years of marriage (or divorce), do you still hold on to romantic notions of the ideal spouse?  Not surprisingly, most parents will shut up about “fit” at that point and talk about more important things, like “general compatibility.” 

    Thanks for demolishing the ideal of “fit.”

  • _perplexed_

    Well then put it to the test:  Evaluate each application, have an admissions officer rate each application on a 1-7 scale where 1=my gut says this applicant will do very poorly if admitted and 7= my gut says this student will excel if admitted, and then see whether this gut feeling has incremental validity when grades and test scores are in the mix– if so, then by all means, use that gut rating!

  • reineke

    Although speaking as a Girard scholar, I am thrilled to see mimetic
    theory applied in such a cogent and insightful way, I would not dismiss the
    concept of “fit” out of hand.  Key to
    determining a solid fit is in getting inside the college experience rather than
    looking at it from the outside.  Tour guides and
    admissions offices  tend to focus on “fit”
    as described in this article.  But if a
    visiting student (or the parent in tow) can cut through a student guide’s
    canned speech to ask about the guide’s current classes and interactions with
    professors and students in those classes, enormously revealing insights are
    often forthcoming.  So also can visits to
    at least two classes during the campus visit be revelatory.  Student guides’ attitudes about their classes
    were THE deciding factor in my own daughter’s quest for “fit.”  At one college, the tour guide’s passion for
    learning (confirmed in her classroom visits as she observed other engaged
    students) matched my daughter’s own desire for learning.  Was it mimetic?  Yes, but in embracing the inner scholar in
    herself, my daughter drew on what should count most in making a decision for
    college.  Unfortunately, admissions
    offices are often complicit in advocating fit for the wrong reasons.  At my own institution, the academic experiences
    of students are all but missing from the campus visit.   An admissions office film featuring one of
    my gifted students—a model for the kinds of students faculty wish dominated our
    campus—shows my student talking about ice cream in the dining center, not about
    how his classroom experiences and co-curricular leadership activities have
    supported his extraordinary academic achievements.

  • josephofoley

    A tiny point and perhaps an irrelevant one — in the Tobias Smollett translation of Don Quixote, Dulcinea does exist.  She was the “hale, buxom country wench, called Aldonza Lorenco, who lived in the neighborhood, and with whom he had formerly been in love;…”

  • markcmoody

    Thank you for the very kind comments. It is a nerve-wracking thing for a civilian like me to throw around scholarly concepts in the Chronicle!  All the points made in these comments are valid– we tease some of this out to a greater degree in our spoken presentation.  There are many directions to go from here.  Reineke, it is exactly that distinction between “inside” and “outside” that we try to explore– the challenge in our work with kids is to guide them to an understanding of the things that will shape their lived “inside” experience in college. I’m still working on strategies to accomplish that!  What I can say is that I think that the work of admission and guidance counselors should be rooted in an awareness of adolescent development, and that it should seek to encourage personal reflection and a growth in self-knowledge– even if it takes some discomfort to get there. The college decision process is for many students a significant rite of passage to maturity and a chance to gain some resiliency and a tolerance for healthy uncertainty and risk.  As adults around them, we can very effectively derail that or we can work to support a process that allows for personal growth.  Josephofoley– fair point. The most correct thing to say is that we never encounter Dulcinea as readers.  Thanks again for reading carefully and commenting on this!

  • http://twitter.com/UDL_Universe UDL_universe

    What may be most interesting is evolving possibilities toward assistive technology!

  • katisumas

    That is so interesting!

  • http://www.linkedin.com/in/cherissegardner Cherisse Gardner

    I can imagine a day when this is realized as an interpretive device providing a way for  non-signers to interact with the deaf.