• Tuesday, May 29, 2012

Previous

Next

‘Embedded Librarian’ on Twitter Served as Information Concierge for Class

February 25, 2011, 2:08 pm

What if a reference librarian was assigned to a college course, to be on hand to suggest books, online links, or other resources based on class discussion? A media-studies course at Baylor University tried the idea last semester, with an “embedded librarian” following the class discussion via Twitter.

At the start of each class session, the professor, Gardner Campbell, asked the 11 students to open their laptops, fire up Twitter, and say hello to their librarian, who was following the discussion from her office. During the hourlong class, the librarian, Ellen Hampton Filgo, would do what she refers to as “library jazz,” looking at the questions and comments posed by students, responding with suggestions of links or books, and anticipating what else might be helpful that students might not have known to ask.

“I could see the sort of germination of an idea, and what they wanted to talk about,” she said, noting that it let her in on the process of students’ research far sooner than usual. “That was cool for me,” she added. “When I work with students at the reference desk, usually they’re already at a certain midpoint of their research.”

When the class was discussing the work of the science-fiction author Clifford D. Simak, for instance, she tweeted a link to his archives at the University of Minnesota.

“One of the students said, ‘Hey, is there anything like that for Rilke?’,” Ms. Filgo said. “He was all excited. I don’t even think he knew of the idea that a library might collect an author’s papers.”

Mr. Campbell, who just left Baylor to take a job as a professor and e-learning administrator at Virginia Tech, said one moment, in particular, made the experiment worthwhile. The students were discussing a rare book by Theodor Holm Nelson, a sociologist who coined the term “hypertext.” The book, Computer Lib, is really two books in one, with an unconventional layout that tries to simulate linking among segments and marginal comments that Mr. Nelson said would come as text was increasingly stored on computers.

Ms. Filgo tweeted that she had something for the class. Then she grabbed a copy of Computer Lib from the library’s shelves and walked over to the classroom. She had never actually met the students in person, so they were surprised when she appeared with a copy of the book to pass around, as they were still discussing it.

“There was apparent magic to it,” Mr. Campbell said. “It made that class unforgettable.”

Ms. Filgo said she would try it again, but she worries that it would be difficult to expand the effort to a wider number of classes. “It took out three hours of my workweek,” she says. “The question is how can you scale this up?”

This entry was posted in Uncategorized. Bookmark the permalink.

  • Print
  • Comment
  • andymorton72

    Leave it to Gardner! I’m glad he’s coming back to Virginia.

  • http://www.goannatree.com/ Goannatree

    Having worked with Ellen and Gardner at Baylor this comes as no suprise: they are both innovative, and seek to use every opportunity with students to show how technology can be integral to a course and to real learning, not just a tack-on to be trendy. I wish more libraries and departments understood how important librarians who are eager to be involved in the ongoing learning experience of the students actually are! Research and Reference Librarians are fundamental to my ability to produce innovative and original scholarship!

  • http://www.academicroom.com Online Courses

    A great complement would be the ability for the class to share digital content (articles, books, videos, bibliographies) with other within the discipline or with others taking a similar course in other universities. Academic Room may make that possible one day: http://www.academicroom.com.

  • charliegyrl

    I LIKE THIS IDEA

  • library_yeti

    No disrespect to the librarian or the faculty member involved in this innovative, collaborative effort, but the use of the moniker “information concierge” reflects a fundamental problem in academia: that librarians are viewed as being on-hand to fetch information for students and simply hand it to them the way a concierge would fork over a toothbrush to a hotel guest who lost their luggage. Instead, the role of the academic librarian should be perceived (and promoted) as instructive. As an academic librarian my role, and that of my colleagues, is to teach students to navigate, evaluate, and use information effectively and ethically on their own. Seems like that would be difficult to accomplish while attempting to help an entire class at once in real-time via Twitter. But perhaps Ms. Filgo is a fast typist.

    Though I have served as an “embedded librarian” for many years and recognize the value of such a service when appropriate, I am left wondering: if this class meets on campus, why didn’t the librarian simply conduct a library instruction session for the group in-person or online, then follow up with students individually on an as-needed basis using Twitter, Facebook, F2F appointment, email, chat, etc.? That sounds like a greater recipe for success (in terms of conveying the knowledge and skills to research successfully on their own).

  • mbelvadi

    This is an interesting experiment, but the last line is the most important, because it pretty much answers itself – you can’t scale this up. If that librarian makes an average academic librarian’s salary, the cost of her 3 hours is far more than what the university would pay an adunct for that class time. This is a really big problem with educational experiments – I hear all the time about exciting, innovative projects either at the K-12 or higher ed level that are run by one dynamic teacher/librarian with one class for a single semester where everyone involved say that the project was lots of fun and maybe even pedagogically successful, but it’s obvious from the outset that it can’t lead anywhere because it involved an extraordinary amount of resources that could never be invested per class on any widespread basis. Maybe some tiny grain of such experiments can be used in a scalable fashion to improve education. I’d be interested to know what the other librarians at Baylor think of this experiment, and if they’re developing plans to scale any part of it.

  • dolllar

    A main goal of this is to break down the invisible barriers between students and reference librarians. It may not “scale up,” but as an occasional course infusion for selected classes I think it’s a great idea.

  • lenanozizwe

    I think this is an innovative way to use Twitter. I’ve used it in classes to communicate with an expert that was not able to come to campus. I must say that I never had such excited students as the day the class became a trending topic in our area.

  • emilyhorning

    Of course you couldn’t scale up, but even if you could you wouldn’t want to; there are many classes for which an embedded librarian wouldn’t be helpful or necessary. Better to target courses like this one that make heavy use of library collections and resources in assignments and discussion. Then follow up with individual students on their research topics. Yale’s Personal Librarian Program (http://www.library.yale.edu/pl/) is designed to work this way. We’ve found the PL model to be more effective than offering a library instruction session (because students’ topics are too diverse or imperfectly formed, they learn little in a general session) and a more efficient use of librarians’ time.

  • http://twitter.com/lizmclean Elizabeth F McLean,

    Kudos to the instructor who 1)championed the embedded role of the librarian 2) and got some JIT interaction btw teachable moments and information sources!

  • http://twitter.com/oreinauer oreinauer

    I’m assuming that the students communicated about the course content outside of class via Twitter as well? If so, one way to scale up (while also scaling back?) would be to add the librarian to the Twitter discussion without the real-time in-class component. In that way the librarian could communicate at his or her convenience with the group or with individual students, and could potentially juggle more than one class at a time. Embedding a librarian in a course blog or discussion board is a similar option. I love this example though – very innovative. Keep up the good work!

  • http://twitter.com/lawmiller Steven R. Miller

    Information specialists using social media outside the classroom to reach today’s students and support the curriculum can offer additional pedagogical value.

  • http://twitter.com/lawmiller Steven R. Miller

    Information specialists using social media outside the classroom to reach today’s students and support the curriculum can add much pedagogical value.

  • http://twitter.com/nashworld Sean Nash

    This IS essentially the situation I have had in the public high in which I teach. This interaction has been archived here: http://fromdayone.wikispaces.com/ You wouldn’t believe the impact this has had on those students involved…

  • rweaver

    This just popped up in my gmail and already there are comments from 3 days ago!?!? My IM and email need geritol. But not this subject. I have forwarded it on to other Reference and Emerging Technologies librarins. They should get it befoe week’s end.

  • http://www.writessay.com Essay Writer

    Good to hear such idea. Hope they will push through with it to have better access with librarians.

  • http://twitter.com/Philboyd_Studge Craig Cameron

    Utilizing the expertise of those individuals w/library science degrees is of great assistance to students navigating through Higher Ed.

  • http://www.TerriGriffith.com/blog Terri Griffith

    Imagine that kind of process in a business meeting or brainstorming session. Happens on an ad hoc basis now, but as a role you could build on the idea given you were expecting the help.

  • http://twitter.com/LifebrarianN NanC

    Instead of worrying, Ellen, I suggest your just helping similarly when you can. Providing too much can promote complacency. Word gets around about exciting, sporadic happenings. Work well done will benefit someone somewhere.

    You could reveal a bit about how you were able to do this…perhaps having one student (for assignment credit or such) shadow/observe your preparations/work behind the scenes to get a bit of a sense of how and where resources could be found and that librarian expertise can be crucial. Preparation prior to this kind of class could be done by students to reveal what they need/want to learn.

    Some other students might well benefit from your extending this twitter help a few steps further. How about trying that same format at the reference desk? You could try this twitter reference class in place of a library orientation class; in the next class the students could extemporaneously reveal their reactions and explain how it might have been possible (with your gentle expertise as a guide when asked.) Expect that some might find reference you might not have considered relevant/important! They can tell you what more info they feel they need re the library resources and services which can engage their learning for a subsequent class either in their own course or another on campus in which they might ‘sit in’ for this.

    Thanks Jeff, for passing this along. Kudos to Ellen and Gardner for engaging learners early so they can choose their paths to knowledge. Wait too long and they can become students who just want to ‘get an assignment done.’

    twitter = tremendous possibilities

  • davidshumaker

    One way this scales is to get reference librarians out from behind the reference desk entirely. Others can provide the basic directional info and tech support that make up most of the traffic there, and by establishing relationships with students via classroom presence and co-teaching, the embedded model will move even more of the deep and complex questions into a consulting environment, where they belong anyway.

    That said, it’s also true that the embedded librarians don’t need to be in every class. Pick your spots, and preferably get in front of students early. We have at least anecdotal evidence that information literacy skills taught early can persist through the student’s undergrad career. (Beyond, too, we hope!)

  • ivanacg

    As a secondary school/public librarian I vouch that “early” Information Literacy is never early enough! Elements of it should be introduced when a teacher asks a student to gather information, which can be as early as 4th grade, and students are generally sent directly to the Internet to gather information. More apt teachers do give some guidance and links for this kind of exploration. Some awareness of online sources available through public libraries (e.g Britannica Junior Online) might exist but the access to them is indeed cumbersome.

    It is also worrisome how many teachers, even at secondary school level in some subjects, do not press students to note the sources of their information… it does not have to be an elaborate format, just plain “who, where, when” kind of acknowledgments so students are saved from “inadvertent plagiarism”. As a token towards showing students how to avoid plagiarism, in high school the librarian gets asked to present in one session “how to cite”. I point out that MLA, APA, etc are ways of formatting, so it is showing students how to format, not cite. Generally students have a difficult time to learn overnight *how* to cite, which is an elaborate skill, among the higher order skills, the last in line — (i) how to evaluate, (ii) paraphrase… (iii) cite.

    I don’t think I could teach a student how to ask for directions in a foreign language in one session and yet we expect students to cover the basics of information literacy in sporadic sessions. I agree that at university level an embedded librarian does not need to be in every class. That’s because much of the information literacy should have been “embedded” already, as early as possible.

  • carabadang

    I can confirm Olivia Reinauer’s comment. Being “embedded” as the consulting librarian in a senior history seminar as part of the Blackboard discussion allows me to drop into this forum regularly without carving out the 4 hours per week that the seminar is actually meeting face to face. Not ideal – real time does add verve and excitement (loved the book delivery moment in the article – ta-dah!). But students float ideas early in the online discussion of class readings and I can get in on the ground floor of their research process.

  • http://www.facebook.com/sebastian.panakal Sebastian Panakal

    Jeff,

    Thank you for sharing. I am uploading this on my blog http://rajettan-eteaching.blogspot.com so that teachers whom I teach and the school/college librarians who visit my site could make use of your creative idea.

  • http://twitter.com/aviana Ann Randall

    Thought-provoking idea.

  • bethelcollege

    Okay, enough teasing.  What’s the tool?  Myers-Briggs? StrengthsFinder? DISQ? 

    Inquiring minds want to know!

  • cwinton

    Keep in mind your statement ” I did my best to be honest, to depict myself as I believe I am, rather than as I wished I were.”  Under that circumstance I’m not surprised at the test’s accuracy. If you should make it a high stakes element, applicants will simply mine sources to find out what response will produce the most favorable profile and answer the test questions accordingly.

  • academicentrepreneur

    Let me get this straight:

    The test is unspecified, as are the data for its validity and reliability. We don’t even know if it’s being used appropriately, even if it is valid and reliable.

    It’s being administered in a highly stressful situation–a job application–in which candidates have an incentive to be circumspect in their responses (to put it politely).

    The author (an N of 1) took it under very different, much less-stressful circumstances and is generalizing from that experience.

    The example of a descriptor (“unable to complete routine tasks”) is what’s often called a Barnum-like statement because it can be interpreted so broadly. Without knowing anything about the author, I can generate some very likely additional hits with statements like, “Others don’t always appreciate your insights and intelligence.” or “There are times when you’re more emotional than usual.”

    My conclusion is that the well-intended author and the well-intended HR department could probably benefit from a good course in psychometrics.

  • keithtravels

    As a deep skeptic, to my surprise I too found personality test results both accurate and revealing (in this case the Keirsey-Bates and an Enneagram).  What I gained was the understanding that my personality style is rare, and that it helped explain a lot of the challenges I was facing as department chair.  To some extent it allowed me to modify my style in order to handle challenges in a more effective manner.  After fearing the tests, I have come to consider them a useful tool for personal understanding IF (as noted below) the participant answers honestly and if the results are exclusive to the test-taker.  As a job candidate, I would try to game the test once I saw patterns emerging in the questions.  A reasonably bright person can detect the better associations and answer accordingly.

  • david_evans

    Bethel, the Chronicle editor and I agreed not to name the test.  If they give me permission, I will gladly do so.

    Academic_entrepreneur, there is a very considerable body of validation for the test we use.  Since I’m not disclosing the vendor, I’m afraid you’ll have to take my word for it, but I did some looking at their literature (including peer-reviewed scholarship, not just self-dealing studies), and it looks solid to me. 

    Indeed it’s true that I took it under different circumstances from those encountered by a job applicant, but since I’m not applying for my job right now, there’s nothing I can do to fix that scenario. 

    Finally, the nature of the test is such that “gaming” it, and “circumspection,” are not terribly likely, I think.  This is not a test made up by “Joe’s Testing, Inc., LLC,” but a high-level test used by a large number of very serious organizations, businesses, and institutions.

  • schultzjc

    I ignored a personality inventory test 45 years ago, and had a pretty good career, but doing the “wrong” thing in terms of skills and satisfaction.  Once I found my current position (at age 60) I instantly recalled what that test had said and realized what could have been had I taken it seriously.

    That said, I would find it offensive to be handed such a test as part of a job interview. That need speaks volumes about the inability of faculty, staff and administrators to evaluate candidates.

  • johnnirenberg

    How would you feel if your current job was dependent on that score?

  • maggiecase

    The truly scary part in all this is who interprets the results in light of the position?
    So is your “inability…” a liaibility or does it make you a creative genius? Guess we need another test for that.

  • tgroleau

    As a college freshman (roughly 30 years ago), I took some sort of career/interest test at our career center.  It wasn’t a “personality test”.  Instead it matched your answers to the answers of successful people in a variety of fields and gave your a list of “people who provided answers like yours are working as ….”  

    I was too polite to laugh out loud at the career counselor when the list showed my closest matches were Military Officer and College Professor.  I had no interest in the military and I certainly didn’t want to spend 10 years in college to become a professor (I didn’t want to be ANY kind of teacher).  Clearly, the test was junk.

    By coincidence, I had enrolled in an introductory ROTC course because a) it was free and b) it filled out my schedule nicely.  To my surprise I ended up seriously considering the ROTC program.  However, I didn’t sign up.  Based on my interviews and research, I really didn’t think that I fit in that world.  I could still laugh at the career test.

    I graduated and worked for four years before returning to school for a masters degree with every intention of going back into industry.  Instead I stayed in school, got my PhD and became a college professor (with exactly 10 total years in college).  I guess I can’t laugh at the test anymore.

    Data is not the plural of anecdote but I’ve heard enough success stories about these various tests to trust that they have some level of statistical validity.  Like any statistical tools, they’re far from perfect, but they can be useful.

    Side note:  I’m still a bit surprised that Military Officers and College Professors gave generally matching answers to this test.  Does anyone else find that an odd match?

  • david_evans

    Given that the results matched my work very closely, I’d have no problem whatever with it.  My job is dependent on my performance, which, as I have said, lines up quite closely with the test, so I’m fine with it.

  • david_evans

    1.  Our HR director and several other staff have training in interpreting the results.  Moreover, like ANY other aspect of the hiring process, it’s just one small component in an overall picture.

    2.  Why, of course, I AM a creative genius.  Why do you ask?

  • cwm4c

    Not at all.  We have several professors on campus who are former military officers.  There is a strong desire to serve others in both professions–maybe the test caught that trait.

  • maggiecase

    Well I think it depends on your teaching philisophy TG; command and control or facilitate. Also; couldn’t it be the power of suggestion at work in your case? Unsure what to do( as most of us are at at least some point), ah teaching might fit the bill. I do not means this in any way as a failing; subconscious shaping perhaps.

  • maggiecase

    1) one small but decisive component based upon your tiebreaker anecdote. But it gives the veneer of objectivity
    2) don’t have as much faith in HR training I guess. Recently read a published article in which a consultant claimed leaders have different brains than the rest of us and think differently; inability to complete routine tasks was mentioned but interestingly no citations ( although the paper was academic)

  • dpc61820

    “My sample size is small…”

    No. You don’t actually have a sample at all. You have an anecdote.

    Please correct to “My sample size is n/a…”

  • eng101

    When I was in my early 20s, just starting grad school, I took the Myers-Briggs and learned that my type (INFP) is pretty rare in general society but not so rare among humanities professors in higher education. Of course, learning that “humanities professor” fit me wasn’t a surprise, but it was nice to see that the somewhat unnerving “mirror-image” description the MBTI offered of my personality also said my type gravitates towards higher education and the humanities. I was in the right field for me!

    I ended up leaving grad school two semesters into my PhD in English (due to a mix of burnout, lack of finances, and certain family/personal issues sucking up my energy at the time) and got a corporate job (technical writing) that paid well and had great benefits.

    Fast-forward 20 years of an occasionally satisfying but mostly frustrating tech-writing career, and I’ve gotten out of the business to be an adjunct English instructor making less than half the pay of tech writing, with no benefits of course. I recently read an INFP type description in a book borrowed from a friend. I just shook my head. It’s creepy how, after all those years, I went back to my type’s preferred career choice like some sort of homing pigeon.

  • jlrudmann

    The confirmation bias is a well know cognitive error. It’s the tendency to focus on confirming “evidence” while ignoring non-confirming information. This particular cognitive error seems to be in play here.
    As a psychologist experienced in test development, I doubt the ability of any personality “test” to provide meaningful information about potential job candidates. Extensive research by the Navy has confirmed that the Myers-Briggs Typology Inventory is not valid. The Enneagram test! Give me a break – super psuedo-science.

  • schultzjc

    I don’t think that’s appropriate.  My job should depend on performance, irrespective of the personality “fit”.  In fact, I had a very successful research career with a not-so-good fit. Were I struggling in my currrent job, I’d welcome any attempt to understand why aimed at improving performance.

  • muntzp

    Not sure why you wouldn’t want to name the test . . . would be interested to hear the reasoning.  I’m with bethelcollege in being very interested to know which test your school uses, regardless of whether you personally found it worthwhile.  I’ve understood that some assessments, such as StrengthsFinder, are not supposed to be used in hiring decisions.   

  • cwalck

    These instruments are great for personal development and thinking about career paths, but it is unethical to use them for evaluation/selection purposes.  They reflect preferences which may lead to certain behaviors, but are not a measure of capability or performance.  This is even more true if you are using an abbreviated version of the instrument.  I hate to think you may have used it make a hiring decision!

  • tgroleau

    The power of suggestion is always a possibility.  

    I was enrolled in the ROTC course before I took the test.  I was two hours short of a full load and the course was two hours with free tuition.  That was the only reason I took it but maybe the test made me look at the class differently.

    As for teaching, I was well into my masters degree (a terminal masters in applied math with a good placement record) before I began thinking about moving into a doctoral program and pursuing teaching. This was about 8 years after taking the test but it could still have been in the back of my mind and influencing my impressions of and interactions with graduate school faculty.

  • art_of_nurture

    When I was an undergraduate, some of my cohorts reported giving answers to such questions that they thought would give them preference in the eyes of those who were to interpret the tests. 

  • david_evans

    Everyone, thanks for your comments.  I knew this post would be controversial, and I am finding the discussion helpful. 

    Several comments:

    One of the sources of my skepticism about this instrument was that, like tgroleau, when I was in high school (about 32 years ago now), I took one of those career inventory tests they gave at the advising center, and despite my strong insistence that I was not interested in a job that involved a lot of math, my recommended career was “Accountant” (though in self-fulfilling prophecy mode, I now manage a budget that’s way up in the eight figures, so I guess they were on to something all those years ago). 

    On confirmation bias, well, sure.  However, the results I got fit me like a custom-tailored suit, in basically every respect, positive and negative.  (And no, it didn’t say I was a “creative genius,” though it did say I am “impatient for results,” which is certainly true.)  This was pretty fascinating to me.  As a tool of self-understanding, it was certainly useful.  I doubt I’ll ever be good at routine tasks, but it was helpful to see that characteristic so clearly.

    On gaming the test/giving answers that make you “look good,” again I think that the test doesn’t lend itself to that. 

    As it turns out, we didn’t use it to make our hiring decision, though it was interesting to discuss the results in light of the large amount of feedback we’d received from the campus community, and it perhaps shaped our discussion of the candidates.  However, it was certainly not definitive.

    Regarding not naming the instrument, I presume that the Chronicle doesn’t want either to provide free advertising or, conversely, to expose the vendor to opprobrium, either one.  I don’t know whether our contract with them requires confidentiality, but it probably doesn’t except with regard to individual results, which is, or course, an ethical imperative.

    Something that interests me about many of these comments is the implicit notion that use of such an instrument somehow contaminates what was previously a Platonically pure interview process.  Given the overwhelming literature from a variety of disciplines (psychology, sociology, management, cognitive science, etc.) that indicates the unreliability of interviews to determine potential job performance, it’s hard to imagine that use of a normed, validated instrument is the hiring equivalent of the serpent sneaking into the Edenic realm of the job interview. 

    Like everything else about hiring, in the academic world or elsewhere, the “human factor” (subjectivity, biases, snobbery, personality, etc.) is such a huge part of the interview process that it’s difficult to believe that what is on some level a more “objective” component of evaluating candidates is “unethical” or “troubling” in any compelling way, especially when that instrument is employed with the same skepticism and critical approach with which one should also treat letters of application, references, and other aspects of the candidates’ presentations. 

    Applied mechanistically, on the other hand, this would be a disaster.  But so would hiring people strictly on, say, the prestige of their credentials. 

  • manoflamancha

    No, because they both like regimented life styles, and captive audiences.

  • kweber

    To be fair, there were at least a couple of other examples he used for comparison–his president and the others on campus who “have noted that their test results haven’t been quite as accurate.”

  • freya1

    This is a
    controversial idea indeed! It can almost relate to the standardized tests we
    expect students to take and be issued a “number” or a grade and that is
    supposed to be an accurate reflection of how well they are learning and what
    they are learning, and it often found not to be accurate at all. I find
    whenever I take a personality test similar to this one, I find it difficult
    because all too often it depends on the different situations I am in. I make
    different decisions based on the people involved, the institution or situation
    involved, and often times the location. I find I make decisions based on
    several factors, but most common is the culture of where I’m at, the people
    involved, and honestly how I’m feeling that day or at that time. My personality
    and habits can change frequently as well. Also when taking tests for an
    employment opportunity (especially when jobs are difficult to come across
    currently), quite often you want to make yourself seem appealing to that
    institution so that may change your answers and responses as well. It is
    interesting that yours came out accurately, so perhaps this particular test
    found a way to overcome or avoid the common nuances such as changing your
    answers to fit the need of the employer or differences of opinion, however I
    remain skeptical of a test to determine an accurate personality that only takes
    a few minutes and is computer generated.

  • http://www.facebook.com/mark.federman Mark Federman

    A massive problem with these so-called personality tests is that they are highly culturally biased in their assessment. People who have been raised and socialized in a North American culture will tend to conform to the norms established (and therefore the conclusions reached) by the assessment algorithms attuned to the North American context. Those who were raised and socialized in what we might call non-Western (or even European as opposed to North American) cultures tend to have results skewed in some non-normative (relative to N.A.) direction.

    As context for my opinion, I am trained and certified in one of these methods (and have been exposed to many others), and this is a recognized limitation. The method in which I have been trained is specifically not a personality test, but has a focus on guiding specific interactions among people of differing “types.” It’s useful for tailoring interactions (like delegation, coaching, correction, praise, motivation etc.) so that they can be “heard” more effectively by the recipient. Even in this relatively low-stakes application, there is a big caveat for cultural norms when dealing with people from other-than-North-American origins. Even the test results are normalized by culture for the locale in which the test is administered (which admittedly doesn’t help much when a person of Asian, African, Indian, etc. origin takes the test and is scored in North America).

    Part of the other problem with such tests is that people tend to “wear” the test results – internalizing the often horoscope-like characterizations to justify their habits and behaviours that too often prevent them from learning, growing, and developing. Anecdotally, (i.e., N=1, but I do constructivist-paradigm research, so that’s valid! :) a colleague and good friend has been told she’s an introvert according to MBTI so often that she often uses it as an excuse to avoid non-introvert opportunities. On the various occasions in which I’ve encouraged her to take the opportunity, she’s done splendidly well and enjoyed herself without the anxiety that should otherwise compromise a supposedly life-long introvert. Indeed, her success at shucking off the introvert label in these occasions has directly led to her professional success. She still claims she is an introvert, and has the MBTI assessments to “prove” it.

    Personally, I think that such tests, assessments, inventories, and surveys are pseudo-science at best (yes, quantitative, statistics-based research can prove pretty much anything you manipulate it to prove), and individually hampering and destructive at worst.

  • landrumkelly

    The future of H.R. is the future of higher education. 

    Oh, hell. . . .

  • cliffhurst

    This is a fascinating string of comments. As I join the discussion, you should know that I am a consultant who has been using a variety of psychometric instruments as aids in helping my clients to make better hiring decisions, promotion decisions, and employee development plans for more than 20 years. I am nearing completion of my PhD now, in late career. I have used a axiometric assessment tool that measures a person’s value structures as part of mydissertation research in the area of entrepreneurial judgment.

    Instruments of this sort can bring insight into the hiring process that interviews, reference checks, background checks, statements of teaching philosophy, and record of publications cannot disclose. Ethical use of these instruments is never used as a replacement for these other steps in selection; it is used to augment them.Nowhere is social desirability bias more evident than in the ritual known as the interview process.HR departments know this. That’s why good ones today augment interviews with assessments.

    I find it interesting that many of the comments made by others presumably have come from faculty members at schools who rely on other proxies of performance,  including SATs, ACTs, GPAs and written essays to select the students whom they teach. These measures of potential academic performance also have their shortcomings, but this does not mean that we ought not to use them.

    I work with a number of instruments that are exceedingly well validated to ensure their conformity with EEOC guidelines and sound management practices. They have to be, or consultants using them–and their clients–would soon be out of business. There are many others–including scales and surveys developed by academics to prove some sort of pet model they have come up with, that seriously lack validation.

    When it comes to personality tests, it is my own opinion that, for many positions, it is not who a person is that matters as much as how well that person knows himself or herself that makes the difference. Psychometric and axiometric instruments can be a powerful tool for enhancing self-awareness. Try them open-mindedly, as the author of this post did.

  • siopadministrator

    In this article, David Evans seems surprised that personality tests can accurately reveal a great deal of useful information about candidates being considered for hiring or promotion. He shouldn’t be.

    There is, and has been for many years, a highly respected academic discipline devoted to the development, validation and use of employments tests. It is derived from the social sciences and is a major area within the field industrial and organizational (I-O) psychology. Indeed I-O psychologists provide a valuable contribution to the HR profession in creating
    and interpreting the kind of assessments Dr. Evans experienced.

    The headline referring to testing as possibly being pseudo-science is misleading. I-O psychologists employ strict and rigorous scientific methods to testing and assessments that are predictive of job performance. These evidence-based assessments have been proven to be valid and relied upon by HR professionals in helping them make personnel decisions.

    Most I-O psychologists, many of whom serve on university and college faculties, are members of The Society for Industrial and Organizational Psychology (SIOP), which has been a division of the American Psychological Association in some form for more than 50 years. I-O psychologists study and apply scientific principles concerning a wide range of areas related to the workplace, including productivity, motivation, performance and personnel
    selection.

    Though Dr. Evans may have been ambivalent about testing initially, it was good to see that he concluded his article by acknowledging that test results “cannot be lightly dismissed as we evaluate candidates for key managerial positions.” Tests are indeed useful tools—a fact that I-O psychologists, as highly trained experts in the robust and established discipline of employment testing, have long recognized.

    Adrienne Colella
    President, Society for I-O Psychology (SIOP)

  • kristinac

    To clarify, when psychologists talk about “personality,” they typically are referring to general behavioral, attitudinal, and cognitive trends that we see across time.  So even though in some situations you act differently, this does not mean that your personality changes.  Take, for example, someone who is an extravert.  Certainly, even the most extraverted of people may have days when they feel a little less sociable, or situations where they may be hesitant to start conversations.  However, their personality suggests that, across situations, they will be more likely to talk, interact, and initiate conversations than someone who is more introverted.

  • kristinac

    While I absolutely with your first point about cultural biases, I have to disagree with your second point about people “wearing” their results for a number of reasons.  First, the Introvert-Extravert labels for the MBTI are slightly different than how they are defined in the Big 5.  Rather than being about sociability and those sorts of things, the MBTI extraversion-introversion dichotomy is more about where people draw their energy.  Introverts need time alone to recharge, while extraverts feel invigorated by being around others.  It seems like a small difference, but with your example, your friend could certainly be engaging and charming in social situations, but perhaps her avoidance of so-called “non-introvert” opportunities is simply her taking the time she needs to recover from that social time.

    Additionally, if you believe that the great wealth of literature and empirical support about personality testing is complete hooey, I don’t understand why you would be on this forum — much of higher ed is based on empirical research, and distrusting all of it without considering individual studies or lines of arguments is throwing out the baby with the bathwater.  Calling personality testing as a whole “pseudoscience” is, I think, an absurd blanket statement.  Some personality tests are terrible and do appear to be pseudoscience (e.g. phrenology, graphology, and tests you find in Cosmo about what flavor of cake you are or whatever) some are OK but demonstrate psychometric problems (such as the MBTI you’ve cited), and some have a great deal of empirical support that they do measure something that differentiates people, and can tell us important things about what those people like, what jobs might suit them well, and what outcomes we might expect for them (measures of the Big 5 might be an example of this).

  • antiutopia

    First, I have to express appreciation for any administrator who is able to admit to prejudicial thinking based on ignorance and then go out and check.  That’s someone worth working with.  

    I’ve had pretty good experiences with personality tests such as the Myers-Briggs and others, but my Chinese horoscope is fairly accurate too.  My astrological sign is also a good descriptor of me, and I think I can find my attributes well-represented in Book II of Aristotle’s Rhetoric as well.  I think we’ve been able for quite some time to delineate a range of personality types by observable habits and attitudes.  They are generalizations, however, so it’s also easy to find ways to fit a generalization of any kind, whether a personality type, astrological sign, or personality profile.  I’m not sure I’d be comfortable basing a hiring decision on a personality profile, though.     

  • satris

    About military officers and college professors:  I think one thing that ties the two together is that military officers have a wide range of facilities open to them on base and college professors have a wide range of facilities open to them on campus.  In addition, they are members of a larger community that recognizes their role as honored and respected.  They belong.