Category Archives: Assessment&Evaluation

September 18, 2013, 5:09 pm

Moving From Data-Driven to Design-Driven Innovation

I was tweeted into a conversation about assessment and I wanted to take a minute to comment. Elliott Shore (ARL) recently called for a radical change in library assessment—with the gist being a move from descriptive to predictive. I’d like to push it further into the realm of innovation-generation.

I tried to contribute my part to that conversation at last year’s Library Assessment Conference with a paper Too Much Assessment, Not Enough Innovation. I wasn’t booed off the stage, but I definitely felt avant-garde compared to the mainstream assessment crowd. But of all the papers I’ve written recently that’s my favorite one because I enjoyed digging deeply into places like PARC and Bell Labs.

Anyway, twitter isn’t ideal for long-thought sharing so I’m dashing this off over lunch. I’ve been reading the October 2012 issue of Fast Company (yes, I’m a bit behind) and…

Read More

January 14, 2013, 3:17 pm

FOCUSING YOUR FOCUS GROUPS: Ten Ways They Can Enhance Discovery

I saw that there was a session at midwinter talking about focus groups. Since I won’t be there I wanted to take a few minutes and share my thoughts. I don’t have the original announcement, but I was disappointed with the phrasing. It asked something like are focus groups effective? I would prefer a conversation around how to use focus groups effectively.

 

I have found thematic conversations with various user (and non-user) segments to be an important component of my discovery strategy. Focus groups often get knocked because of three main things: loud people dominate the discussion, people tend to tell you what they think you want to hear, and people can’t imagine breakthrough change. Those are all legit criticisms, however, if you plan according you can neutralize those issues.

 

Students working through an…

Read More

October 22, 2012, 1:08 pm

Containers for Information or Platforms for Scholarship? R&D and the Networked Perspective

In seven days I’ll be giving a talk on R&D for academic libraries but here is the enhanced version of the conference paper. This is a follow-up (actually a sequel) to Think Like A Startup. I described the intentions of this paper last month so I’ll save us all from repetition. The key point is that assessment programs should be engines for change seeking progress not sustainment.

I reread the paper on Saturday and the thing that stood out was how much content I had to cut in order to get it into the ballpark of the conference’s word limitations.

If this paper is too long for you (or if you think assessment is boring) then at least go watch Dan Pink’s Ted Talk. The part about “functional fixedness” is critical and it highlights the potential tunnel vision we can develop preventing us from empowering the evolution of libraries.

Another key point is the need to…

Read More

September 26, 2012, 2:43 pm

Too much assessment…

If you follow me on Facebook or Twitter then you’ve heard me talk about the paper I was writing over the summer. It’s for ARL’s Assessment Conference and at one point it was over 14,000 words.

 

It was probably one of the most challenging things I’ve written because of time (3 months) and space (5,000 word max) limitations. The background reading was amazing; I skimmed 30 books and read nearly 50 articles, blog posts, and reports. I immersed myself into R&D culture. And sadly there was so much material I couldn’t use and even worse, so much material that I just didn’t have time to read.

 

I had two objectives with this paper:

 

  1. I wanted it to be a follow-up or sequel to Think Like a Startup. That paper resonated with a lot of people, so my working title was “operate like an R&D lab.” I took the section about assessment and gave it it’s own platform. I…

Read More

July 23, 2012, 3:48 pm

Data-Driven Decision-Making vs. Discovery-Driven Planning (don’t measure a butterfly using the metrics of a caterpillar)

I’ve been thinking a lot about caterpillars lately. I read the Very Hungry Caterpillar to my son every night and it always makes me think of organizations going through transformative change.

What’s fascinating to me isn’t just the physical transformation that occurs. Obviously sprouting wings and becoming more colorful is amazing, but the internal composition changes too. Their appetites change. Their digestive systems change. But what really gets me is the perception-shifting that must occur. Imagine you’re stuck crawling on the ground and slowly climbing trees, flowers and bushes then suddenly you’re able to fly–to move nimbly. Imagine the cognitive transformation that first day when life is about exploring a much wider universe.

You think ARL will be ok with me citing a children’s book? I also want to pitch Willy Wonka as the role model for R&D. That’s for another …

Read More

September 26, 2011, 5:24 pm

What It Takes To Become A Scholar: helping students scale the taxonomy

We’re working on round two of our graduate student ethnography project. I plan to release the findings in January as a collection of whitepapers. The batch will include themes such as mentoring, collaboration, criticism, and work process. But my favorite thread is the progression from student to scholar. It has been fascinating to review the self-analysis and see how a scholar is defined.

Two predominate qualities emerged: independence & creation.

Some sample quotes:

1. “As you go through grad school at least in English, you begin to develop more into sort of an independent writer and scholar…. You start thinking more in terms of I’m making a book length argument that has to be interesting vs. I’ve gotta write this short argument that I think is what my professor wants.”

2. “As an undergrad, you were presented with deadlines that were dictated by others. And they we…

Read More

May 24, 2011, 12:11 pm

ORGANIZING A DESIGN CHARRETTE: gathering a visual response for learning spaces (a packet)

We’re gearing up for a sizeable renovation and I’ve been trying to include students and faculty in on the interior design discussion. I’m planning a full post on the campaign later this summer—most likely after ALA—but in the meantime, I wanted to share this with others working on similar projects.

I’m a big fan of the design charrette activity. Having used it before, I definitely wanted to include it in my current efforts. It’s easy to conduct and students always seem to enjoy it as an outlet for creative expression. It’s also helpful to gather a visual response when talking about learning spaces.

This time around I wanted to conduct the charrette in a very public setting. I wanted everyone who entered the library to see it happening and to have the opportunity to participate—or at the very least to take s look at the renderings and other information that was…

Read More

May 2, 2011, 1:03 pm

THE VIRTUAL REALITY: Exploring graduate student use patterns of the UCSB Library

The UC institutional repository doesn’t have a space for library-generated papers, so I’ll just post it here. My SEO is decent so hopefully it will be discoverable.

THE VIRTUAL REALITY
Exploring graduate student use patterns of the UCSB Library
An ethnographic study

Prepared by Lindsay Vogt, Anthropology Graduate Student, UCSB
in collaboration with Brian Mathews, UCSB Library

Executive Summary (with an internal link to the full report)

I’ll let the report speak for itself. This is part one of a larger project, currently on hiatus, but hopefully will pick up again after ALA in New Orleans. In a nutshell, I hired an anthro grad student to help me study grad students. This paper is an internal document, but I figured since many librarians are interested in anthro research these days that it might be of some value. After our next report is done we’ll turn our work into a…

Read More

April 27, 2011, 11:53 am

Can you encourage people to recycle? Our attempt to switch behavior

This is a follow-up to my post Assessing Your Greenness: a serendipitous stroll toward sustainability.

Our campus recently conducted a waste stream audit in the library. Download UCSB_Library_Waste_Audit What it boils down to is that our patrons generate a lot of garbage.This gives you a sense of coffee cups alone:

Coffe_cups

It will be interesting to see what happens when we temporarily close our coffee operations during a long renovation. Obviously students will still bring coffee into the building, but I suspect they’ll consume less when the impulse to purchase is no longer right down the hall.

Waste_stream

 

The biggest takeaway from this experience is that students are not recycling– at least not as much as we expected. UCSB prides itself on sustainability and being eco-minded so I was anticipating higher numbers. What it boils down to is that 66% of our waste is recyclable, yet only…

Read More

September 30, 2010, 7:28 am

Applied Diplomacy: an assessment attempt to discover what people know, don’t know, and want to know

I’m experimenting with an assessment tactic. I have a pile of multicolored sticky notes in my office related to another project AND I was preparing for a meeting with our User Services Group to talk broadly & briefly about the topic of assessment AND I was reading Visual Meetings–  so basically I was looking for a way to blend all those components.

 

A theme that I keep coming back to is do we know our users? We can read about them and obviously we interact with them daily, but do we really know them? Do we really know the process that grad students experience as they prep for candidacy?  Do we really know what happens in the dorms the night before a big assignment is due? Do we really know faculty tendencies when starting a new article or when they prep to teach a brand new course?

 

I tend to be overly curiously, but what about my colleagues? What I realized is that I didn’t know …

Read More