Category Archives: Assessment&Evaluation

September 26, 2012, 2:43 pm

Too much assessment…

If you follow me on Facebook or Twitter then you’ve heard me talk about the paper I was writing over the summer. It’s for ARL’s Assessment Conference and at one point it was over 14,000 words.

 

It was probably one of the most challenging things I’ve written because of time (3 months) and space (5,000 word max) limitations. The background reading was amazing; I skimmed 30 books and read nearly 50 articles, blog posts, and reports. I immersed myself into R&D culture. And sadly there was so much material I couldn’t use and even worse, so much material that I just didn’t have time to read.

 

I had two objectives with this paper:

 

  1. I wanted it to be a follow-up or sequel to Think Like a Startup. That paper resonated with a lot of people, so my working title was “operate like an R&D lab.” I took the section about assessment and gave it it’s own platform. I…

Read More

July 23, 2012, 3:48 pm

Data-Driven Decision-Making vs. Discovery-Driven Planning (don’t measure a butterfly using the metrics of a caterpillar)

I’ve been thinking a lot about caterpillars lately. I read the Very Hungry Caterpillar to my son every night and it always makes me think of organizations going through transformative change.

What’s fascinating to me isn’t just the physical transformation that occurs. Obviously sprouting wings and becoming more colorful is amazing, but the internal composition changes too. Their appetites change. Their digestive systems change. But what really gets me is the perception-shifting that must occur. Imagine you’re stuck crawling on the ground and slowly climbing trees, flowers and bushes then suddenly you’re able to fly–to move nimbly. Imagine the cognitive transformation that first day when life is about exploring a much wider universe.

You think ARL will be ok with me citing a children’s book? I also want to pitch Willy Wonka as the role model for R&D. That’s for another …

Read More

September 26, 2011, 5:24 pm

What It Takes To Become A Scholar: helping students scale the taxonomy

We’re working on round two of our graduate student ethnography project. I plan to release the findings in January as a collection of whitepapers. The batch will include themes such as mentoring, collaboration, criticism, and work process. But my favorite thread is the progression from student to scholar. It has been fascinating to review the self-analysis and see how a scholar is defined.

Two predominate qualities emerged: independence & creation.

Some sample quotes:

1. “As you go through grad school at least in English, you begin to develop more into sort of an independent writer and scholar…. You start thinking more in terms of I’m making a book length argument that has to be interesting vs. I’ve gotta write this short argument that I think is what my professor wants.”

2. “As an undergrad, you were presented with deadlines that were dictated by others. And they we…

Read More

May 24, 2011, 12:11 pm

ORGANIZING A DESIGN CHARRETTE: gathering a visual response for learning spaces (a packet)

We’re gearing up for a sizeable renovation and I’ve been trying to include students and faculty in on the interior design discussion. I’m planning a full post on the campaign later this summer—most likely after ALA—but in the meantime, I wanted to share this with others working on similar projects.

I’m a big fan of the design charrette activity. Having used it before, I definitely wanted to include it in my current efforts. It’s easy to conduct and students always seem to enjoy it as an outlet for creative expression. It’s also helpful to gather a visual response when talking about learning spaces.

This time around I wanted to conduct the charrette in a very public setting. I wanted everyone who entered the library to see it happening and to have the opportunity to participate—or at the very least to take s look at the renderings and other information that was…

Read More

May 2, 2011, 1:03 pm

THE VIRTUAL REALITY: Exploring graduate student use patterns of the UCSB Library

The UC institutional repository doesn’t have a space for library-generated papers, so I’ll just post it here. My SEO is decent so hopefully it will be discoverable.

THE VIRTUAL REALITY
Exploring graduate student use patterns of the UCSB Library
An ethnographic study

Prepared by Lindsay Vogt, Anthropology Graduate Student, UCSB
in collaboration with Brian Mathews, UCSB Library

Executive Summary (with an internal link to the full report)

I’ll let the report speak for itself. This is part one of a larger project, currently on hiatus, but hopefully will pick up again after ALA in New Orleans. In a nutshell, I hired an anthro grad student to help me study grad students. This paper is an internal document, but I figured since many librarians are interested in anthro research these days that it might be of some value. After our next report is done we’ll turn our work into a…

Read More

April 27, 2011, 11:53 am

Can you encourage people to recycle? Our attempt to switch behavior

This is a follow-up to my post Assessing Your Greenness: a serendipitous stroll toward sustainability.

Our campus recently conducted a waste stream audit in the library. Download UCSB_Library_Waste_Audit What it boils down to is that our patrons generate a lot of garbage.This gives you a sense of coffee cups alone:

Coffe_cups

It will be interesting to see what happens when we temporarily close our coffee operations during a long renovation. Obviously students will still bring coffee into the building, but I suspect they’ll consume less when the impulse to purchase is no longer right down the hall.

Waste_stream

 

The biggest takeaway from this experience is that students are not recycling– at least not as much as we expected. UCSB prides itself on sustainability and being eco-minded so I was anticipating higher numbers. What it boils down to is that 66% of our waste is recyclable, yet only…

Read More

September 30, 2010, 7:28 am

Applied Diplomacy: an assessment attempt to discover what people know, don’t know, and want to know

I’m experimenting with an assessment tactic. I have a pile of multicolored sticky notes in my office related to another project AND I was preparing for a meeting with our User Services Group to talk broadly & briefly about the topic of assessment AND I was reading Visual Meetings–  so basically I was looking for a way to blend all those components.

 

A theme that I keep coming back to is do we know our users? We can read about them and obviously we interact with them daily, but do we really know them? Do we really know the process that grad students experience as they prep for candidacy?  Do we really know what happens in the dorms the night before a big assignment is due? Do we really know faculty tendencies when starting a new article or when they prep to teach a brand new course?

 

I tend to be overly curiously, but what about my colleagues? What I realized is that I didn’t know …

Read More

May 10, 2010, 4:19 pm

Need Homework Help? Don’t ask us! A review of Course Hero’s reference service (Part 2 of 3)

I found Course Hero’s Homework Help to be very tempting. Here
is the screen you are greeted with when you first log-in:

  
CourseHero_welcome_screen
  

Inviting, huh? And pretty straightforward. Subject. Ability
to attach your assignment. Your Due Date & Time. Submit it and then you just chill and wait while the
experts work on your homework.

 

Here is how it works:

  
Course_hero_how_works 

And who are these experts? Tutors who are allegedly affiliated with top tier universities. Let's meet a few:

  
Coursehero_expert1
Course_hero_expert2
Course_hero_expert3 

Ok, great. Let’s give them a test. I started with a fairly common question that you might get at any ref desk or via email or chat or txt around the country:

"I need to find
three "peer-reviewed" articles for my english class. I have to write
about Hawthorne and the occult. I was looking on google but can't seem to get
the full articles, just some abstracts?”


 This is the response I received:

  
Course_hero_question1
  

"Us" huh? Got a team…

Read More

May 5, 2010, 11:38 am

Assessing Your Greenness: a serendipitous stroll toward sustainability

Last week a student walked
into the admin office with an idea. He was very excited and enthusiastic. He
suggested that we get rid of the paper towel dispensers and install the new quick
dry systems like they have in McDonalds. Apparently there are newer models that
are much improved over the older style. I’ve discovered that there is much debate on this topic.

 

The student and I talked for
a good 15 minutes: the environmental impact, workload, sanitation, noise,
costs, etc. It was good to go through the process. To me the conversation
wasn’t necessarily about hand dryers, but about brainstorming and encouraging
this student to ponder the outcomes. I’m not opposed to the idea but the two
concerns I have are the sanitation of the “activate” button, and the noise that
the machines might make in our quiet study zones.

 

All and all it was a good
talk. I told him I was exploring a …

Read More

April 21, 2010, 12:25 pm

Building A User-Generated e-Reserve System: a glance into the archives of CourseHero (part 1)

My
post “
File Sharers Swap Scholarly Materials
Too

has been the most read item on this blog. People seem to really like that theme
so I’ll explore it a bit more. Often when we talk about Open Access,
Institutional Repositories, the Publishing Crisis, or similar topics it tends
to be very esoteric. There is a lot of rhetoric, debate, and models that honestly
I think only accountants and lawyers can get excited about. I’m not so sure
that the average faculty member really cares about the economics of the
publishing industry or a court’s interpretation of fair use. We’ll save that
for another day.


What
I’m really interested in is how all this stuff applies to the world outside of
libraries. I found it fascinating that The
Pirate Bay
had some (expensive) academic materials and not just Jay-Z
tracks or episodes of LOST. So, what if there was a site designed to collect
academic…

Read More