Back in November, I posted about Just-in-Time Teaching (JiTT), a pedagogy practice that has its roots in physics but has since been adopted in many other disciplines. That post spurred discussion about how some other widely-adopted practices in physics education research (PER) may or may not be present in other disciplines. So I’m starting a periodic series, in which I introduce several PER practices and open the topic up for discussion. My goal is that our community of readers will hopefully learn something new but also have the opportunity to discuss how these principles are used elsewhere. Good pedagogy is universal.
Concept inventories are a tool used in PER to measure how learning is affected by implementing practices such as JiTT. Wikipedia gives a good description: “A concept inventory is a criterion-referenced multiple choice test designed to evaluate whether a person has an accurate working knowledge of a specific set of concepts.” Several have been developed to measure learning in physics topics such as mechanics, electricity and magnetism, optics, and basic circuits. The inventories are often utilized to measure student understanding of a topic before and after instruction, as a part of a design in which factors such as type of instruction are controlled. For example, last fall I participated in a study which aimed to measure student understanding before and after instruction via a clicker-facilitated interactive lecture demonstration (ILD). The study was constructed such that the only formal instruction the students had on the topic came from the ILD, and the results of the concept inventory gave a representation of the students’ change in conceptual understanding as a result of participation in the ILD. Usually, student learning is measured as some ratio of a concept inventory score before and after instruction.
In my department, concept inventories are also used to measure student learning outside of traditional homework sets and tests for the purpose of assessing departmental practices. One item we’re interested in is how students over time understand the principles of mechanics taught in our introductory physics course as factors such as textbook, instructors, and even learning environments change. This data is tracked over several years to ensure that our standard of student learning is maintained. For example, we were fortunate this fall to begin teaching in our college’s beautiful new science center, and our intro physics classroom has been designed with PER principles in mind. Our data will allow us to have some sense of what influence this new learning space might be having on our students.
Standardized tests can be controversial, but one thing I appreciate about concept inventories is that they’re not really meant to be a final, summative assessment of student learning, although they can be used that way. I don’t tie my students’ performance on these tests to their final grade. And students have commented to me that they appreciate having the chance to see how far they’ve come after a semester’s instruction. Taking the post-instruction concept inventory actually seems to give them a bit of a confidence boost, in preparation for final exams, or a wake-up call for how much studying they need to do to catch up.
My favorite collection of physics concept inventories is listed at North Carolina State’s Assessment Instrument Information Page. A quick perusal of the list might give you ideas as to how some of the concept inventories, such as the Test of Measurement Uncertainty Concepts, could be used for other disciplines.
That’s the story on concept inventories from the PER world. Let’s open up for discussion on how they might be used in other fields. Do you know of any depositories of concept inventories in your area? What are some potential caveats for using concept inventories? True to the ProfHacker ethos, what are some ways you could hack concept inventories for your own use? Let us know in the comments.
Image by Flickr user PinkMoose / Creative Commons licensed
Follow Profhacker through your favorite RSS reader: 
23 Responses to Concept Inventories: Beyond Physics
schellenberg - February 10, 2011 at 6:20 am
Information on the Geoscience Concept Inventory, intended primarily for introductory physical geology courses, can be found at: https://www.msu.edu/~libarkin/gci.html
cmelani - February 10, 2011 at 10:51 am
I think it is important to emphasize that concept inventories are formative assessment instruments designed to elicit student thinking. They can inform both the student and the instructor about potential problems with understanding (Ascertain what the learner knows and teach accordingly: Ausubel), but they are far too “blunt” an instrument to assess deep conceptual understanding. I can answer the force concept inventory questions, but i am not convinced that a have a deep conceptual grasp of the concepts. IMHO the push to develop yet more of these inventories without thought to more meaningful summative assessments is problematic. Using concept inventories as pre/post assessments is a fundamentally flawed methodology, and should not be relied on as evidence of deep conceptual understanding
That being said there are inventories for most science and engineering disciplines – there is a list here https://engineering.purdue.edu/SCI/workshop/tools.html)
wlgoffe - February 10, 2011 at 9:51 pm
The item that really caught my eye was this use of concept inventories: “One item we’re interested in is how students over time understand the principles of mechanics taught in our introductory physics course as factors such as textbook, instructors, and even learning environments change. This data is tracked over several years to ensure that our standard of student learning is maintained.”
I’ve never seen that done in my discipline (economics) and I’d hazard a guess it is rare in others based on what many (most?) academics say about assessment. Maybe physics and their general approach to teaching has some broader principles for all of us?