The calculus class took their third (and last) hour-long assessment yesterday. In the spirit of data analytics ala the previous post here, I made boxplots for the different sections of the test (Conceptual Knowledge (CK), Computation (C), and Problem Solving (PS)) as well as the overall scores. Here are the boxplots for this assessment — put side-by-side with the boxplots for the same sections on the previous assessments. “A2″ and “A3″ mean Assessments 2 and 3.

Obviously there is still a great deal of improvement to be had here — the fact that the class average is still below passing remains unacceptable to me — but there have been some definite gains, particularly in the conceptual knowledge department.

What changed between Assessment 2 and Assessment 3? At least three things:

- The content changed. Assessment 2 was over derivative rules and applications; Assessment 3 covered integration.
- The way I treated the content in class changed. Based on the results of Assessment 2, I realized I needed to make conceptual work a much greater part of the class meetings. Previously the class meetings had been about half lecture, with time set aside to work “problems” — meaning, exercises, such as “find the critical numbers of \(y = xe^{-x}\). Those are not really problems that assess conceptual knowledge. So I began to fold in more group work problems that ask students to reason from something other than a calculation. I stressed these problems from the textbook more in class. I tried to include more such problems in WeBWorK assignments — though there are precious few of them to be had.
- The level of lip service I gave to conceptual problems went up hugely. Every day I was reminding the students of the low scores on Conceptual Knowledge on the test and that the simplest way to boost their grades in the class would be to improve their conceptual knowledge. I did not let their attention leave this issue.

Somewhere in a combination of these three things we have the real reason those scores went up. I tend to think the first point had little to do with it. Integration doesn’t seem inherently any easier to understand conceptually than differentiation, particularly at this stage in the course when differentiation is relatively familiar and integration is brand new. So I think that simply doing more conceptual problems in class and stressing the importance of conceptual knowledge in class were the main cause of the improvements.

Quite interestingly, the students’ scores on computation also improved — *despite the reduced presence of computation in class* because of the ramped-up levels of conceptual problems. We did fewer computational problems on the board and in group work, and yet their performance on raw computation improved! Again, I don’t think integration is easier than differentiation at this stage in the course, so I don’t think this improvement was because the material got easier. Maybe the last test put the fear of God into them and they started working outside of class more. I don’t know. But this does indicate to me that *skill in computation is not strictly proportional to the amount of computation I do, or anybody else does, in class*.

To overgeneralize for a second: **Increased repetition on conceptual problems improves performance on those problems dramatically, while the corresponding reduction in time spent on computational exercises not only does not harm students’ performance on computation but might actually have something to do with improving it. **If we math teachers can understand the implications of this possibility (or at least understand the extent to which this statement is true) we might be on to something big.

The scores on problem solving went two different directions. On the one hand, the median went up; but on the other hand the mean went down. And the middle 50% didn’t get any better on the top end and got worse on the bottom end. I’m still parsing that out. It could be the content itself this time; most of the actual problems in integration tend to take place near the end of the chapter, after the Fundamental Theorem and u-substitution, so the kinds of problems in this section were less than a week old for these students. But quite possibly the improvement in conceptual knowledge brought the median up on problem solving, despite the newness of the problems. Or maybe the differences aren’t even statistically significant.

What I take away from this is that if you want students to do well on non-routine problems, those problems have to occupy a central place in the class, and they have to be done not outside of class where there’s no domain expert to guide the students through them but in class. And likewise, we need not worry so much that we are “wasting precious class time” on group work on conceptual problems at the expense of individual computation skill. Students might do just fine on that stuff regardless, perhaps even better if they have enhanced conceptual understanding to support their computational skills.

It all goes back to support the inverted classroom model which I’ve been using in the MATLAB course, and now I’m wondering about its potential in calculus as well.