About a year ago, I started partitioning up my Calculus tests into three sections: Concepts, Mechanics, and Problem Solving. The point values for each are 25, 25, and 50 respectively. The Concepts items are intended to be ones where no calculations are to be performed; instead students answer questions, interpret meanings of results, and draw conclusions based only on graphs, tables, or verbal descriptions. The Mechanics items are just straight-up calculations with no context, like “take the derivative of \(y = \sqrt{x^2 + 1}\)”. The Problem-Solving items are a mix of conceptual and mechanical tasks and can be either instances of things the students have seen before (e.g. optimzation or related rates problems) or some novel situation that is related to, but not identical to, the things they’ve done on homework and so on.

I did this to stress to students that the main goal of taking a calculus class is to learn how to solve problems effectively, and that conceptual mastery and mechanical mastery, while different from and to some extent independent of each other, both flow into mastery of problem-solving like tributaries to a river. It also helps me identify specific areas of improvement; if the class’ Mechanics average is high but the Concepts average is low, it tells me we need to work more on Concepts.

I just gave my third (of four) tests to my two sections of Calculus, and for the first time I started paying attention to the relationships between the scores on each section, and it felt like there were some interesting relationships happening between the sections of the test. So I decided to do not only my usual boxplot analysis of the individual parts but to make three scatter plots, pairing off Mechanics vs. Concepts, Problem Solving vs. Concepts, and Mechanics vs. Problem Solving, and look for trends.

Here’s the plot for Mechanics vs. Concepts:

That r-value of 0.6155 is statistically significant at the 0.01 level. Likewise, here’s Problem Solving vs. Concepts:

The r-value here of 0.5570 is obviously less than the first one, but it’s still statistically significant at the 0.01 level.

But check out the Problem Solving vs. Mechanics plot:

There’s a slight upward trend, but it looks disarrayed; and in fact the r = 0.3911 is significant only at the 0.05 level.

What all this suggests is that there is a stronger relationship between conceptual knowledge and mechanics, and between conceptual knowledge and problem solving skill, than there is between mechanical mastery and problem solving skill. In other words, while there appears to be some positive relationship between the ability simply to calculate and the ability to solve problems that involve calculation (are we clear on the difference between those two things?), the relationship between the ability to answer calculus questions involving *no* calculation and the ability to solve problems that *do* involve calculation is stronger — *and so is the relationship between no-calculation problems and the ability to calculate*, which seems really counterintuitive.

If this relationship holds in general — and I think that it does, and I’m not the only one — then clearly the environment most likely to teach calculus students how to be effective problem solvers is not the classroom primarily focused on computation. A healthy, interacting mixture of conceptual and mechanical work — with a primary emphasis on conceptual understanding — would seem to be what we need instead. The fact that this kind of environment stands in stark contrast to the typical calculus experience (both in the way we run our classes and the pedagogy implied in the books we choose) is something well worth considering.