next »

[conjugate]

The most interesting observation we might make about US education is one which NAEP can't make because they've used every excuse under the sun to not release state by state scores for 12th graders.  But SAT does, and it found that almost without exception the worst performing states are the states who spend as much as five times per student as the highest scoring states.  The differences in education quality is not insignificant--it's more than a standard deviation, or 170 SAT points.

Actually, it usually takes two SDs to be significant (p<0.05 for a two-tailed test). IIRC, I don't believe that 170 points makes that cut.

Er, you guys are mixing your meanings about SD's here. You can detect a difference - sometimes small - in the means of two normally distributed populations and prove that the difference is statistically significant at some p value with a test. It's certainly possible that, for example, the mean SAT score in state X is at least 1 SD (with respect to the SAT distribution) lower than in state Y, at a p value of <0.01 (which corresponds to something >3 SD's away in the distribution used to test this hypothesis).

Because the sampling deviation is the population deviation divided by the square root of the sample size.  Central Limit Theorem strikes again! 

Hah.  I just had to post this to show off that I remember some statistics.

[cgfunmathguy]

How sad--or how wonderful--is it that I, a math phobic (in spite of doing well using statistical process control in a manufacturing environment) , have been following this discussion and even I understand DvF and cgfunmathguy's logic regarding methods, and understand it as valid?

For someone that likes to spout statistics, you (jacobisrael) would seem to be either being purposefully obtuse at understanding the importance of underlying methodology, or you would seem to have some different agenda that I would prefer not to explore.

*Thanks DvF and cgfunmathguy for providing clarity amidst the static*

*Disappears, with a better understanding of International educational comparison statistical studies than I had before I read*

(In fairness, some thanks to jacobisrael for providing the counterpoint that allowed the clarity to shine through, although it really didn't take me this far into the thread to see it.)

One cannot see light, unless there is darkness to provide a contrast.  (paraphrase, Bob Ross)

*poof*

You are most welcome, Mystic. If only our students understood these issues as well.

[wanna_writemore]

My survey history courses are full of engineering, business, and science majors.  When I ask them in class to tell me how many years passed between 2 major events, I get a roomful of blank stares.  I sometimes have to tell them that subtraction will get them there and prod them to figure it out.

It scares me that people this incapable are in my classes.  If they're not incapable, they're just lazy, and that's just as bad.


Explaining how to calculate their current course averages receives quite the result.  They all seem to think I'm a math whiz.  I'm not. I'm fairly competent in basic math to deal with bank accounts and personal finances, the minimal math needed in teaching and researching history, etc., but nothing complex.

[daniel_von_flanagan]

My survey history courses are full of engineering, business, and science majors.  When I ask them in class to tell me how many years passed between 2 major events, I get a roomful of blank stares.

That's because they didn't know they were supposed to bring their calculators to a humanities class. - DvF

[conjugate]

My survey history courses are full of engineering, business, and science majors.  When I ask them in class to tell me how many years passed between 2 major events, I get a roomful of blank stares.

That's because they didn't know they were supposed to bring their calculators to a humanities class. - DvF

It saddens me to see them use the calculator to figure out 84 ÷ 2.

[daniel_von_flanagan]

My survey history courses are full of engineering, business, and science majors.  When I ask them in class to tell me how many years passed between 2 major events, I get a roomful of blank stares.

That's because they didn't know they were supposed to bring their calculators to a humanities class. - DvF

It saddens me to see them use the calculator to figure out 84 ÷ 2.

Obviously, I agree, but (as you likely have already observed) students today are so wedded to calculators they can't even do noncomputational problems without having one nearby, as a kind of charm.

My son had his calculator confiscated once in middle school because he was using it to play a game. When his dean called me, I said they could keep the thing, we'd only bought it because it was a requirement and I would prefer my son to not use one.  The dean was surprised - she'd only ever talked about this kind of thing to people in Education departments, never to actual STEM practitioners, and didn't realize that many of us find calculators an impediment to learning.

I wanted to require slide rules for my classes, but nobody makes them anymore. - DvF

[mystictechgal]

I still have 3 or 4 slide rules if you really need one.  It was a requirement in one (HS non-math class) to use slide rules (only in start of class quizzes, never in the actual class) that 'put the nail in the coffin', as the saying goes, for me, where math was concerned.  I can laboriously, if I have the directions handy, still manage to use one for the most basic calculations, but I've never been either fast or facile with it.

Last year my husband and I happened to mention them when talking with the teens at the zoo.  They'd never heard of them before.  I brought one in, along with the directions, and we had great fun watching as a few of the maths whizzes among them tried to decipher the basics.  They couldn't grasp it even after my husband, who was much better at math (and slide rules) than I, showed them how the calculations were done.

The most fun was watching the expressions on their faces when we pointed out that we first got to the moon--and, later, managed to do the calculations necessary to get Apollo 13 home safely (they'd all seen the movie)--using slide rules.

They still fascinate me, and I'm still more than a little po'd that my teacher used them in more of a punitive manner than in any way that would have actually fostered and encouraged any interest I might have had, given that their only place in the class was tangenital to everything else.  Altogether the single most frustrating experience I have ever exerienced in any class I have ever taken.  Truly a hate/love experience, given that I've kept them all these years.  (Then again, I have an abacus or two around here, too.)

[daniel_von_flanagan]

I still have 3 or 4 slide rules if you really need one.

I have a few, but would like 30 or so cheapies that I could pass out to my students for use on quizzes and exams.  The problem with calculators is that some of them are too smart (a few can do Calculus) and promote cheating.  A slide rule can multiply, take roots, and compute basic transcendental functions all very easily, and that's really all I'd like my students to have access to in a computing aid. - DvF

[jacobisrael]

How sad--or how wonderful--is it that I, a math phobic (in spite of doing well using statistical process control in a manufacturing environment) , have been following this discussion and even I understand DvF and cgfunmathguy's logic regarding methods, and understand it as valid?

For someone that likes to spout statistics, you (jacobisrael) would seem to be either being purposefully obtuse at understanding the importance of underlying methodology, or you would seem to have some different agenda that I would prefer not to explore.

*Thanks DvF and cgfunmathguy for providing clarity amidst the static*

*Disappears, with a better understanding of International educational comparison statistical studies than I had before I read*

(In fairness, some thanks to jacobisrael for providing the counterpoint that allowed the clarity to shine through, although it really didn't take me this far into the thread to see it.)

One cannot see light, unless there is darkness to provide a contrast.  (paraphrase, Bob Ross)

*poof*

You are most welcome, Mystic. If only our students understood these issues as well.

It doesn't seem like you understood the point?  Or maybe you don't want to understand the point?

It's a correct, accurate, and honest statement to say that the difference from state to state in SAT math scores is more than a standard deviation.  If you don't like the way the College Board calculates it, you need to talk directly to them and stop debating it here.

Since it appears to have hit a sore spot, let's be more specific with the figures.  Before SAT scores were "recentered" [a euphamism for "raising" scores artificially more than a standard deviation to conceal the 140 point drop in SAT scores], Iowa's SAT math score was 583, which was 119 points higher than Rhode Island's score of 464, and the standard deviation for Iowa was 99, meaning that Rhode Island scored 1.2 S.D. lower than Iowa.

That's a statement of fact.  That's not an opinion.  If the College Board is wrong, then you need to talk to TIMSS also, because they observed the same phenomena.

Pennsylvania scored 1.24 S.D. lower. Washington, DC, scored 1.4 S.D. lower.

You don't think that's worth examining?  When the highest scoring states spend one fourth or one fifth as much per student for education as the lowest scoring states, we need to know why.   And guess what?  According to NCES, and the "Glen Report", THEY CAN'T TELL YOU WHY.

When scores are different by THAT much, and when the difference is confirmed by TIMSS benchmarking studies, it would take an utter fool to not grasp the reason.

You can't recommend a solution if you don't even know the problem. 

[jacobisrael]

Are you sure that you've read that TIMSS study about our 12th grade scores?  The methodology for picking the cohorts was the same in both the 8th and 12th grade

They nevertheless are not the same cohort.  The reason is that the 8th graders were in 8th grade that year, and the 12th graders were in 12th grade that year.  In many cases, when the 12th graders were in middle school they had different curricula than the 8th graders did when they were in middle school.

This is not complicated stuff.  Really. - DvF

Now I understand your point.  Thank you very much for clarifying it.

Please point me to the evidence that there was a national, across the board, change in the curricula between 1991 and 1995 if you believe this to be a possible explanation.  Can the same be said for all of the other countries which took TIMSS?

If anything DID change (and this is not to even hint that anything changed) then would you not agree that our change was clearly for the worse and theirs was for the better?

Austria's scores were an exception in Europe, as they followed a similar pattern to the US, only more extreme.  While our boys' scores decreased 56 points, theirs decreased 85 points.  And while our girls' scores decreased 104 points, their decreased 137 points.  So while just the increase in the gender gap was 48 points in the US, it was 52 points in Austria.  This is not an insignificant decrease, since the standard deviation for US girls was 53, making this 0.91 S.D.  Since the standard deviation for Austrian girls was larger, at 71, the increase in their gender gap was smaller, at 0.73 S.D.

But there was already an 8 point gender gap in Austrian 8th graders, making their total gender gap by 12th grade 0.85 S.D.

I'm not clear on how changes in the curricula could have affected any of this.  I don't even know what can be changed to cause such huge race and sex gaps, or to make them bigger or smaller.  So it would be greatly appreciated if you'd provide an example.

Actually, I can think of one small example.  Not too long ago, Chinese educators were invited to visit the US to study our education system.  They asked many great questions, and my input was they should implement calculus in high school as Japan had.  They did that, and now 95% of Chinese students complete calculus before they graduate from high school.

Pretty smart, eh?  What have our educators done lately to top that?

I've tried to stay out of this one as DvF has done an admirable job of presenting the points I wanted to make. However, please allow me to add my two cents' worth. First, you are comparing different systems that do different things. You are comparisons are being made between countries where there are NATIONAL curricula, those where there are STATE curricula, and at least one where it is a hodgepodge of STATE and LOCAL curricula. So, we are comparing apples to oranges to pears

Also, we need to address the differences in systemic student handling. In the US, we send the vast majority of our students to high school; other countries reverse this entirely. Thus, the 12th-grade cohorts aren't even comparable between countries, even though they are presented as such by the media (among many others). While the 4th-grade cohorts may be similar, there is even some question about the comparing 8th-grade cohorts by some. For the two reasons above, I don't believe TIMSS is as valid an indicator of differences between national systems as its exhorters proclaim.

Finally, a word about why DvF keeps trying to get you to understand why comparing cohorts is important. Many states have been adjusting/rewriting their regulations (Pennsylvania), their state-mandated tests (Ohio), and their state-mandated curricula (Georgia) for the past decade or more. In mathematics, the National Council of Teachers of Mathematics (NCTM) issued its first set of standards on K-12 mathematics in 1989. This was the first step in the reform process, and several states began the process of reforming state curricula in the early 1990s. Others waited longer. However, the process is not an instantaneous one. As an example, Georgia instituted the Georgia Performance Standards (GPS) in 2003 or 2004. The standards still aren't fully implemented throughout the schools yet, and they won't be for two more years. So, yes, cohort matters, and we need to deal with the data that way. The only fair comparisons about gains and losses in the report's 12th-grade cohort would be to take the 2007 report's 12th-graders and compare that gap (assuming all the other confounding variables didn't exist) to the gap found in the 2003 report's 8th-graders and to the gap found in 1999 report's 4th-graders. This assumes that the tests across that EIGHT-YEAR SPREAD are equivalent.

This is about like (no, it's exactly like) saying that you know one girl who's taller than many of the boys, therefore girls are just as tall as boys.

In regard to height, the standard deviation for both sexes is the same, 2.8 inches.

But the GAP between the mean scores is, yet again, two standard deviations (1.893 to be exact).

There's no way to announce that a gender gap of 1.893 standard deviations is not significant. It has a HUGE impact on our world that simply cannot be ignored, not even in a theoretical sense.

[cgfunmathguy]

It doesn't seem like you understood the point?  Or maybe you don't want to understand the point?

It's a correct, accurate, and honest statement to say that the difference from state to state in SAT math scores is more than a standard deviation.  If you don't like the way the College Board calculates it, you need to talk directly to them and stop debating it here.

Since it appears to have hit a sore spot, let's be more specific with the figures.  Before SAT scores were "recentered" [a euphamism for "raising" scores artificially more than a standard deviation to conceal the 140 point drop in SAT scores], Iowa's SAT math score was 583, which was 119 points higher than Rhode Island's score of 464, and the standard deviation for Iowa was 99, meaning that Rhode Island scored 1.2 S.D. lower than Iowa.

That's a statement of fact.  That's not an opinion.  If the College Board is wrong, then you need to talk to TIMSS also, because they observed the same phenomena.

Pennsylvania scored 1.24 S.D. lower. Washington, DC, scored 1.4 S.D. lower.

You don't think that's worth examining?  When the highest scoring states spend one fourth or one fifth as much per student for education as the lowest scoring states, we need to know why.   And guess what?  According to NCES, and the "Glen Report", THEY CAN'T TELL YOU WHY.

When scores are different by THAT much, and when the difference is confirmed by TIMSS benchmarking studies, it would take an utter fool to not grasp the reason.

You can't recommend a solution if you don't even know the problem. 

Actually, you have missed my point. I did NOT say that the numbers are wrong. I am NOT disputing their calculation. I am NOT even saying that it's not worth examining. My point was/is that stating that something is significant just because the number seems large is an invalid argument statistically. "Significant differences" is a term with a fairly precise meaning and is only stated along with a confidence level. This is something that anyone who has passed an introductory statistics course should know. Your emphasis on the size of the difference (whether normed or not) shows me that you really don't understand this very basic idea. Someone who did understand it would have already reported that the difference was ___, which is significant at the ___% level. You haven't done this.

For another view of it, let's look at your classroom. In a large lecture class, grades tend to be distributed "normally". This being the case, "curving" (with its true meaning) would assign Cs to the 68% of the students whose scores are within 1 SD of the mean. So, let's assume that the mean on Test 1 was 75 with a standard deviation of 8. So, any student with a score between 67 and 83, inclusive, should get a C. However, Susie with her 81 and Johnny with his 69 both got Cs! Is the difference significant? We don't know until we run tests on the scores. Even though the difference is 12 points (which is 1.5 SD), it is likely that this difference is NOT "statistically significant" at any appreciable level. To constantly quote raw numbers with no test results is worthless and misleading. Even those with an agenda don't do this because they know they will be accused of trying to bamboozle the people reading the report.

Take a stats class, and then come back into the discussion.

[daniel_von_flanagan]

When scores are different by THAT much, and when the difference is confirmed by TIMSS benchmarking studies, it would take an utter fool to not grasp the reason.

OK, here an exercise of the sort I assign in my senior-level undergraduate statistics courses:  Given 50 independent, identically-distributed normal random variables, what is the exact probability that all of them fall within 1 standard deviation?  Within 2 standard deviations?  What is the answer if we trim N extreme values from either end?  (Hint: "order statistics")

New rule: nobody, in a discussion like this, gets to call anyone else an "utter fool" if they cannot solve such a problem, and recognize its relevance to the discussion. - DvF

[cgfunmathguy]

This is about like (no, it's exactly like) saying that you know one girl who's taller than many of the boys, therefore girls are just as tall as boys.

In regard to height, the standard deviation for both sexes is the same, 2.8 inches.

But the GAP between the mean scores is, yet again, two standard deviations (1.893 to be exact).

There's no way to announce that a gender gap of 1.893 standard deviations is not significant. It has a HUGE impact on our world that simply cannot be ignored, not even in a theoretical sense.

I don't understand how you claim dealing with height differences between the sexes is like the rest of the discussion. Now, you REALLY are comparing apples to oranges.

Provide the statistical analysis that supports calling differences significant along with the confidence level used for the test or STFU. I will ignore the remainder of your posts until this is done.

On preview: Thank you, DvF.

[high_energy_photons]

I wanted to require slide rules for my classes, but nobody makes them anymore. - DvF

You should talk to your local school districts.  They may have some in storage from back in the day.  I had a teacher who taught us how to use slide rules as part of an appreciation of technology lesson, and she got some really nice slide rules from the district.  I was not allowed to use calculators until middle school, and even then not until close to the end (and not graphing).  Before we started using calculators, we had a brief lesson on abacuses.  Then we had a few weeks of slide rule.  Finally, we got the calculators, and we understood their limitations by that point.

It makes me sad that so many of my students don't understand that their calculators are stupid machines.  Calculators are a drain on thinking, and I ban them as much as I can (banned them when I taught high school).

[mystictechgal]

I wanted to require slide rules for my classes, but nobody makes them anymore. - DvF

You should talk to your local school districts.  They may have some in storage from back in the day.  I had a teacher who taught us how to use slide rules as part of an appreciation of technology lesson, and she got some really nice slide rules from the district.  I was not allowed to use calculators until middle school, and even then not until close to the end (and not graphing).  Before we started using calculators, we had a brief lesson on abacuses.  Then we had a few weeks of slide rule.  Finally, we got the calculators, and we understood their limitations by that point.

It makes me sad that so many of my students don't understand that their calculators are stupid machines.  Calculators are a drain on thinking, and I ban them as much as I can (banned them when I taught high school).

Google slide rule+buy and you'll get a lot of sites.  With a statement like this: "TEACHERS, do you need inexpensive rules for classes?
We have stock set aside for this purpose, and offer it at reduced prices. EMAIL SUSAN HERE [link removed] if you need help with this purchase, and have limited or no funding."  this site looks promising: http://www.sphere.bc.ca/test/cheap.html#catalog