Elsewhere on the Chronicle site is an article by me and four scientists on overpublication in the sciences. Here are the opening paragraphs:
“Everybody agrees that scientific research is indispensable to the nation’s health, prosperity, and security. In the many discussions of the value of research, however, one rarely hears any mention of how much publication of the results is best. Indeed, for all the regrets one hears in these hard times of research suffering from financing problems, we shouldn’t forget the fact that the last few decades have seen astounding growth in the sheer output of research findings and conclusions. Just consider the raw increase in the number of journals. Using Ulrich’s Periodicals Directory, Michael Mabe shows that the number of ‘refereed academic/scholarly’ publications grows at a rate of 3.26 percent per year (i.e., doubles about every 20 years). The main cause: the growth in the number of researchers.
“Many people regard this upsurge as a sign of health. They emphasize the remarkable discoveries and breakthroughs of scientific research over the years; they note that in the Times Higher Education’s ranking of research universities around the world, campuses in the United States fill six of the top 10 spots. More published output means more discovery, more knowledge, ever-improving enterprise.
“If only that were true.”
Our thesis is that the explosion of research publication in the sciences in the last few decades has strained the quality controls to the breaking point. It has burdened peer review and other mechanisms to the point that the amount of research has not raised the overall quality of the enterprise, but has instead made it more difficult for researchers to produce their best work.
Many commenters have weighed in, criticizing us for recommending various controls on the rising output. Many of them make salient points, such as questioning the use of citation counts in order to judge the value of scientific work. But the big picture remains, I think, unaddressed. It can be summed up this way.
Individuals and institutions have allowed research output itself to be a prime factor in the judgment of individuals and institutions. Quantity is becoming a claim all its own, and the trend is getting worse.
9 Responses to Output Is Not Productivity
trendisnotdestiny - June 16, 2010 at 11:34 pm
Marc,Not that I am a corporatist, but aren’t you forgetting about data mining, ultra high functioning computers synthesizing vast quantities of data as well as the larger centers of economic neoliberal power which sell us: more is more?Quality control in this abstract setting seems like a canard as the whole process of setting standards, criteria, limits changes only the boundaries and not the underlying problems associated with market driven policies for socially driven human beings in education, health care and retirement… Some things need not be profit driven… I empathize with your discourage of the avalanche but wonder if you aren’t being co-opted by other “de-schooling” interests….For what it is worth, I appreciated your time and efforts
trendisnotdestiny - June 16, 2010 at 11:35 pm
I meant Mark! My apologies
nordicexpat - June 17, 2010 at 3:29 am
Wow, Mark, didn’t generate enough comments over there so you repeated it here :-). At the risk of adding to the pile of commentary (thus proving your thesis?), here’s my two cents:The biggest problem with your claim is that you really don’t produce evidence to support any of your contentions. Take this statistic: “[20 years ago] Only 45 percent of the articles published in the 4,500 top scientific journals were cited within the first five years after publication . . . Péter Jacsó found that 40.6 percent of the articles published in the top science and social-science journals were cited in the period 2002 to 2006.”What on earth does this mean? That the uncited articles shouldn’t have been published in the first place? Did peer review fail here? Did it fail because the reviewers had too many other (bad quality) articles to review and let these slip through? Are the articles unread because they were drowned out by the avalanche of low-quality papers? Is the 4.4 drop significant of something? Or this:”Even if read, many articles that are not cited by anyone would seem to contain little useful information.”Huh? On what basis is this claim made? The fact that they are not cited? But then you have to get into why articles are cited, since they are many reasons why articles may not get cited, and lack of little useful information may only be one of them. (Case in point, the Journal of the International Phonetic Association published an article on the phonetic description of Goizueta Basque. It just came out, but I would not be suprised if it isn’t cited much. But does that mean it isn’t valuable?) Or this:”Then, once it is published, it joins the multitudes of other, related publications that researchers must read and evaluate for relevance to their own work. Reviewer time and energy requirements multiply by the year. The impact strikes at the heart of academe.”Again, if this were, wouldn’t it mean that less articles would be produced because of the amount of time this takes? But you say this isn’t happening, so what exactly is this impact “that strikes at the heart of academe?”Simply put, we live in age where there is perhaps too much information rather than too little (as was perhaps the case in the past). But an educated citizenry needs to be able to negotiate this information, not wish it away. You are not going to put that genie back in the bottle, no matter how nostaglic you are for a mythical golden age. I don’t mean to be snarky, but couldn’t you have done a proper research article, subject to the contraints of peer review, that would demonstrate these claims with empirical evidence? It sounds like a classic case of “the pleasure of indulgement — let’s generate even more quasi-academic articles and blog posts for people to read and respond to — with the dignity of denial — all this so-called research is really a waste of time, isn’t it?”
markbauerlein - June 17, 2010 at 11:06 am
Fair criticisms, nordicexpat, and each of our contentions does indeed require more evidence. In fact, the scientist-co-authors have a much larger version of the article coming out in Academic Questions later this year. I would say, however, that while I agree that “an educated citizenry needs to be able to negotiate this information, not wish it away,” the volume is getting so large that it is unmanage-able. That’s the critical point. We don’t want to return to the Old Way, but we do need to develop systems and policies that address the proliferation of material.
_perplexed_ - June 17, 2010 at 12:27 pm
I frequently review poster submissions for large conferences, where there is room to accept nearly all submissions. In this reviewing context, it is important to distinguish between those posters that make only a very small postive contribution, and those which would actually negatively impact progress in the field. Only the latter get rejected. Contrary to your thesis, I wish that standard could always be applied: Nothing gives me more confidence that important work is not being shut out by gatekeeper bias than the appearance of clearly second-rate material.
markbauerlein - June 17, 2010 at 2:15 pm
Isn’t “gatekeeper bias,” perplexed, similar to “disciplinary standards”?
_perplexed_ - June 17, 2010 at 9:16 pm
There’s a big difference between work that fails to meet standards of scholarship (e.g., use of flawed methods when well-known alternatives are available) and “gatekeeper bias” (e.g., rejecting a work because a disagreement with foundational assumptions). I think most gatekeepers work hard not to let mere preference influence editorial decisons and recommendations, and I think they are largely successful. But “largely” isn’t always. Just because it is sometimes hard to discern the difference between standards and biases doesn’t mean the difference isn’t there.
tech2doc - June 23, 2010 at 12:50 pm
The same can be applied to the American workforce, in my humble opinion. Several times during my previous career as an IT consultant, a systems implementation project was cancelled because the CxO of the corporation decided to steer into a different direction. The hard work of 100+ employees and contractors was undone almost overnight. We count these statistics and pat ourselves on the back for having one of the highest work output per capita, yet so much of our work results in zero gain.This country has too many people. We look for ways to keep people busy and we create entire industries out of inserting middlemen where they are not needed. A simple example: Mail-in-rebates. Only our country has a multi-million dollar industry dedicated to wasting stamps, envelopes, paper and peoples time when the price could simply be removed at the register. There are literally hundreds of industries in this country which do nothing more than impede the efficiency of others and create additional costs for zero gain beyond giving people jobs which serve no real productive value.It only seems fair that education should have the same need to keep people busy without adding value. Isn’t that half the reason why education costs have soared in the past few decades?
nakliye - July 4, 2010 at 7:30 am
EVDEN EVE NAKLİYATNAKLİYAT HİZMETİNAKLİYE RESİMLERİEV TAŞIMA BİLGİLERİKÜBRA TAŞIMACILIK