Open Access Does Not Equal More Citations, Study Finds

April 1, 2011, 4:12 pm

A new study suggests that while open access appears to increase the readership of scholarly articles, it doesn’t increase how often they’re cited.

The study stands in contrast with earlier research that suggested open-access articles were referenced by other scholars more frequently.

Philip M. Davis, a postdoctoral associate in the department of communication at Cornell University, was given access to 36 subscription-based journals produced by seven different publishers. In 2007 and early 2008, he randomly made approximately 20 percent of their articles free.

He tracked the number of abstract views, full-text downloads, PDF downloads, and citations within the next year for the 3,245 articles in the study. The findings were published Wednesday in the Federation of American Societies for Experimental Biology Journal.

Free articles were downloaded more than twice as frequently as the paid-access articles, and PDFs of the free articles were downloaded 1.6 times more frequently.

But despite appearing to be more widely read, the free articles were no more likely to be cited than their counterparts.

Mr. Davis suggests that this may be because access is not a problem for most researchers who would cite the articles in their own work.

“For the most part, authors are located at institutions with very good access to the literature,” he says. “If authors don’t have formal access through their library, they seem to get informal access,” such as by asking colleagues at other universities who do have access to send a copy, or getting a copy from the authors.

In a post on Nature’s The Great Beyond blog, open-access advocate Stevan Harnad, a research chair in cognitive science at the University of Quebec at Montreal and professor of cognitive science at the University of Southampton, questions Mr. Davis’s methodology and motivation.

Mr. Davis says he doesn’t see his study as a blow to open access—if anything, he thinks it calls into question the wisdom of looking only at citation counts to measure the impact of a journal article, particularly given the ease of tracking article downloads online. “Twenty years ago, there was no way of measuring readership,” he says.

He says there should be some attempt to take into account the impact that journal articles have on all the readers who don’t conduct their own research on a similar topic.

“When we really think about what citations measure,” he says, “they’re really only looking at access to literature for a very narrow group.”

This entry was posted in Uncategorized. Bookmark the permalink.

E-mail

pdawalt

It is a shame that politics plays a part in everything. This is so counter productive.
cbres

I met Mr Fingerhut at a meeting of provosts in Columbus and was very impressed by his commitment to higher ed, his grasp of issues and his outright intelligence.
DF

What’s counterproductive about listening to the voters? Oh, yeah, I forgot. Most academics want smart people to decide everything.
paldy

Oh, now this exudes with intelligence. I’d say you don’t know a thing about academics. You only have images which you believe – fact or not. Now you can call me names, and sling innuendos for my challenge to you. That seems to be the norm from people who make comments like you. So far off base. A shame to read in a blog that should be above the usual bloggie people that post.
DF

I have been a professor for 20 years, so I know all too well how most academics cannot bear letting conservative voters have a say.
firstone

I don’t usually respond to articles, but thought this article and the first set of responses deserved more discussion. The shame of losing Mr. Fingerhut has nothing to do with liberal or conservative, but rather the loss of one of the most creative and thoughtful leaders in American higher education today. It would be hard to look at his work in Ohio and brand it as liberal. He fought for stronger public accountability, mostly through performance based funding. These are hardly liberal ideas. What made him so special, is that he pursued this agenda with his heart and mind both solidly grounded in the public good that public higher education provides and the need to restore the public trust in this enterprise.
arthist030

Wait, if he confirmed that open access articles are more likely to be read, isn’t that in itself enough of a justification to promote open access?

And, isn’t the whole point of citation counts to measure the influence of an article — not whether it happened to be published open-access or not? If it were the case that open-access increased citations, wouldn’t that mean that citation counts mattered less, because they reflected publication format (vs. quality of work)? People would say “that work is highly cited only because it’s open-access.”

So the current situation – where open-access increases readership (a good thing) but doesn’t affect citation counts (thus preserving the integrity of citation counts as an index of quality) is fine.

I don’t see what the big problem is here.
http://www.crsc.uqam.ca/ Stevan Harnad

“THE SOLE METHODOLOGICALLY SOUND STUDY OF THE OPEN ACCESS CITATION ADVANTAGE”

First, downloads of research findings are important, and they’re being measured. And evidence on the open-access *download* advantage is growing. See S. Hitchcock’s “The effect of open access and downloads (‘hits’) on citation impact: a bibliography of studies” http://opcit.eprints.org/oacitation-biblio.html

But the reason the open-access *citation* advantage — see Ben Wagner’s “Open Access Citation Advantage: An Annotated Bibliography” http://www.istl.org/10-winter/article2.html — is especially important is that refereed research is conducted and published so it can be accessed, used, applied and built upon in further research. It is conducted by researchers, for uptake by researchers, for the benefit of the public that funds the research. Researchers’ careers and funding depend on their research impact.

The greatest growth potential for open access today is through open access self-archiving mandates by universities and research institutions (e.g., Harvard and MIT), the universal providers of research.
http://roarmap.eprints.org/

Universities adopt open access mandates in order to maximize their research impact. The large body of evidence, in field after field, that open access increases citation impact, helps motivate universities to mandate the open-access self-archiving of their research output, in order to make it accessible to all its potential users — not just to those whose universities can afford subscription access — so all can apply, build upon and cite it. (Universities can only afford subscription access to a fraction of research journals.)

The Davis study lacks the statistical power to show what it purports to show, which is that the open access citation advantage is not causal, but merely an artifact of authors self-selectively self-archiving their better (hence more citable) papers. The Davis study’s sample size was smaller than that of many of the studies reporting the open access citation advantage. Davis found no citation advantage for randomized open access. But that alone does not demonstrate that open access is a self-selection artifact — in that study or in any other study — for Davis did not replicate the widely reported self-archiving advantage either, and the studies on that advantage are often based on far larger samples. So the Davis study is merely a small non-replication of a widely-reported outcome. (There are a few other non-replications; but most of the studies to date replicate the citation advantage, especially those based on bigger samples.)

Davis says he does not see why the inferences he attempts to make from his results — that the reported open access citation advantage is an artifact, eliminated by randomization, that there is hence no citation advantage, which implies that there is no research access problem for researchers, and that researchers should just content themselves with the open access download advantage among lay users and forget about any citation advantage — are not welcomed by researchers.

These inferences are not welcomed because they are based on flawed methodology and insufficient statistical power and yet they are being widely touted (particularly by the publishing industry lobby) as being the sole methodologically sound test of the open access citation advantage! Ignore the many positive studies. They are all methodologically flawed. The definitive finding, from the sole methodologically sound study, is negative. So there’s no access problem, researchers have all the access they need — and hence there’s no need to mandate open access self-archiving.

No, this string of inferences is not a “blow to open access” — but it would be if it were taken seriously.
mbelvadi

While profs are mostly concerned with citation rates by their peers, in furtherance of their own careers, it’s important to remember that other mission of higher ed, the “ed” part. Open access exposes undergraduates, who wouldn’t have the time or patience to wait for interlibrary loan, or the connections to get copies “informally”, to more cutting edge research than they otherwise would see. Course-assigned papers’ citations obviously aren’t going to get picked up in this kind of research. And making only 20% of a given journal “open access” is close to worthless for students, because the searchable lists of journal “holdings” (what’s immediately available to for the student to read) that libraries maintain usually include only journals that are either under paid subscription or 100% (or close to it) open access. Undergrads aren’t known for using Google effectively to locate scholarly articles.
I know that serving undergrads better is not the focus of the open access movement, but it’s perhaps a valuable unintended consequence which is also worth considering in the discussion. After all the undergrads, and their parents, are among the taxpayers who paid for the research too. And there might be second-order effects much later on, as some of these undergrads go on to be grad students and “real researchers,” and might be inspired by or even just remember that important article they found as an undergrad and use it in their own research.
phil_davis

I encourage readers to consult the FASEB article before commenting and avoiding the cut-and-paste response typical of online discussions (e.g. stevanharnad). Our research reveals that providing free access to the scientific literature has real benefits outside the research community, although it may not translate into more citations. These results make theoretical and practical sense. And while they may contradict a history of studies that may state the contrary, readers should weigh the evidence based on the strength of the methodologies and not simply on whether they support or contradict a particular political view.
11134078

Should I happen to live long enough to get down to the large-scale work I had hoped to do in retirement and so far have been kept from by illness, I shall have read an enormous amount of stuff that will not be cited because it has gone into my head as background. And yet it may be far more important for the somewhat indirect influence it will have had on me than the directly and immediately relevant, and therefore cited, material.
cybrarian_ca

I’m a big proponent of open access, but I see one potential problem with Mr. Davis’s work. Yes, access to these paid subscription journals is less likely to be a problem for researchers affiliated with academe – especially research universities. But one of the big benefits of open access is that the research can be shared with those who cannot afford increasingly expensive scholarly journals. I’m an academic librarian in a mid-sized college that isn’t quite a research university, but has some strong research programs – and my collections budget has dropped by 15% this year alone. Another 15% is funded by very unstable means, and I fear losing that funding. How long will I have decent collections? Well, I’m not happy with them now in some areas, and it’s getting worse. And I’m in the US. I was also involved for some time with a project called Bioline, which digitizes and distributes research journals from developing countries. There are several projects that make expensive research available to institutions in countries that are very poor. But overall, access is difficult, and it’s becoming more and more difficult even in wealthy countries. Library budgets are contracting, while costs rise.
phil_davis

As a former science librarian, I sympathize with your position. I was involved in three consecutive journal cancellation projects before returning to graduate school.

Yet, promoting open access publishing on the basis of declining library budgets may miss the real reason to support open access publishing. As I report, freely-accessible articles reach a far larger audience of potential readers than subscription-access publishing. It is not necessary to hold steadfast to a conjecture that open access necessarily results in greater citation impact. The real beneficiaries of open access may be those who read, but not necessarily contribute to, the scholarly literature.
jabberwocky12

This is one study, and one study only. For a balanced overview, see a literature review of many studies at: http://www.istl.org/10-winter/article2.html The weight of the evidence shows a citation advantage of OA.

BTW – one of those studies quoted in the literature review is one by Mr Davis himself. And it shows a citation advantage.
Al_de_Baran

” It is my Bible, my Talmud, my Quran. It is everything.”

I am second to none in my disgust over the pending disappearance of the extremely valuable *Statistical Abstract*, but the sort of ridiculous hyperbole quoted above, even if semi-humorous, does neither the author nor like-minded proponents of the volume any favors.