Saturday, May 23, 2009

Open Access mandates: Judging success

As Alma Swan has graphically demonstrated (here and here), mandates have begun to propagate nicely.

It is worth noting that many of the new ones are being introduced by faculty themselves, rather than by administrators imposing mandates on them from above. It is also noteworthy that many of the mandates in the recent upsurge have been introduced by library faculties.

But what level of compliance can we expect from these mandates? After all, a mandate is only as good as the compliance rate it achieves. And how do we judge success so far as compliance is concerned anyway?

Arthur Sale’s analyses of the effect of mandates on Australian researchers suggest that a high level of compliance with a mandate can be achieved within two years. (Sale appears to have judged success as being a compliance rate of 70%)

Perhaps the most controversial and hard-won mandate was the one introduced at the US National Institutes of Health (NIH) in May 2005. Initially this was a request.

By November 2005 it was reported that fewer than 5% of NIH grantees were complying with the request. As a result, last year the mandate was upgraded to a requirement.

The new policy came into effect in April 2008. Since then, an NIH spokesperson tells me, “Compliance has increased almost 250% ... It has jumped from 19% of our target estimate 80,000 papers per year arising from NIH funds during the voluntary policy to almost half (49%) of the target estimate of papers arising from NIH funds at the end of 2008.”

And compliance, he added, continues to improve. “In January and February 2009 we collected over 3 times as many manuscripts as we did in January and February 2008, before the requirement took effect, and March and April numbers appear even higher.”

A similar increase in deposits has been experienced at the University of Stirling following the introduction of a mandatory self-archiving policy by its Academic Council last year (which came into force last September).

Last week it was announced that deposit rates in the University’s repository (STORRE) have grown from 20 a month to 120 a month, and STORRE now hosts 1,000 papers, reports and book chapters.

Commenting on the JISC-REPOSITORIES mailing list, the University of Stirling’s eLearning Developer Michael White said, “Whilst we are aware that we don't yet have 100% compliance with our mandate, the key point is that we only managed to get 63 journal articles over the 3 years prior to the announcement of the mandate, but have got 687 items (excluding eTheses) in the year since (with the vast majority coming in after the mandate came into force in September).”

A number of questions naturally arise:

— Can we expect the NIH compliance rate to match the levels reported by Sale in Australia by next April (i.e. two years after it became mandatory)?

— Can we expect the surge of new mandates to achieve the same levels of compliance reported by Sale?

—  Is there any significance in the fact that many of the new mandates are being introduced by library faculties, and can we expect that to affect compliance rates?

— Will the fact that many of the new mandates are self-imposed affect compliance rates? (Will it make them appear more voluntary than mandatory)?

— Will the fact that many of the new mandates include opt-outs affect compliance rates? (Will that make them appear more voluntary than mandatory?)

— What is full compliance so far as a self-archiving mandate is concerned? (Is Sale’s 70% level the objective, or should the research community be aiming higher?)

What other questions should we be asking, particularly when trying to judge the success of a mandate?

All comments welcome!