Estimating open access Mandate effectiveness : the MELIBEA score

Abstract

MELIBEA is a directory of institutional open‐access policies for research output that uses a composite formula with eight weighted conditions to estimate the “strength” of open access (OA) mandates (registered in ROARMAP). We analyzed total Web of Science‐(WoS)‐indexed publication output in years 2011–2013 for 67 institutions in which OA was mandated to estimate the mandates' effectiveness: How well did the MELIBEA score and its individual conditions predict what percentage of the WoS‐indexed articles is actually deposited in each institution's OA repository, and when? We found a small but significant positive correlation (0.18) between the MELIBEA “strength” score and deposit percentage. For three of the eight MELIBEA conditions (deposit timing, internal use, and opt‐outs), one value of each was strongly associated with deposit percentage or latency ([a] immediate deposit required; [b] deposit required for performance evaluation; [c] unconditional opt‐out allowed for the OA requirement but no opt‐out for deposit requirement). When we updated the initial values and weights of the MELIBEA formula to reflect the empirical association we had found, the score's predictive power for mandate effectiveness doubled (0.36). There are not yet enough OA mandates to test further mandate conditions that might contribute to mandate effectiveness, but the present findings already suggest that it would be productive for existing and future mandates to adopt the three identified conditions so as to maximize their effectiveness, and thereby the growth of OA.