Open preprint reviews by Bernd Pulverer

Modeling Science Trustworthiness Under Publish Or Perish Pressure

David Robert Grimes, Chris T. Bauch, John P. A. Ioannidis

Review posted on 19th May 2017

Good to see basic modelling of the scientific rewards system is consistent with the intuitive conclusion that ‘if publishing is the sole criteria under which academics are judged, then dubious conduct can thrive’ - along the lines articulated by DORA [http://www.ascb.org/dora/].

Would be important to see in how this theoretical model maps onto real world data.

The crux seems to be, as so often, carrot vs. stick: to balance the incentives for diligence with the apparent need for some penalty for bad conduct.
Currently, if a young researcher obtains a result s/he knows is unreliable or even flawed, but that may further their career if published, they can decide not to publish and forfeit their grant/tenure/career or publish and risk later retraction which might lead to the same outcome. Can we expect this researcher resist the publish or perish pressures?

On the other hand, as discussed at the METRICS workshop (https://metrics.stanford.ed..., we want to encourage self correction and penalties may counteract that.
Can the model help inform what sort of penalties are constructive?

Two minor comments:
1) ’top-tier journals possess a limited number of publication slots’ [p. 2]: top-tier journals have in principle infinite space online and especially OA ones would actually benefit financially from publishing more, but they decide not to publish more as selectivity is seen as a valuable service to the community.
2) ‘rewarding diligence improves the proportion of funding allocated to diligent groups’ [p. 13]: seems to be a little circular

show less