Not a Lot About Excel and Some Links
Reproducibility of computational experiments hit the headlines the other day. Instead of citing one of the many news posts about it[(e.g. this one) http://www.bbc.co.uk/news/magazine-22223190], let’s go straight to the source:
Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff, by Thomas Herndon, Michael Ash, Robert Pollin, 15 April, 2013.
It’s quite pleasing that this came out the same day as The Recomputation Manifesto: that is the submission date to arXiv was Friday 12th but I think they circulated it on Monday 15th. Also lucky in a sense as it saved me adding to it as a poster-boy example.
It’s also particularly pleasing that this error was uncovered because students were set an exercise to replicate that study. What a wonderful thing, and I wish it was far more common in computational science.
Apart from these short comments there are a couple of reasons I don’t want to say much about this.
First is that Victoria Stodden has blogged excellently on the implications. Second is that, since this is an Excel problem, at the minute this is not something we could accept into recomputation.org, at least accept and provide a full virtual machine. Which is a pity, but the licencing issues are important.
While I’m here I want to mention a couple of new links I have added to the resources page, which I did not know about (but should have) and found from Victoria’s blog post.
Setting the Default to Reproducible, Feb 2013, Report on the
ICERM Workshop on Reproducibility in Computational and Experimental Mathematics, December 2012. Very useful report with an Appendix full of advice and tool descriptions.
Reprozip. A tool similar to CDE but focussed specifically on allowing reproduction of comptutational experiments.