Back

The AEA Data and Code Availability does not just require that authors publish a replication package - it also encourages authors to improve upon the replication package. This thread is about an example.

In a nutshell, when authors, or replicators, or somebody on the internet, identifies an issue with a package, they have a few options to identify and publish a solution for it, so that others can benefit the update:

- submit a comment to the journal
- write a new article
- record the issue on platforms such as the Social Science Reproduction Platform https://www.socialsciencereproduction.org/ or Replication Wiki https://replication.uni-goettingen.de/
- post a revision of replication package according to the AEA Revision Policy.

This is about the latter.

Timeline:
Oct 2019: Replication package submitted to Data Editor (one of the first 200 cases). 1st report sent back to the author, author revised package, re-submitted improved package.

Nov 2019: The revised package was accepted (after 1 round). We were unable to access all data (this is important!), but improved the documentation of the data sources.

April 2020: Publication of the manuscript: https://doi.org/10.1257/aer.20171732, and of the replication package: https://doi.org/10.3886/E112121V1.

Sometime in 2020: a replicator, with access to the data, reports to the author various small issues, in particular missing files.

Jan 2021: Author reaches out to AEA Data Editor, and wants to correct the data deposit, so that future replicators can access the missing files. A case is created by the AEA Data Editor to follow up.

Life intrudes

Oct 2022: Author updates the replication package, together with a Changelog (as required by policy), creating a V2.

The revised package is published: https://doi.org/10.3886/E112121V2, and the V1 now has a banner identifying that a newer version is available.

Notes and links: See https://aeadataeditor.github.io/posts/2022-11-09-updating-post-publication

Conclusion: This is a great example of how the ability to update replication packages is a key part, previously underexploited and not transparent, in improving science.

Happens multiple times per year, not always for this (positive) reason.

Revisions and new versions can be created on all regularly used repositories, such as openICPSR, Zenodo, and Dataverse.

When authors publish their replication packages prior to submission, such revisions allow them to incorporate the changes made during the editorial process, whether related to the scientific or the computational content of the paper. End

(I hope these all cross-posted to the birdsite, if not, find it https://aeadataeditor.github.io/posts/2022-11-09-updating-post-publication and https://mstdn.social/@aeadata/109315024319500617

reverse tagging: #econtwitter #econtoots #reproducibility #transparency #aea