Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: Reproducing results - was managing updates
From
Argyn Kuketayev <[email protected]>
To
[email protected]
Subject
Re: st: Reproducing results - was managing updates
Date
Wed, 2 Mar 2011 11:22:30 -0500
in IT it's a solved problem. first of all, there are build/revision
numbers of any software library or tool. if i download Java VM, it
comes with a build number. the same with any libraries, such as
Commons-Math. second, all custom codes and build scripts are in source
control systems such as SVN. third, we have regression and unit tests.
i can go to SVN, checkout any particular revision and run my test
suite, it will produce exactly the same result every time, because all
components are controlled.
in old times, systems like Smalltalk used to be like Stata, where you
can run something and update the core VM. it's quite hard to control,
you have to keep snapshots of the system all the time. that's why i
was surprised that Stata didn't seem to have a standard revision
control procedures. the re-produceability of results is a big thing to
me. i'm trying to figure out how to do it before we got too deep in
development.
On Wed, Mar 2, 2011 at 5:28 AM, Allan Reese (Cefas)
<[email protected]> wrote:
>
> This appears to connect with a current debate in Science and Nature
> magazines. For a general introduction see blog
> http://wattsupwiththat.com/2011/02/26/the-code-of-nature-making-authors-
> part-with-their-programs/
>
> If you have a command log file started automatically from profile.do,
> keep an output log file of critical runs, and keep versions of data
> files (rather than -replace- them) you are well ahead of most people.
> -update query- would insert in your output the dates of last update.
> Obviously you need to keep any locally-written code files. If an
> analysis run in 2011 gives answer A but "the same" analysis run in 2013
> gives answer B, the issues are (1) is either A or B provably or credibly
> correct, and (2) if A is undermined, is anyone suggesting bad faith
> rather than bad luck?
thanks for the link to a blog
--
Argyn Kuketayev
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/