Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: st: How to reference results from a big dataset within a program
From
"Chen,Minxing" <[email protected]>
To
"'[email protected]'" <[email protected]>
Subject
RE: st: How to reference results from a big dataset within a program
Date
Wed, 28 Aug 2013 16:46:07 +0000
Hi Joerg, the "pre-simulated dataset" I mentioned early is a really long list of results I want from simulated joint binary probabilities. The reason I didn't want to do this simulation within my program is it will take significantly long time (e.g. many hours). Instead of doing the simulation within my program and get the result I need for next step, I thought referencing the result from the data I included with the program will be much easier. Minxing
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Joerg Luedicke
Sent: Tuesday, August 27, 2013 4:46 PM
To: [email protected]
Subject: Re: st: How to reference results from a big dataset within a program
What do you mean with "pre-simulated dataset"? Is it _one_ fabricated dataset, or results from 400k replications of a Monte Carlo simulation?
Either way, it seems a bit puzzling why you would even need to include a fixed set of data in your program? What information exactly are you pulling from this dataset? Depending on the specific purpose of including these data in your program, maybe you could derive some analytic solution?
Joerg
On Tue, Aug 27, 2013 at 5:25 PM, Chen,Minxing <[email protected]> wrote:
> Dear all,
>
> I submitted a manuscript and program to Stat Journal several months ago, and now got feedback from reviewer. I need a big help from your all regarding how to reference a big dataset within a program.
>
> Basically, in the program I submitted, I had to reference results from a big pre-simulated dataset (four variables, but around 400,000 observations). In my previous submission, I simply submitted the pre-simulated dataset with my program, and within the program I called up that simulated dataset by using code such as " use c:\ado\personal\simudata". I was hoping when people download the program from SSC, the pre-simulated dataset will be also downloaded to the directory "c:\ado\personal\".
>
> Now my reviewer indicated that I can't expect users to do that, I can't even tell the user to place the file there because such a directory may not be creatable for the user (e.g. they might not have a C: drive). The reviewer suggested me to find some other way to get the information in my pre-simulated dataset, such as incorporating the data into the program.
>
> I tried to copy of the simulated data within my program by using syntax such as "input x y z k", however, since there are so many observations (a little more than 400,000), and there are system limit for the maximum lines of syntax within a program (around 3500), I was not able to do this way. The reviewer also mentioned that I may use "Mata library" function, but I am pretty new to Stata Mata. Is there anyone that may be able to help regarding this issue?
>
> Thank you so much!
> Minxing Chen
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/faqs/resources/statalist-faq/
> * http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/faqs/resources/statalist-faq/
* http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/faqs/resources/statalist-faq/
* http://www.ats.ucla.edu/stat/stata/