RE: st: How to reference results from a big dataset within a program

["Chen,Minxing" <MChen3@mdanderson.org>]

Thank you so much Maarten, I will definitely take a look at the Mata function and weight which method (other methods suggested by our other people) would be more fit my need. Minxing

-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Maarten Buis
Sent: Wednesday, August 28, 2013 3:59 AM
To: statalist@hsphsun2.harvard.edu
Subject: Re: st: How to reference results from a big dataset within a program

I think the Mata route would probably be a lot safer. Below is an example of how to write such a Mata function and you can look at -help
m1_ado- to see how and where to store such a Mata function.

*------------------ begin example ------------------ clear all

// create a Mata function containing your sumulated data // In addition I let it return a specific value and return // that as a Stata scalar, but you can do other stuff with it mata void refmatrix(real scalar row, real scalar col, string scalar name) {
    real matrix ref
    ref = 1, 2 \
          3, 4
    st_numscalar(name, ref[row,col])
}
end

// in the Stata program I would use this Mata function as follows:
tempname value
local r = 1
local c = 2
mata: refmatrix(`r',`c',"`value'")
di `value'
*------------------- end example -------------------
* (For more on examples I sent to the Statalist see:
* http://www.maartenbuis.nl/example_faq )

Hope this helps,
Maarten


On Wed, Aug 28, 2013 at 1:15 AM, Richard Williams <richardwilliams.ndu@gmail.com> wrote:
> I think you could tell people that simudata must be placed somewhere 
> along Stata's adopath. Then the program could say -sysuse simudata-. 
> My guess is that simudata.dta would have to be specified as an 
> ancillary file in the installation package, which means that the user 
> will have to do a little more work than usual to get the package installed.
>
> Alternatively, how were the simulations done? If with drawnorm or 
> something like that maybe you could include code that would recreate the data.
>
>
> At 04:25 PM 8/27/2013, Chen,Minxing wrote:
>>
>> Dear all,
>>
>> I submitted a manuscript and program to Stat Journal several months 
>> ago, and now got feedback from reviewer. I need a big help from your 
>> all regarding how to reference a big dataset within a program.
>>
>> Basically, in the program I submitted, I had to reference results 
>> from a big pre-simulated dataset (four variables, but around 400,000 observations).
>> In my previous submission, I simply submitted the pre-simulated 
>> dataset with my program, and within the program I called up that 
>> simulated dataset by using code such as " use 
>> c:\ado\personal\simudata". I was hoping when people download the 
>> program from SSC, the pre-simulated dataset will be also downloaded to the directory "c:\ado\personal\".
>>
>> Now my reviewer indicated that I can't expect users to do that, I 
>> can't even tell the user to place the file there because such a 
>> directory may not be creatable for the user (e.g. they might not have 
>> a C: drive). The reviewer suggested me to find some other way to get 
>> the information in my pre-simulated dataset, such as incorporating the data into the program.
>>
>> I tried to copy of the simulated data within my program by using 
>> syntax such as "input x y z k", however, since there are so many 
>> observations (a little more than 400,000), and there are system limit 
>> for the maximum lines of syntax within a program (around 3500), I was 
>> not able to do this way. The reviewer also mentioned that I may use 
>> "Mata library" function, but I am pretty new to Stata Mata. Is there 
>> anyone that may be able to help regarding this issue?
>>
>> Thank you so much!
>> Minxing Chen
>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>> *   http://www.ats.ucla.edu/stat/stata/
>
>
> -------------------------------------------
> Richard Williams, Notre Dame Dept of Sociology
> OFFICE: (574)631-6668, (574)631-6463
> HOME:   (574)289-5227
> EMAIL:  Richard.A.Williams.5@ND.Edu
> WWW:    http://www.nd.edu/~rwilliam
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/



--
---------------------------------
Maarten L. Buis
WZB
Reichpietschufer 50
10785 Berlin
Germany

http://www.maartenbuis.nl
---------------------------------
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/