Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

st: cycling through individual indices


From   Johannes Geyer <JGeyer@diw.de>
To   statalist@hsphsun2.harvard.edu
Subject   st: cycling through individual indices
Date   Wed, 17 Sep 2008 18:34:30 +0200

Dear Statalisters,

I have Stata 10.1 IC and I try to create individual specific sums in a 
large dataset. The problem is a bit complicated and I have to cycle 
through all individuals and variables using the "in" qualifier. I am 
curious if anyone has an idea how to solve this problem more efficiently. 
Here is the problem:

The data are in wide format and look like

ID      Agemonth        Var_1   Var_2...                ...Var_623 Var_624
1       532             2       2                       14      14
2       345             7       7                       14      Mis
3       236             3       3                       Mis     Mis
4       267             2       2                       12      12

and so forth; there are about 50,000 observations. "Agemonth" indicates 
the observation period which is individual specific: "1" means January of 
the year after the person turned 14, "2" is February and so forth. That 
means e.g. "ID" 1 was observed 532 months after the year he/she turned 14. 
The index of the variables indicate the same time index. Thus, person 1 
was observed from Var_1 until Var_532. Unfortunately, that does not mean 
that Var_533 or even Var_623 is missing but it may have a value like in 
the example above.

Var_# has a number of distinct values and I need to sum them up in each 
case. If I had no invalid observations I could type

egen sum1 = anycount(Var_*), values(1)

However, then I count also invalid observations.

I ended up with looping through individuals (~50,000) and variables (624), 
summing up one by one but I really doubt that this is the "best" solution 
(and hope that it is not):

*******************
#d;

gen sum1 = 0;
sort ID;
gen index = _n;
qui sum index;

forvalues indis = `r(min)'/`r(max)' {;

        di "`indis'";

        forvalues f = 1/624 {;

                if `f' <=Agemonth in `indis' {;
                        qui replace sum1 = sum1 + (Var_`f' == 1) in 
`indis';
                        };

                };
};
*******************

Another possibilty would be to have the data in long format - however, 
since I have so many periods it takes a while to reshape the data, even in 
portions. I tried that with a 10% sample and "reshape" took more than one 
hour (maybe I have to ask for a better computer...). 

Any help would be appreciated!

Thank you,

Johannes


----------------------
Johannes Geyer
Deutsches Institut für Wirtschaftsforschung (DIW Berlin)
German Institute for Economic Research 
Department of Public Economics
DIW Berlin
Mohrenstraße 58
10117 Berlin
Tel: +49-30-89789-258

owner-statalist@hsphsun2.harvard.edu schrieb am 17/09/2008 16:36:14:

> Bill Gould has a Mata matters column out real soon now in Stata Journal
> 8(3) 2008 that should help here. 
> 
> Nick 
> n.j.cox@durham.ac.uk 
> 
> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu
> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Stas
> Kolenikov
> Sent: 17 September 2008 15:22
> To: statalist@hsphsun2.harvard.edu
> Subject: st: analogues of macros in Mata?
> 
> Suppose I form say a filename using some string operations, something
> like
> 
> myfilename = prefix + "-" + suffix
> 
> and then want to save something along the lines of
> 
> mata matsave {myfilename} Matrix1 Matrix2 Matrix3
> 
> Is there any way to have Mata go into the contents of myfilename
> object there, rather than create the file "myfilename.mmat" in the
> current directory? Or may be explicit version
> 
> mata matsave prefix+"-"+suffix Matrix1 Matrix2 Matrix3
> 
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index