Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: st: RE: How to adjust the content of a local macro?


From   "Martin Weiss" <martin.weiss1@gmx.de>
To   <statalist@hsphsun2.harvard.edu>
Subject   RE: st: RE: How to adjust the content of a local macro?
Date   Wed, 9 Dec 2009 23:49:11 +0100

<>

" I will have the ?meanonly- option in mind. Did not know about it. :)"


Re -summ, meanonly-, you may also want to read NJC`s
http://www.stata-journal.com/article.html?article=st0135




" I am indeed confused with the Stata usage of ` ' and have been having  
problems with this. minDate is a ?tempname- so if I just write  
`minDate' I don't get the value, just the tempname. Doing it as  
``minDate'' provides me with the value. But I am sure there are  
smarter ways to do this. . ."


Why do you need -tempname-s here, anyway? See the example where I can
reassign the minimum and maximum repeatedly w/o -tempname-s:



***
forv i =1/5{
capt drop myvar
set obs 10
gen myvar=runiform()
su myvar, mean
local min=r(min)
local max=r(max)
di in r "min: `min', max: `max'"
}
***



HTH
Martin

-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu
[mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Joachim
Landström
Sent: Mittwoch, 9. Dezember 2009 23:28
To: statalist@hsphsun2.harvard.edu
Subject: Re: st: RE: How to adjust the content of a local macro?

I am amazed with the level of help that you get from here. Let me  
start off expressing my gratitude for this.

To answer some of Nick's points:

> 1. The loop consists of repeated -drop-ping of observations not   
> desired, working with the remaining subset and then a -restore- of   
> the original. It is difficult to say in general what is most   
> efficient and what most elegant but for a situation like that below   
> I'd normally just add an extra condition excluding the observations   
> not wanted, rather than repeatedly doing major surgery on the   
> dataset. However, others could equally point out that applying -if-   
> on a very large dataset can be time-consuming.

Answer #1: I have been using ?preserve- & -restore, preserve- in loops  
because of system limitations. I have a dataset as below

Contains data from us_data_ret.dta
   obs:    17,516,468
  vars:             5                          10 Nov 2009 07:02
  size:   525,494,040 (56.6% of memory free)
------------------------------------------------------------
               storage  display     value
variable name   type   format      label      variable label
-------------------------------------------------------------
id              int    %8.0g                  ID
dscd            str6   %9s                    DSCD
date            float  %td
year            int    %8.0g
totalReturn     double %10.0g
-----------------------------------------------------------
Sorted by:  id  date

that I join with annual market data as below:


Contains data from C:\Users\Joachim\AppData\Local\Temp\ST_02000005.tmp
   obs:         1,461                          Total returns for  
value-weighted ave S/B & H/L BM portf and their diffs. See Fam
  vars:             5                          9 Dec 2009 23:08
  size:        46,752 (99.9% of memory free)
----------------------------------------------------------------------------
----------------------------
               storage  display     value
variable name   type   format      label      variable label
----------------------------------------------------------------------------
----------------------------
date            float  %td
SMBtr           float  %9.0g                  Small-Minus-Big Total  
Returns. See Fama & French (1993).
HMLtr           float  %9.0g                  High-Minus-Low Total  
Returns. See Fama & French (1993).
riskFreereturn  float  %9.0g                  US T-Bill 3 month, middle rate
marketReturn    double %9.0g                  Frank Russel 3000, Total  
Return Index
----------------------------------------------------------------------------
---------------------------
Sorted by:

The first dataset above is in "long format" and has 10826 unique ids  
(US firms). The date variable ranges from 03Jan1978 to 30dec2008 with  
weekly stock market data.

Then I wish to do a Fama & French 3F CAPM calculation per id and then,  
based on the 3F CAPM I wish to calculate the expected annual return  
for each firm-year when each firm is traded on the stock market. Now I  
know I may be using too much lingo, but this boils down to an OLS  
regression per id provided that enough data is available (for me that  
implies three years of weekly data).

Since I do not have enough computing power/memory (I am using a  
Windows 32-bit system) to add more variables into the joined dataset I  
have resorted use the ?preserve- & -restore, preserve- method to drop  
all but a single id and then do the necessary operations on that  
subset. Then I save the data into a tempfile. Later (after 6-7 hours),  
when the loop is done I use ?append- to build the dataset anew (having  
dropped unnecessary data).

Any suggestions to more elegant methods are greatly appreciated.

> 2. If only the minimum and maximum are needed from a -summarize- it   
> is best just to use a -meanonly- option. (The name -meanonly- is   
> misleading, as I've had occasion to remark before.)

I will have the ?meanonly- option in mind. Did not know about it. :)

> 3. Code like
>
> 	local `minDate' = r(min)
> 	<stuff> if <stuff> date >= ``minDate''
>
> looks legal but odd. You are probably using more levels of macros   
> than you need. It's hard to tell because the code isn't completely   
> self-contained (that's not a criticism; it wasn't necessary for your  
>  question).

I am indeed confused with the Stata usage of ` ' and have been having  
problems with this. minDate is a ?tempname- so if I just write  
`minDate' I don't get the value, just the tempname. Doing it as  
``minDate'' provides me with the value. But I am sure there are  
smarter ways to do this. . .

> 4. Code in which you loop over the contents of a local macro and   
> change that macro within the loop can be tricky. Watch out!

My computer at the university is running a test this evening with the  
revised code and I was intuitively also worried to change ?panelVar-  
inside the loop which it guides. I have done it as Martin proposed and  
added a ?adjustedPanelVar- that I modify. I need it for a second loop  
where I append the dataset once the first loop is done.

> 5. The -if- condition in
>
> 	summarize totalReturn if totalReturn != .
>
> is unnecessary as -summarize- always ignores missings.

That is always good to know.

> 6. To get minimum and maximum dates in a panel, no looping is necessary as
>
> egen mindate = min(date), by(id)
> egen maxdate = max(date), by(id)
>
> will do it. Similarly it looks as if your main problem does not need  
>  any looping either, as it should yield to -egen- operations. Look  
> at  -egen, count()- in particular.

Yes I have used ?min()-elsewhere  but I cannot easily add more  
variables to the complete database due to memory limitations. Memory  
is a luxury to me. . .

> 7. More generally, it is not always positive to know too many other   
> languages if they lead you to seek a Stata equivalent of other code   
> when there's a Stataish way to do it without any real programming.

I completely agree. Coming from SAS I am used to ?proc sql- and SAS'  
-proc iml-. Had I just known about STATA 1998 this would not have been  
a problem. Now I have to "delearn" the SAS version of sql while  
learning STATA and that is like a baby learning to walk. I try my best  
but keep bruising myself. . . :)

I will respond to Martin's second post tomorrow by posting the code  
and an an excerpt of the data (beware that it looks much like crochet  
since I have mainly been trying to get the code to run without exiting  
with errors).

/Joachim


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index