st: RE: Need help with Programming/ running a forvalues loop inside foreach loop

[Joe Canner <jcanner1@jhmi.edu>]

Mike,

I don't see anything particularly wrong with the -forvalues- loop, but I think the -use- statement should also have a -using-:

use `x' var5 var6 var7 using "$data\dataset_1",clear

That aside, what specific symptoms are you seeing when you do the -forvalues- loop?  Are you getting errors?  What is in memory when it finishes?

Regards,
Joe Canner
Johns Hopkins University School of Medicine

-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Michael Stewart
Sent: Tuesday, November 05, 2013 3:42 PM
To: statalist
Subject: st: Need help with Programming/ running a forvalues loop inside foreach loop

Dear Statalist member,

I am writing to seek help with stata programming and I am dealing with
rather large datasets and am resorting to loading relevant variables
of the dataset for analysis secondary to memory constraint.

I especialy need help running a  forvalues loop inside a foreach loop .

Details are as follows :
*************************
My data is in a folder "Z:data"

My files are named dataset_1 to dataset_10 (denoting years 2001 till
2010) and each file is in the order of  gigabytes with 40 variables
each .

For my analysis I have to use (either var1 OR var2 OR var3) along with
(var5+var6+var7) from each dataset.

As it is a multiyear analysis, I need to combine (either var1 OR var2
OR var3) along with (var5+var6+var7) from each dataset(dataset_1 to
dataset_10) and analyze.


My  dofiles is as follows

*******************************
clear
macro drop _all
global data "Z:data"

foreach x in  var1 var2 var3 {
use `x' var5 var6 var7 "$data\dataset_1",clear
forvalues i=2/10{
append using "$data\dataset_`i'",keep(`x' var5 var6 var7  )

}
svyset [pweight= var5],strata  (var6 ) psu ( var7 )
svy:mean `x'

}
*************************************************************

QUESTION:  How can I make the forvalues loop  merge only relevant
variables from each dataset into one big dataset  before proceeding to
the next step so that I will have multi year dataset for analysis at
subsequent steps

To further clarify the question:

After the foreach loop  takes var1 + var 5-7 from dataset_1; I would
like the forvalues loop to take var1 + var 5-7 from dataset_2/10 ,
merge them and procedd further statistical analysis.

In next cycle, the foreach loop will take var2 + var 5-7 from
dataset_1; I would like the forvalues loop to take var2 + var 5-7 from
dataset_2/10 , merge them and procedd further statistical analysis.

But with my loop, the forvalues loop is not merging multipe
files(dataset_2 to dataset_10) before proceeding for furtehr analysis.

Any suggestions/recommendations are welcome

Thank you very much for your time.

-- 
Thank you ,
Yours Sincerely,
Mike.
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/