st: RE: automate selection of variables and observations from multiple CSV

["David Radwin" <dradwin@mprinc.com>]

You can use a combination of for* loops in this fashion. You may also want
to -rename- some variables to include their month and year.

********************
* input each CSV, select variables and observations, then save in Stata
format

forvalues y = 1999/2008 {
	foreach m in 01 02 03 04 05 06 07 08 09 10 11 12 {
		insheet using `y'`m'.txt, comma clear
		[keep variables and observations you want]
		save `y'`m'
		local filenames `filenames' `y'`m'
		}
	}

* starting with blank dataset, append Stata .dta files together

clear
append using `filenames'
*********************

For more help with loops, see one or more of these:

http://www.stata-journal.com/article.html?article=pr0005
http://fmwww.bc.edu/EC-P/WP612.pdf
http://www.stata.com/bookstore/isp.html

David
--
David Radwin
Research Associate
MPR Associates, Inc.
2150 Shattuck Ave., Suite 800
Berkeley, CA 94704
Phone: 510-849-4942
Fax: 510-849-0794

www.mprinc.com


> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-
> statalist@hsphsun2.harvard.edu] On Behalf Of Argyn Kuketayev
> Sent: Monday, March 14, 2011 11:27 AM
> To: statalist@hsphsun2.harvard.edu
> Subject: st: automate selection of variables and observations from
> multiple CSV
> 
> Hello
> 
> I have a bunch of CSV files like YYYYMM.txt, they contain monthly
> observations of some entities. I want to create a subset of entity
> ids, then create a huge data set which consists of the monthly
> observations of this subset of entities and subset of variables.
> 
> Each monthly file fits into memory, but all of them will not.
> 
> So at the moment I know how to use insheet to get one month in memory.
> Then I could drop unnecessary variables. Then somehow I should drop
> unnecessary entities. Or even better not load them with insheet.
> 
> Then I should do the same with all other months, and somehow append
them.
> 
> can you suggest me which STATA commands could be used for this?
> 
> cheers
> --
> Argyn Kuketayev
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/