RE: st: looping through data files in do-file

[Nick Cox <n.j.cox@durham.ac.uk>]

Stas' closing comments are especially good. Also check out -fs- from SSC. 

Nick 
n.j.cox@durham.ac.uk 

Stas Kolenikov

On Thu, Oct 21, 2010 at 9:55 AM, Broudy, David, DOH
<David.Broudy1@state.nm.us> wrote:
> I'm trying to create a do file that will open a monthly data file, run a
> couple of commands to create a variable, select and keep records, save a
> new file and go to the next month.  Later these monthly abstract files
> will be merged into an annual file.
>
>  I'm having trouble building the string with the path and file names.
> Each month's data differs only in a two character number that represents
> the calendar month  01 is Jan etc
>
> The new file will be named the same as the input file but with _R13
> inserted.
>
>
> version 9
>
> local year "_2009"
>
> global filename "CPT_"
>
> global suffix ".dta"
>
> global filepath "i:\My_Docs_on_I\BEHRdata\CPT_2009\"
>
> global mod  "_R13"
>
> foreach month in "02"  {
>
> display "$filepath""CPT_""`month'""_2009"".dta"
>
> use " "$filepath""CPT_""`month'""_2009"".dta" ",clear
>
>
>
> This seems to work and displays a correct path and filename.
>
> But when I put the same string in a use statement it gives an error.
> I've tried a lot of combinations of single and double quotes-see run
> below.
>
>  . do "C:\DOCUME~1\DAVID~1.BRO\LOCALS~1\Temp\STD05000000.tmp"
>
> . version 9
>
>  . local year "_2009"
>
>  . global filename "CPT_"
>
>  . global suffix ".dta"
>
> . global filepath "i:\My_Docs_on_I\BEHRdata\CPT_2009\"
>
> . global mod  "_R13"
>
> . foreach month in "02"  {
>
>  2. display "$filepath""CPT_""`month'""_2009"".dta"
>
>  3. use " "$filepath""CPT_""`month'""_2009"".dta" ",clear
>
>
>
> . }
>
> i:\My_Docs_on_I\BEHRdata\CPT_2009\CPT_02_2009.dta   [this is the string
> displayed]
>
> invalid 'i'    [this apparently is an error trying to execute use]

The -use- becomes:

use " "i:\whatever""CPT_""02""_2009"".dta" "

Stata reads: use the file with the name " ". Aha, I can do that, but
let me read further as there might be some option that I would need to
process. Wait, what's all this crap that starts with letter i
following the filename?

Also, if Stata were a compiled language, it would say, "You defined a
bunch of globals and never used them". Don't mess up with so many
globals, especially if you are only using them in one do-file, and
they are a matter of convenience in file naming. Do you expect to have
any suffices other than .dta? Do you expect to have any mods other
than _R13? Do you expect to have any other paths rather than the one
you spelled out? Are you going to call any other files that are going
to use these globals? If the answer to all these questions is "No",
get rid of the globals, or at least make them locals that tend to be
easier to parse. There is always a clear ending with the locals, while
the globals may need an additional {}, as Maarten Buis suggested.

What I would probably do is

cd [to whatever dir]
local allfiles : dir . files CPT_*_2009.dta
foreach f in `allfiles' {
   use `f', clear
   SomeDataProcessingModule
   save `f'_R13, replace
}


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/