Re: st: RE: Append multiple files from .txt file with "file read"

[Sergiy Radyakin <serjradyakin@gmail.com>]

On Thu, Dec 5, 2013 at 7:36 PM, Nicole Boyle <nicboyle@gmail.com> wrote:
> Hi David,
>
> Thanks for your reply. Your rather elegant solution worked perfectly
> for appending the datasets. However, the resulting data are not
> matched 1:1 on ID, an issue which is my fault: It seems that I
> incorrectly chose -append- for adding data horizontally on a 1:1 ID
> match when -merge- is probably the better solution.
>
> Looking at the help files, -merge- doesn't seem to be able to handle
> multiple files like -append- can. Nor can -append- match 1:1 like
> -merge- can. (Please correct me if I'm wrong)
>
> Is there a similar solution to the one you presented, but with -merge-?

this will stitch several files together, adding columns from each
subsequent file, all saved files must be sorted by ID. You must start
with IDs in memory.

local flist `"`:dir . files "*.dta"'"'
foreach f of local flist {
  merge ID using `"`f'"'
  assert _merge==3
  drop _merge
  sort ID
}

Best,Sergiy



>
> Nicole
>
>
> On Thu, Dec 5, 2013 at 3:52 PM, Radwin, David <dradwin@rti.org> wrote:
>> Nicole,
>>
>> The -append- command lets you add multiple files at once without the
>> need for loops, ancillary text files, etc. Why not just make a local
>> macro of all the Stata files in your directory and then -append- them
>> all at once?
>>
>>
>> local f : dir . files "*.dta"
>> display as result `"`f'"'
>> append using `f'
>>
>>
>> David
>> --
>> David Radwin, Senior Research Associate
>> Education and Workforce Development
>> RTI International
>> 2150 Shattuck Ave. Suite 800, Berkeley, CA 94704
>> Phone: 510-665-8274
>>
>> www.rti.org/education
>>
>>
>>> -----Original Message-----
>>> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-
>>> statalist@hsphsun2.harvard.edu] On Behalf Of Nicole Boyle
>>> Sent: Thursday, December 05, 2013 3:30 PM
>>> To: statalist@hsphsun2.harvard.edu
>>> Subject: st: Append multiple files from .txt file with "file read"
>>>
>>> Hello all,
>>>
>>> First and foremost, I have yet to fully understand how to use macros,
>>> so please forgive me if the solution to this problem is painfully
>>> obvious. I actually hope it's painfully obvious.
>>>
>>> I'm trying to combine multiple .dta files (1:1 horizontally appended)
>>> by calling several .dta filenames stored in a .txt file. However, in
>>> the process of doing this, whenever I try to run:
>>>
>>> .    use `line'
>>>
>>> Stata returns the error:
>>>
>>> .    invalid file specification
>>>
>>>
>>> Here's the code I'm trying to execute (sourced from here*). To start,
>>> I'm trying to execute this code on a .txt file containing just two
>>> lines (aka: two .dta filenames), but the final file will have 25
>>> lines:
>>>
>>>    pwd
>>>    cd ~/Desktop/merge
>>>    ! ls *.dta >filelist.txt
>>>    file open myfile using "filelist.txt", read
>>>    file read myfile line
>>>    use `line'  /* ERROR HERE */
>>>    save master_data, replace
>>>    file read myfile line
>>>    while r(eof)==0 {
>>>    append using `line'
>>>    file read myfile line
>>>    }
>>>    file close myfile
>>>    save master_data, replace
>>>
>>>
>>> I first thought the problem was that "filelist.txt" wasn't being read.
>>> However, I believe it IS being read, since running the following:
>>>
>>>    ! ls *.dta >filelist.txt
>>>    file open myfile using "filelist.txt", read
>>>    file read myfile line
>>>    while r(eof)==0 {
>>>    display "`=word("`line'",1)'"
>>>     file read myfile line
>>>     }
>>>
>>> only displays the second (but not the first) line of the two-line .txt
>>> file.
>>>
>>> Perhaps my issue has something to do with Stata overlooking the first
>>> line of the .txt file? Or perhaps my general macro-incompetence (more
>>> likely)?
>>>
>>> Any help will be greatly appreciated. Thanks so much for your
>>> consideration.
>>>
>>> Nicole
>>>
>>>
>>> *Code from
>> http://www.ats.ucla.edu/stat/stata/faq/append_many_files.htm
>>
>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>> *   http://www.ats.ucla.edu/stat/stata/
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/