Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Appending multiple large datasets


From   Sergiy Radyakin <serjradyakin@gmail.com>
To   "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
Subject   Re: st: Appending multiple large datasets
Date   Fri, 7 Jun 2013 16:02:07 -0400

"Windows informs me Stata has stopped working"
this does not mean that Stata has stopped working. This means that
Stata is busy, and failed to respond to Windows' request after a
certain period. Usually this occurs if the program is not redrawing
it's window for a while.

You may want to :
1) extract a few lines from each of your original files into a
separate smaller copies and try appending them for diagnostics;
2) try your script putting the same filename twice (either first or
second) and see whether any or both fail;
3) insert -describe- command after each -insheet- and tell us the results.

Best, Sergiy


On Fri, Jun 7, 2013 at 3:52 PM, Ryan Stevens <stevrl7@gmail.com> wrote:
> No error messages, Windows informs me Stata has stopped working, so it
> could be a problem with Windows. The insheet commands individually
> work, I have checked this. I'll not set the memory myself. And yes,
> Excel reads them fine. I can try importing as XLS to see if that might
> change something. As a side note, I've been able to workaround this by
> merging the files since none of the data is duplicated between my
> files. This does work, but I'm still wanting to figure out what
> exactly is wrong.
>
> Ryan
>
> On Fri, Jun 7, 2013 at 3:44 PM, Sergiy Radyakin <serjradyakin@gmail.com> wrote:
>> So what exactly did not work?
>> Which error message are you getting?And after which line?
>> Insert command count after each insheet. Check how many observations
>> got imported.
>> Do not set memory in Stata 12. It can do it better then the user.
>> Are your CSV files really "comma separated values"? Does Excel read
>> them OK? Try saving from Excel and then importing XLS file into Stata
>> 12.0.
>> Sergiy.
>>
>> On Fri, Jun 7, 2013 at 1:57 PM, Ryan Stevens <stevrl7@gmail.com> wrote:
>>> Hi all,
>>>
>>> I'm experiencing a problem appending 2 datasets containing roughly 1
>>> million observations each. I'm currently running Stata 12.0. My code
>>> can import each dataset fine, but it always gets hung up when I try to
>>> append the two datasets. My code using temporary files is below and is
>>> very simple. I've tried creating physical .dta files and saving them
>>> and that did not work. I can assure you my directories work fine, so
>>> it's no issues with names of my files. I'm wondering if there are any
>>> thoughts on where this problem could lie.
>>>
>>> clear
>>> capture log close
>>> set memory 100m
>>> set more off
>>> macro drop _all
>>>
>>> *Import data
>>> insheet using "consump1.csv"
>>> tempfile master_file
>>> save "`master_file'"
>>>
>>> clear
>>> insheet using "consump2.csv"
>>> append using "`master_file'"
>>>
>>>
>>>
>>> Thanks for any assistance,
>>> Ryan
>>> *
>>> *   For searches and help try:
>>> *   http://www.stata.com/help.cgi?search
>>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>>> *   http://www.ats.ucla.edu/stat/stata/
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>> *   http://www.ats.ucla.edu/stat/stata/
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index