Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: Re: insheet multi threading
From
Nick Cox <[email protected]>
To
[email protected]
Subject
Re: st: Re: insheet multi threading
Date
Mon, 2 May 2011 15:42:59 +0100
If it's so easy, why not write your own?
Nick
On Mon, May 2, 2011 at 2:30 PM, Argyn Kuketayev
<[email protected]> wrote:
> it's a very basic task, and I'm sure everyone does it daily: read CSV files. it takes
> over 1 hour on 8-core machine to read 13GB file, because CPU load is
> 12% all the time, one core is working.
>
> it's a junior programmer level assignment to parallelize the parsing
> part, that's why i'm surprised Stata didn't do it. it's frustrating
> because sometime i get CSVs during the day, and have to wait long long
> time before i can upload them into Stata. once in .dta format, all is
> fast: reading and writing. so, it's clearly parsing part that is slow.
>
> On Mon, May 2, 2011 at 12:24 AM, Joseph Coveney <[email protected]> wrote:
>> Are circumstances such that you can have Stata convert your CSV files to Stata
>> format overnight? I'm assuming that Stata won't spend much time parsing its own
>> file format the next morning when you go to use the datasets.
>>
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/