Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: st: Re: insheet multi threading
From
"Schaffer, Mark E" <[email protected]>
To
<[email protected]>
Subject
RE: st: Re: insheet multi threading
Date
Mon, 2 May 2011 16:08:03 +0100
Because -insheet- and -infile- are built-in and not ados, and only StataCorp could parallelize them? Or am I missing something?
--Mark
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of Nick Cox
> Sent: 02 May 2011 15:43
> To: [email protected]
> Subject: Re: st: Re: insheet multi threading
>
> If it's so easy, why not write your own?
>
> Nick
>
> On Mon, May 2, 2011 at 2:30 PM, Argyn Kuketayev
> <[email protected]> wrote:
> > it's a very basic task, and I'm sure everyone does it
> daily: read CSV
> > files. it takes over 1 hour on 8-core machine to read 13GB file,
> > because CPU load is 12% all the time, one core is working.
> >
> > it's a junior programmer level assignment to parallelize
> the parsing
> > part, that's why i'm surprised Stata didn't do it. it's frustrating
> > because sometime i get CSVs during the day, and have to
> wait long long
> > time before i can upload them into Stata. once in .dta
> format, all is
> > fast: reading and writing. so, it's clearly parsing part
> that is slow.
> >
> > On Mon, May 2, 2011 at 12:24 AM, Joseph Coveney
> <[email protected]> wrote:
> >> Are circumstances such that you can have Stata convert
> your CSV files
> >> to Stata format overnight? I'm assuming that Stata won't
> spend much
> >> time parsing its own file format the next morning when you
> go to use the datasets.
> >>
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
--
Heriot-Watt University is a Scottish charity
registered under charity number SC000278.
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/