RE: st: Re: insheet multi threading

["Schaffer, Mark E" <M.E.Schaffer@hw.ac.uk>]

Because -insheet- and -infile- are built-in and not ados, and only StataCorp could parallelize them?  Or am I missing something?

--Mark

> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu 
> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Nick Cox
> Sent: 02 May 2011 15:43
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: Re: insheet multi threading
> 
> If it's so easy, why not write your own?
> 
> Nick
> 
> On Mon, May 2, 2011 at 2:30 PM, Argyn Kuketayev 
> <akuketayev@mail.primaticsfinancial.com> wrote:
> > it's a very basic task, and I'm sure everyone does it 
> daily: read CSV 
> > files. it takes over 1 hour on 8-core machine to read 13GB file, 
> > because CPU load is 12% all the time, one core is working.
> >
> > it's a junior programmer level assignment to parallelize 
> the parsing 
> > part, that's why i'm surprised Stata didn't do it. it's frustrating 
> > because sometime i get CSVs during the day, and have to 
> wait long long 
> > time before i can upload them into Stata. once in .dta 
> format, all is
> > fast: reading and writing. so, it's clearly parsing part 
> that is slow.
> >
> > On Mon, May 2, 2011 at 12:24 AM, Joseph Coveney 
> <jcoveney@bigplanet.com> wrote:
> >> Are circumstances such that you can have Stata convert 
> your CSV files 
> >> to Stata format overnight?  I'm assuming that Stata won't 
> spend much 
> >> time parsing its own file format the next morning when you 
> go to use the datasets.
> >>
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
> 


-- 
Heriot-Watt University is a Scottish charity
registered under charity number SC000278.


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/