Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: file too big to import


From   Maarten buis <maartenbuis@yahoo.co.uk>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: file too big to import
Date   Wed, 20 Apr 2011 12:48:18 +0100 (BST)

--- On Wed, 20/4/11, King, Carina wrote:
> I am having problems importing a file. It is stored as csv
> file and contains (I think, as I haven't been able to see
> the full data yet) over 1,400,000 records with 25 variables.
> So far I have tried the following:
> 
> Opening it with insheet using "D:\datasets\mr_final.csv"
> (not enough memory), so and I increased my memory (which is
> now set to 10,000) and it's still not enough. 

Assuming that the data are stored as doubles than you would 
need -di (1.4E6 * 25 * 8 + 4* 1.4E6)/(1024^2)- = 272 Mb to 
load that dataset and you allocated only 10 Mb (10,000 Kb = 
10 Mb), so try setting the memory at 300M.

The formula I used for guesstimating the size of your data 
came from:
http://www.stata.com/support/faqs/data/howbig.html

Hope this helps,
Maarten

--------------------------
Maarten L. Buis
Institut fuer Soziologie
Universitaet Tuebingen
Wilhelmstrasse 36
72074 Tuebingen
Germany

http://www.maartenbuis.nl
--------------------------

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index