Stas Kolenikov wrote:
>So what exactly do you do with your data? 6 million observations would
>require 6Mb per byte variable, and 50Mb per double variable, so you
>can have a regression with say 10 variables mixing binary and
>continuous predictors in your memory. For anything greater than that,
>you would be better off with more RAM... which is kinda obvious. The
>best acadmic machine that I ever had access to, as far as I can
>recall, had something like 48Gb of memory under Solaris. I'd say
>figure out what is the most complex model you are going to estimate,
>in terms of the number of variables; and get a machine with the RAM
>that's at least twice as big as that. Or better four times as big as
>that, so that your model can still double in complexity, or there are
>some intermediate results, etc. If you are thinking about 40
>variables, you would probably need 8Gb to be on a relatively safe
>side. I don't think the OS issue is very much relevant except for
>overall reliability; if you trust WinXP enough, that's up to you -- I
>would probably move to Linux with a SCSI RAID, to make things move
>faster, especially if you are doing a lot of data handling like
>merging, subsetting, etc.
>
>Some other specific recommendations is to keep your data as small as
>possible in terms of both observations, variables, and data types
>(always type -compress- before -save-).
>
>On 12/13/05, Anirudh V. S. Ruhil <[email protected]> wrote:
>
>
>>There may have been some traffic about a particular element or the other of
>>my question, but nevertheless ...
>>
>>We are working with extremely large datafiles (hitting 6 million obs with
>>only partial list of covariates attached), and our usual academic P4
>>puppies with 512MB RAM, Win XP, and Stata 9 are simply unable to do a
>>thing. Many of you have run into similar issues in the past and so may be
>>able to tell me when you did step out to buy new hardware and software,
>>what exactly were the key specs, what platform, etc. did you find useful.
>>Moving off Win XP is not an issue so be bold.
>>
>>thanks in advance
>>
>>Ani
>>*
>>* For searches and help try:
>>* http://www.stata.com/support/faqs/res/findit.html
>>* http://www.stata.com/support/statalist/faq
>>* http://www.ats.ucla.edu/stat/stata/
>>
>>
>>
>
>
>--
>Stas Kolenikov
>http://stas.kolenikov.name
>
>*
>* For searches and help try:
>* http://www.stata.com/support/faqs/res/findit.html
>* http://www.stata.com/support/statalist/faq
>* http://www.ats.ucla.edu/stat/stata/
>
>
>
hi,
remember the important issue of the amount of RAM that windows will
allow Stata to allocate. see
http://www.stata.com/support/faqs/win/winmemory.html
if you have Stata 32-bit, and a 32-bit machine, Linux is a better option to use above 2Gb of RAM.
cheers,
miguel
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
