Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: st: Op. sys. refuses to provide memory - a cautionary tale


From   "Jay Tuthill" <jtuthill@bfcwo.com>
To   <statalist@hsphsun2.harvard.edu>
Subject   RE: st: Op. sys. refuses to provide memory - a cautionary tale
Date   Mon, 23 Aug 2010 12:45:10 -0400

Hi All,

I'm also in the middle of upgrading my hardware as my current 32 bit
computers will not let me run very efficiently the datasets I want. I'm
considering the new Intel workstation system SC5650SCWSR which offers
dual Xeon processors and up to 192GB of RAM. Am curious if anyone has
compared the Windows 7 64bit versus Window Server 2008 R2 64bit
operating systems. (I have built my own computers for several years and
have a wide latitude in how I configure them.)

Thanks...Jay

-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu
[mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Eric Booth
Sent: Saturday, August 21, 2010 5:24 PM
To: <statalist@hsphsun2.harvard.edu>
Subject: Re: st: Op. sys. refuses to provide memory - a cautionary tale

<>

On Aug 21, 2010, at 4:19 PM, Eric Booth wrote:

>  but beyond this limit Stata won't slow down as you add or allocate
more RAM.  

Clarification:
That should say, "but UP TO (or before you reach this memory limit)
Stata won't slow down as you add or allocate more RAM..."

On Aug 21, 2010, at 4:19 PM, Eric Booth wrote:

> <>
> On Aug 20, 2010, at 4:07 PM, Tony wrote:
>> Too much RAM will slow it down.
> 
> Stata will certainly slow down if you set and use more memory in Stata
than is physically available on your machine because you start using
virtual memory, but beyond this limit Stata won't slow down as you add
or allocate more RAM.  
> That is, it will take the same time to run a do-file on a 1G dataset
whether you allocate 2G, 8G, or 20G of memory to Stata; however it will
get excruciatingly slow if you set your memory to 500m and  use a 1G
dataset.
> 
> 
>> On Fri, Aug 20, 2010 at 1:28 PM, Craig, Benjamin M. wrote:
> 
>>> The purpose is real world speed, so has anyone actually noticed if
going
>>> up to 24GB RAM, solid state drive expedited your jobs
> 
> 
> I haven't tested the idea of getting a SSD drive, but I think the
speed advantage would be evident mainly when you were opening(reading)
or saving(writing) a large dataset since your using the data in memory
the rest of the time.  I do want to try out installing a SSD drive for
working with data that is larger than my physical RAM and requires me to
use virtual memory to work with it (I've maxed out my physical RAM with
8G sticks in each of the slots).  I've read about moving your page/swap
file to a SSD which should speed things up when working in virtual
memory (but since SSDs wear out faster with more read/writes, this might
be a concern).  Also, you could move the location of the tempfiles that
Stata creates to that path by setting your OS system temp file location
(see:  http://www.stata.com/statalist/archive/2009-05/msg00416.html).
Maybe someone here has tried working with SSD and large datasets ?
> 
> Again, more RAM is always better IMO--but it only speeds you up in the
sense that it prevents you from using page file.  There are also speeds
associated with RAM (mine is 1066 DDR3), but I don't know much
differences in memory speed matters.
> 
> 
>> On Fri, Aug 20, 2010 at 1:28 PM, Craig, Benjamin M. wrote:
>>> Does any listserv member think that I should go from 6 to 12 cores?
>>> 
>>> Six Core Processor,X5680,3.33GHz,12M,6.4GT/s
>>> Dual Six Core Processor,X5680,3.33GHz,12M,6.4GT/s
> 
> Depends on what you are doing.  If you've got a time intensive
procedure that you're running on your 6 core machine, try running it
with your -set processors- at 1, 2, 4, and 6 and see what kind of speed
increase you observe, e.g.:
> 
> *****!
> timer clear 
> forval n = 1(2)6 {
> clear all
> set mem 32g
> set processors `n'
> timer on `n'
> 	<your command goes here>
> timer off `n'
> 	}
> timer list
> *****!
> 
> ~ Eric
> 
> 
> __
> Eric A. Booth
> Public Policy Research Institute
> Texas A&M University
> ebooth@ppri.tamu.edu
> Office: +979.845.6754
> 
> 
>> 
>> On Fri, Aug 20, 2010 at 1:28 PM, Craig, Benjamin M.
>> <Benjamin.Craig@moffitt.org> wrote:
>>> Thanks Nick, I have learned that to truly take advantage of the
latest
>>> version of Stata, 64-bits and 4 or more cores is required. To be a
bit
>>> more specific, lets assume I am using 6-core Stata MP on Windows 7
>>> Professional, 64-bit for computationally intensive simulation
analyses.
>>> 
>>> Does any listserv member think that I should go from 6 to 12 cores?
>>> 
>>> Six Core Processor,X5680,3.33GHz,12M,6.4GT/s
>>> Dual Six Core Processor,X5680,3.33GHz,12M,6.4GT/s
>>> 
>>> Is it worthwhile to upgrade from RAM and Hard drive? For example,
>>> 
>>> 12GB DDR3 ECC SDRAM Memory,1333MHz,6X2GB
>>> 
>>> 1TB SATA 3.0Gb/s,7200 RPM HardDrive with 32MB DataBurst Cache
>>> 
>>> The purpose is real world speed, so has anyone actually noticed if
going
>>> up to 24GB RAM, solid state drive expedited your jobs? In theory, it
>>> should, but I am hoping that someone has purchase a computer
recently to
>>> test this hypothesis.
>>> 
>>> Cheers,
>>> Ben
>>> 
>>> 
>>> Benjamin M. Craig, PhD
>>> 
>>> Assistant Member, Health Outcomes & Behavior, Moffitt Cancer Center
>>> 
>>> Associate Professor, Economics, University of South Florida
>>> 
>>> 12902 Magnolia Dr, MRC-CANCONT, Tampa, FL 33612-9416
>>> 
>>> Phone (813) 745-6710; Fax (813) 745-6525
>>> 
>>> -----Original Message-----
>>> From: owner-statalist@hsphsun2.harvard.edu
>>> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Nick Cox
>>> Sent: Friday, August 20, 2010 5:00 AM
>>> To: 'statalist@hsphsun2.harvard.edu'
>>> Subject: RE: st: Op. sys. refuses to provide memory - a cautionary
tale
>>> 
>>> I think this partially answers itself in that I don't think that it
can
>>> fairly be expected that a company website is a proper place for a
>>> company, in this case StataCorp, to offer opinions about anything
that
>>> is currently controversial.
>>> 
>>> That said, Benjamin's question is obviously practical and a fair one
for
>>> members of this list to venture opinions and comment from
experience. In
>>> addition, presumably people other than econometricians are not
excluded.
>>> 
>>> 
>>> Nick
>>> n.j.cox@durham.ac.uk
>>> 
>>> Craig, Benjamin M.
>>> 
>>> To clarify, the website answers essential questions (which systems
are
>>> supported?) and provides some advice and education. I need to know
more
>>> details on current controversies relating to multiple core, 64-bit
and
>>> drive speeds in the real world. In this sense, it is a bit
incomplete to
>>> say that more core, more bits, and faster drives are preferable. As
was
>>> previously post, I had thought that a 32-bit dual core desktop was
good
>>> enough for my needs, and was woefully wrong. Others seem to
follow...
>>> 
>>> http://www.stata.com/statalist/archive/2010-07/msg01337.html
>>> 
>>> If a listserv member has tried STATA MP on multiple machines, I
would
>>> like to know what worked best so that I can buy one. There is an
obvious
>>> caveat: speed depends on the task. However, I would counter that
some
>>> data are better than none. For example, I recently bootstrapped a ML
>>> with 1000 iteration and inequality constraints. It took 4 weeks. Do
you
>>> think your machine can do better? Personally, I do not use stata for
>>> database management, and doubt that many econometricians do. If
someone
>>> has a good analytics machine, and he/she thinks that it works well,
I'd
>>> like to know its components. Maybe a consensus will emerge. Maybe
not.
>>> 
>>> Martin Weiss
>>> 
>>> In which respect is the Stata website "incomplete"? There is advice
at
>>> http://www.stata.com/products/opsysmp.html, and how is the website
>>> supposed to give more detailed advice? It does not know your
specific
>>> setup, hence the reluctance to go into greater depth...
>>> 
>>> Craig, Benjamin M.
>>> 
>>> Okay, I give up... I need a new machine. Due to institutional
policies,
>>> I need to buy a Dell. Otherwise, I would very much like any advice
on
>>> this purchase. My best guess is a 64-bit 8-core desktop for a 6-core
>>> version of Stata MP. I don't need a rocket, just a racecar.
>>> 
>>> If you have any specifications that you would like to share with me,
>>> please send them directly or post them on the listserv for others
>>> looking to upgrade. I have read the Stata website, which seems
>>> incomplete.
> 
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index