Stata The Stata listserver
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

st: Re: memory management in Win2000


From   kturner@stata.com (Kevin Turner)
To   statalist@hsphsun2.harvard.edu
Subject   st: Re: memory management in Win2000
Date   Fri, 11 Jul 2003 10:46:14 -0500

Steven Stillman <Steven.Stillman@lmpg.dol.govt.nz> wrote: 

>>This is more of a windows questions than a stata question.  I frequently
>>work on large datasets (150-200m) on a Pent IIII PC w/ Win2000 and 512m ram.
>>I find that it takes a long time (3-5 minutes) for these large datasets to
>>initial open (stata is very fast once the datasets are loaded).  Does anyone
>>have any suggestions on how to get Win2000 to allocate memory quicker to
>>stata (this seems to be what holds things up)?  I was thinking that ala Unix
>>there might be a way to assign a fixed block of ram to stata instead of it
>>being dynamically reallocated every time a dataset is opened.

And Friedrich Huebler <huebler@rocketmail.com> had this suggestion:

>Do you use Stata 8 with a dataset from an earlier version of Stata?
>Try to convert the dataset to Stata 8 format and see if it takes less
>time to open.

This suggestion is a very good one. Using the latest format of dataset for your
version of Stata is always faster.

Another suggestion:
Due to the fact that it takes so long and that the size of the dataset is
already a large percentage of the physical RAM (200/512), the odds are that
Windows is being forced to use virtual memory. You can verify this by using the
Windows Task Manager (typically you can right-click on the task bar to bring up
the Task Manager; Ctrl-Alt-Del is another way to get to it). On the Task
Manager, there is a tab for Performance and on this tab is a couple of tables
showing memory usage. If your Total Commit Charge memory is higher than your
Total Physical memory, you are in a state where Windows is forced to use
virtual memory. 

You can *probably* witness the problem by viewing the Available Physical Memory
_during_ the load of the Stata dataset -- it should hover around 6-3Mb as
Windows continually swaps memory from Physical to Virtual space. 

Solutions:
	1. Buy more RAM. 
	2. Close all unnecessary applications while you are working with these
	   large datasets.
	3. There are a few more odd tricks that can be done to speed up 
	   virtual memory, but may not fix the entire problem: 
	   http://www.pureperformance.com/js/showtip.asp?id=22 


Hope this helps,

--Kevin 
kturner@stata.com
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index