I don't have a solution to your issue but I regularly open 300-500MB
datasets using Stata 7 on Win2K at rates around 150-200MB/minute. The
machines on which that occurs have RAID 5 arrays and 2GB or more of physical
memory, but that shouldn't explain this big of a difference.
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Steven Stillman
(LMPG)
Sent: Monday, July 14, 2003 1:21 AM
To: '[email protected]'
Subject: st: RE: Re: memory management in Win2000
Sorry, I should have been a bit clearer here. I am using stata 7 with stata
7 format datasets. I am not using virtual memory with at least 100m of
physical memory available after opening a 240m dataset. The problem is with
the initial allocation of physical memory that occurs when I open stata for
the first time. This is just plain slow taking approx. 1min to allocate
each 20m. I am assuming this has to do with win2000 and my particular
motherboard, etc. and nothing to do with Stata but I was hoping that some
experts out there might have a solution for speeding things up.
Thanks,
Steve
PS I am about to install stata 8 so we will soon see if this helps speeds
things up.
> -----Original Message-----
> From: [email protected] [SMTP:[email protected]]
> Sent: Saturday, July 12, 2003 3:46 AM
> To: [email protected]
> Subject: st: Re: memory management in Win2000
>
> Steven Stillman <[email protected]> wrote:
>
> >>This is more of a windows questions than a stata question. I
> >>frequently work on large datasets (150-200m) on a Pent IIII PC w/
> >>Win2000 and 512m
> ram.
> >>I find that it takes a long time (3-5 minutes) for these large
> >>datasets
> to
> >>initial open (stata is very fast once the datasets are loaded).
> >>Does
> anyone
> >>have any suggestions on how to get Win2000 to allocate memory
> >>quicker to stata (this seems to be what holds things up)? I was
> >>thinking that ala
> Unix
> >>there might be a way to assign a fixed block of ram to stata instead
> >>of
> it
> >>being dynamically reallocated every time a dataset is opened.
>
> And Friedrich Huebler <[email protected]> had this suggestion:
>
> >Do you use Stata 8 with a dataset from an earlier version of Stata?
> >Try to convert the dataset to Stata 8 format and see if it takes less
> >time to open.
>
> This suggestion is a very good one. Using the latest format of dataset
> for your version of Stata is always faster.
>
> Another suggestion:
> Due to the fact that it takes so long and that the size of the dataset
> is already a large percentage of the physical RAM (200/512), the odds
> are that Windows is being forced to use virtual memory. You can verify
> this by using the Windows Task Manager (typically you can right-click
> on the task bar to bring up the Task Manager; Ctrl-Alt-Del is another
> way to get to it). On the Task Manager, there is a tab for Performance
> and on this tab is a couple of tables showing memory usage. If your
> Total Commit Charge memory is higher than your Total Physical memory,
> you are in a state where Windows is forced to use virtual memory.
>
> You can *probably* witness the problem by viewing the Available
> Physical Memory _during_ the load of the Stata dataset -- it should
> hover around 6-3Mb as Windows continually swaps memory from Physical
> to Virtual space.
>
> Solutions:
> 1. Buy more RAM.
> 2. Close all unnecessary applications while you are working with
> these
> large datasets.
> 3. There are a few more odd tricks that can be done to speed up
> virtual memory, but may not fix the entire problem:
> http://www.pureperformance.com/js/showtip.asp?id=22
>
>
> Hope this helps,
>
> --Kevin
> [email protected]
> *
> * For searches and help try:
> * http://www.stata.com/support/faqs/res/findit.html
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
The information contained in this document is intended only for the
addressee and is not necessarily the views nor the official communication of
the Department of Labour. All final/official papers which are sent from the
Department will be sent by non-electronic means, on appropriate letterhead,
signed by authorised personnel.
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
