Statalist The Stata Listserver


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

Re: st: Re: Setting memory


From   Ronnie Babigumira <rb.glists@gmail.com>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: Re: Setting memory
Date   Wed, 25 Oct 2006 17:40:49 +0200

Micheal

Good stuff. -usemem- has entered the building

Many thanks

Ronnie

Michael Blasnik wrote:
..
You may just want to write your own file opening ado that allocates memory automatically. I've included the ado that I use below. In Windows, I change the action of the file association so that double clicking a dta file calls -usemem- rather than -use-.

You might want to customize it for your machine and typical usage. The key line to edit would be the one that begins -local needmem=...-. The 10 on that line refers to the smallest amount of memory I will allocate to a dataset, regardless of its size. The 1200 refers to the maximum amount of memory I will allocate (since I usually can't get Windows XP to allocate much more than that), and the formula in the middle is what is used between these extremes based on approximates 1.5 times the dataset size. This allocation doesn't always work out, especially if I'm adding a lot of variables or doing a large merge after I open it, but it tends to work often enough. The ado also reduces the current memory allocation if it is greater than 10 times the calculated memory need.

program define usemem
version 8.2
qui memory
local totmem=r(M_total)/(1024^2)
qui {
de using "`1'"
local needmem=min(max(10,round(1.5*r(N)*r(width)/(1024^2),1)) ,1200)
if `totmem'<`needmem' set memory `needmem'm
if `totmem'>(`needmem'*10) {
local needmem=int(`needmem'*10)
set memory `needmem'm
}
}
use "`1'"
end


Michael Blasnik

----- Original Message ----- From: "Ronnie Babigumira" <rb.glists@gmail.com>
To: <statalist@hsphsun2.harvard.edu>
Sent: Tuesday, October 24, 2006 6:36 AM
Subject: st: Setting memory



Setting memory is something I have always done by try and error. However, recently more than 2 people have asked me if there is a smarter way of knowing roughly how much memory to set before trying to read in a dataset. Does anyone have any ideas (for example, would the size of the data or the number of variables be helpful)

Ronnie
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index