Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: set memory question


From   "Eric A. Booth" <ebooth@ppri.tamu.edu>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: set memory question
Date   Tue, 25 Aug 2009 23:04:29 -0500

Hi David:

Try increasing your memory incrementally using a program like the one described here:

  http://www.stata.com/statalist/archive/2009-07/msg00899.html

Try this with virtual memory turned ON and then try it again with it turned OFF & look for any differences in both the -query- in Stata as well as the memory usage reported in the Mac OSX 'Activity Monitor.app' ( located here: "/Applications/Utilities/Activity\ Monitor.app" ) because Stata might think it has more memory allocated than the system is really giving it. The reason for this is that *nix systems have virtually unlimited swap file space & it can also use mounted/storage/network drives or devices for swap space in parallel. So, even if VM is turned off in Stata, the system can readily use HD space for virtual memory AND if you've got an external HD or mounted network drive, your system might be using that to increase the memory that Stata can utilize.

You can try this process again after disabling the Mac OSX virtual memory/swap space by editing your "/etc/rc" file (with caution):

1.typing into terminal:   sudo pico /etc/rc
2. Type # in front of the line :  "dynamic_pager -F ${swapdir}/swapfile"
3.  now type into terminal:  sudo srm /private/var/vm/swapfile*

But I would make a backup of your swapfile* before tinkering with these settings (e.g. "sudo cp /etc/rc /etc/rc.orig" ). Another suggestion might be to find or create a dataset larger than your physical memory limits (4 GB) and ask Stata to open it--then take a look at your -query- and your 'Activity Monitor.app' (that is, if Stata will somehow open a file that large with VM turned off..)

Best,

Eric

__
Eric A. Booth
Public Policy Research Institute
Texas A&M University
ebooth@ppri.tamu.edu
Office: +979.845.6754





On Aug 25, 2009, at 9:40 PM, David Airey wrote:

.

No, my virtual memory is not on (under efficiency settings below):

. query

---------------------------------------------------------------------------------
   Memory settings (also see query memory for a more complete report)
       set memory          50M
       set maxvar          5000
       set matsize         400
---------------------------------------------------------------------------------
   Output settings
       set more            on
       set rmsg            off
       set dp              period     may be period or comma
       set linesize        81         characters
       set pagesize        49         lines
       --------------------------------------------------
       set logtype         smcl       may be smcl or text
       --------------------------------------------------
       set eolchar         unix       may be mac or unix
       set notifyuser      on
       set playsnd         off
---------------------------------------------------------------------------------
   Interface settings
       set revkeyboard     on
       set varkeyboard     on
       set smoothfonts     on
       --------------------------------------------------
       set linegap         1          pixels
       set scrollbufsize   204800     characters
       set varlabelpos                (not relevant)
       set reventries      5000       lines
       --------------------------------------------------
       set maxdb           50         dialog boxes
---------------------------------------------------------------------------------
   Graphics settings
       set graphics        on
       set scheme          s1color
set printcolor automatic may be automatic, asis, gs1, gs2, gs3 set copycolor automatic may be automatic, asis, gs1, gs2, gs3
---------------------------------------------------------------------------------
   Efficiency settings
       set adosize         1000       kilobytes
       set virtual         off
---------------------------------------------------------------------------------
   Network settings
       set checksum        off
       set timeout1        120        seconds
       set timeout2        300        seconds
       --------------------------------------------------
       set httpproxy       off
       set httpproxyhost
       set httpproxyport   8080
       --------------------------------------------------
       set httpproxyauth   off
       set httpproxyuser
       set httpproxypw
---------------------------------------------------------------------------------
   Update settings
       set update_query    on
       set update_interval 7
       set update_prompt   on
---------------------------------------------------------------------------------
   Trace (programming debugging) settings
       set trace           off
       set tracedepth      32000
       set traceexpand     on
       set tracesep        on
       set traceindent     on
       set tracenumber     off
       set tracehilite
---------------------------------------------------------------------------------
   Mata settings
       set matastrict      off
       set matalnum        off
       set mataoptimize    on
       set matafavor       space      may be space or speed
       set matacache       400        kilobytes
set matalibs lmatabase;lmataado;lmataopt;lmoremata;lmoremata10;lso
> mersd;lsq;lsqbm
       set matamofirst     off
---------------------------------------------------------------------------------
   Other settings
       set type            float      may be float or double
       set level           95         percent confidence intervals
set maxiter 16000 max iterations for estimation commands
       set searchdefault   local      may be local, net, or all
       set seed            X075bcd151f123bb5159a55e50022865746ad
       set varabbrev       on
       set emptycells      keep       may be keep or drop
       set processors      2
---------------------------------------------------------------------------------




In *nix (including OS X) doesn't Stata use virtual memory
as well as RAM? I recall doing this in Solaris and Linux.





If I have 4 gb installed on StataMP 11 on OS X 10.5.8, yet

-set memory 5g-

doesn't cause an error message. Neither does -set memory 50g-.

The manuals says it should but look at this (I don't have 50 gb of memory):

. set memory 50g

Current memory allocation

current memory usage settable value description (1M = 1024k) -------------------------------------------------------------------- set maxvar 5000 max. variables allowed 2.105M set memory 51200M max. data space 51,200.000M set matsize 400 max. RHS vars in models 1.254M ----------- 51,203.359M

. memory
                                                bytes
--------------------------------------------------------------------
Details of set memory usage
  overhead (pointers)                               0        0.00%
  data                                              0        0.00%
                                      ----------------------------
  data + overhead                                   0        0.00%
  free                                 53,687,091,200      100.00%
                                      ----------------------------
  Total allocated                      53,687,091,200      100.00%
--------------------------------------------------------------------
Other memory usage
  set maxvar usage                          2,206,954
  set matsize usage                         1,315,200
  programs, saved results, etc.                 1,994
                                      ---------------
  Total                                     3,524,148
-------------------------------------------------------
Grand total                              53,690,615,348


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index