Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Mata Memory


From   Olesya Kotlyachkov <vovko@mcmaster.ca>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: Mata Memory
Date   Mon, 12 Nov 2012 11:14:59 -0500

On 07/23/2012, Natarajan Balasubramanian wrote:

Hi,
I am having trouble with memory allocation in Stata (I tried both IC Stata 11X64 and IC Stata 12X64, both).

Here is the command and the error:
************
felsdvreg y x, ivar(f1) jvar(f2) feff(feff) peff(peff) mover(mover)  group(group) xb(xb) res(res) mnum(mnum) pobs(pobs)

Memory requirement for moment matrices in GB:
    8.045968656
                       J():  3900  unable to allocate real <tmp>[31712,31712]
               decompreg():     -  function returned error
                   <istmt>:     -  function returned error
r(3900);

end of do-file

r(3900);
************
Note that felsdvreg estimates regressions with two high-dimensional fixed effects. In this case, I have about 35,000 of each fixed effect on a dataset of about 200,000.

For the r(3900), this is what the Stata manual says , "You must take up the problem with your  operating system". Based on an older post, (http://www.stata.com/statalist/archive/2006-10/msg00938.html   ), a 64 bit machine (which is what I am using) is better. 8GB is more than the RAM on my machine (4GB) but is there any way to fix this without upgrading the RAM. Note that with a smaller sample (10% of the current one), the same program runs correctly.

Hi -
I am having exactly the same issue with the felsdvreg command. felsdvreg is producing the same error when reaching step 11:
********
Step 11 - Fit restricted models for F-tests Start Mata environment
Memory requirement for moment matrices in GB:
.15945408

decompreg(): 3900 unable to allocate real <tmp>[275922,4415]
<istmt>: - function returned error r(3900);
 ********
Following Stata Manual advice, I took up the problem with my operating system. I am running 4 core Stata-12 on a 64 bit Windows 7 machine with 4 cores, 16 Gb physical RAM, and a large hard drive with system managed page file. I ran the Process Explorer to see what's happening with the system and the memory. Whatever felsdvreg is doing, Stata/Mata take up all 16 gig of RAM, ask for more - and crash. Normally Windows would automatically add the virtual memory when the physical limit is reached, however in this case it does not. Any idea why? I have also tried to set min_memory in Stata right away - clean system after reboot, min_memory of 24 Gb sets fine. Then I start the do-file - and the command does not even run, I only see the felsdvreg command line and no output at all - not even the first step. It does not really freeze, it just does not do anything.

Has anyone seen this happen and especially if you have resolved it - please help me figure it out! I'm interested in both issues - reaching physical RAM limit while in Mata, and setting min_memory and Mata. Any help would be greatly appreciated!

Thanks!
Olesya Kotlyachkov
McMaster University
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index