Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

st: Possible problem with Stata 12 and memory management


From   "Kieran McCaul" <Kieran.McCaul@uwa.edu.au>
To   <statalist@hsphsun2.harvard.edu>
Subject   st: Possible problem with Stata 12 and memory management
Date   Tue, 4 Oct 2011 10:17:02 +0800

...

First the preliminaries:
I have Stata IC 12.0 running under Windows 7 64-bit OS.
I have 4GB of RAM.
I run Stata do-files from UltraEdit.

I'm running a do-file that crashes when it reaches a -reshape-.
Nevertheless, I can get it reshaped, but only if I first save the file
just prior to running the -reshape- and only if I do this from the
Command Window; I can't get the -reshape- to run from within the
do-file.

I am not quite sure what the problem is, but I suspect that it is a
memory issue.
Any help would be appreciated.



The file to be reshaped is large: 

.describe

Contains data from c:\A1\ttt.dta
  obs:       890,474                          
 vars:            24                          4 Oct 2011 09:21
 size:   125,556,834                          
--------------------------------------------------------------
              storage  display     value
variable name   type   format      label      variable label
--------------------------------------------------------------
sequence        str5   %9s                    
diagcode1       str6   %9s                    
diagcode2       str6   %9s                    
diagcode3       str6   %9s                    
diagcode4       str6   %9s                    
diagcode5       str6   %9s                    
diagcode6       str6   %9s                    
diagcode7       str6   %9s                    
diagcode8       str6   %9s                    
diagcode9       str6   %9s                    
diagcode10      str6   %9s                    
diagcode11      str6   %9s                    
diagcode12      str6   %9s                    
diagcode13      str6   %9s                    
diagcode14      str6   %9s                    
diagcode15      str6   %9s                    
diagcode16      str6   %9s                    
diagcode17      str6   %9s                    
diagcode18      str6   %9s                    
diagcode19      str6   %9s                    
diagcode20      str6   %9s                    
diagcode21      str6   %9s                    
diagcode22      str6   %9s                    
order           float  %9.0g                  
--------------------------------------------------------------
Sorted by:  sequence  order


Below is the output showing what I did in order to get the reshape to
work.

1. The first -reshape- was called from within a do-file and it crashed
with error r(603).

. reshape long diagcode, i(sequence order) j(diag)
(note: j = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
file C:\Users\Kieran\AppData\Local\Temp\ST_00000002.tmp could not be
opened
r(603);

end of do-file

r(603);

2. I then ran the -reshape- command from the Command window and it
crashed again.  And I repeated this, just to be sure, and got the same
result.

. reshape long diagcode, i(sequence order) j(diag)
(note: j = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
file C:\Users\Kieran\AppData\Local\Temp\ST_00000002.tmp could not be
opened
r(603);

. reshape long diagcode, i(sequence order) j(diag)
(note: j = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
file C:\Users\Kieran\AppData\Local\Temp\ST_00000002.tmp could not be
opened
r(603);

3. I then compressed the data in memory and ran the -reshape- again:
same error.

. compress
order was float now int

. reshape long diagcode, i(sequence order) j(diag)
(note: j = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
file C:\Users\Kieran\AppData\Local\Temp\ST_00000002.tmp could not be
opened
r(603);

4. So then I tried -set mem-, but that is now disabled in Stata 12.

. set mem 600m
set memory ignored.
    Memory no longer needs to be set in modern Statas; memory
adjustments are performed on the fly automatically.

5. Then I saved the data in memory to a file and re-ran the -reshape-
and it worked.

. save "c:\A1\ttt", replace
(note: file c:\A1\ttt.dta not found)
file c:\A1\ttt.dta saved

. reshape long diagcode, i(sequence order) j(diag)
(note: j = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)

Data                               wide   ->   long
------------------------------------------------------------------------
-----
Number of obs.                   890474   -> 2.0e+07
Number of variables                  24   ->       4
j variable (22 values)                    ->   diag
xij variables:
     diagcode1 diagcode2 ... diagcode22   ->   diagcode
------------------------------------------------------------------------
-----


6. So I edited the do-file and added a -save- command just prior to the
-reshape-  and re-ran the do-file and it crashed again.
. 
. save "c:\A1\ttt", replace
file c:\A1\ttt.dta saved

. 
. reshape long diagcode, i(sequence order) j(diag)
(note: j = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
file C:\Users\Kieran\AppData\Local\Temp\ST_00000002.tmp could not be
opened
r(603);

end of do-file

r(603);

7. I then closed other programs that I had running (Outlook and File
Manager) and re-ran the do-file but it crashed at the -reshape- again.
.
. save "c:\A1\ttt", replace
file c:\A1\ttt.dta saved

. 
. reshape long diagcode, i(sequence order) j(diag)
(note: j = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
file C:\Users\Kieran\AppData\Local\Temp\ST_00000002.tmp could not be
opened
r(603);

end of do-file

r(603);

8. Finally, I closed UltraEdit and ran the do-file from the Stata Editor
but it again crashed at the -reshape-.

. save "c:\A1\ttt", replace
file c:\A1\ttt.dta saved

. 
. reshape long diagcode, i(sequence order) j(diag)
(note: j = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
file C:\Users\Kieran\AppData\Local\Temp\ST_00000002.tmp could not be
opened
r(603);

end of do-file

r(603);




______________________________________________
Kieran McCaul MPH PhD
WA Centre for Health & Ageing (M577)
University of Western Australia
Level 6, Ainslie House
48 Murray St
Perth 6000
Phone: (08) 9224-2701
Fax: (08) 9224 8009
email: Kieran.McCaul@uwa.edu.au
http://myprofile.cos.com/mccaul 
http://www.researcherid.com/rid/B-8751-2008

"One can be beaten up in school simply by referring to oneself as one" -
Sheldon
________________________________________________________________________
__
IMPORTANT: This email is intended for the use of the individual
addressee(s) named above and may
contain information that is confidential, privileged or unsuitable for
overly sensitive persons with low
self-esteem, no sense of humour or irrational religious beliefs. If you
are not the intended recipient, 
any dissemination, distribution or copying of this email is not
authorised (either explicitly or implicitly) 
and constitutes an irritating social faux pas.


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index