Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
| From | "Kieran McCaul" <Kieran.McCaul@uwa.edu.au> |
| To | <statalist@hsphsun2.harvard.edu> |
| Subject | st: Possible problem with Stata 12 and memory management |
| Date | Tue, 4 Oct 2011 10:17:02 +0800 |
...
First the preliminaries:
I have Stata IC 12.0 running under Windows 7 64-bit OS.
I have 4GB of RAM.
I run Stata do-files from UltraEdit.
I'm running a do-file that crashes when it reaches a -reshape-.
Nevertheless, I can get it reshaped, but only if I first save the file
just prior to running the -reshape- and only if I do this from the
Command Window; I can't get the -reshape- to run from within the
do-file.
I am not quite sure what the problem is, but I suspect that it is a
memory issue.
Any help would be appreciated.
The file to be reshaped is large:
.describe
Contains data from c:\A1\ttt.dta
obs: 890,474
vars: 24 4 Oct 2011 09:21
size: 125,556,834
--------------------------------------------------------------
storage display value
variable name type format label variable label
--------------------------------------------------------------
sequence str5 %9s
diagcode1 str6 %9s
diagcode2 str6 %9s
diagcode3 str6 %9s
diagcode4 str6 %9s
diagcode5 str6 %9s
diagcode6 str6 %9s
diagcode7 str6 %9s
diagcode8 str6 %9s
diagcode9 str6 %9s
diagcode10 str6 %9s
diagcode11 str6 %9s
diagcode12 str6 %9s
diagcode13 str6 %9s
diagcode14 str6 %9s
diagcode15 str6 %9s
diagcode16 str6 %9s
diagcode17 str6 %9s
diagcode18 str6 %9s
diagcode19 str6 %9s
diagcode20 str6 %9s
diagcode21 str6 %9s
diagcode22 str6 %9s
order float %9.0g
--------------------------------------------------------------
Sorted by: sequence order
Below is the output showing what I did in order to get the reshape to
work.
1. The first -reshape- was called from within a do-file and it crashed
with error r(603).
. reshape long diagcode, i(sequence order) j(diag)
(note: j = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
file C:\Users\Kieran\AppData\Local\Temp\ST_00000002.tmp could not be
opened
r(603);
end of do-file
r(603);
2. I then ran the -reshape- command from the Command window and it
crashed again. And I repeated this, just to be sure, and got the same
result.
. reshape long diagcode, i(sequence order) j(diag)
(note: j = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
file C:\Users\Kieran\AppData\Local\Temp\ST_00000002.tmp could not be
opened
r(603);
. reshape long diagcode, i(sequence order) j(diag)
(note: j = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
file C:\Users\Kieran\AppData\Local\Temp\ST_00000002.tmp could not be
opened
r(603);
3. I then compressed the data in memory and ran the -reshape- again:
same error.
. compress
order was float now int
. reshape long diagcode, i(sequence order) j(diag)
(note: j = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
file C:\Users\Kieran\AppData\Local\Temp\ST_00000002.tmp could not be
opened
r(603);
4. So then I tried -set mem-, but that is now disabled in Stata 12.
. set mem 600m
set memory ignored.
Memory no longer needs to be set in modern Statas; memory
adjustments are performed on the fly automatically.
5. Then I saved the data in memory to a file and re-ran the -reshape-
and it worked.
. save "c:\A1\ttt", replace
(note: file c:\A1\ttt.dta not found)
file c:\A1\ttt.dta saved
. reshape long diagcode, i(sequence order) j(diag)
(note: j = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
Data wide -> long
------------------------------------------------------------------------
-----
Number of obs. 890474 -> 2.0e+07
Number of variables 24 -> 4
j variable (22 values) -> diag
xij variables:
diagcode1 diagcode2 ... diagcode22 -> diagcode
------------------------------------------------------------------------
-----
6. So I edited the do-file and added a -save- command just prior to the
-reshape- and re-ran the do-file and it crashed again.
.
. save "c:\A1\ttt", replace
file c:\A1\ttt.dta saved
.
. reshape long diagcode, i(sequence order) j(diag)
(note: j = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
file C:\Users\Kieran\AppData\Local\Temp\ST_00000002.tmp could not be
opened
r(603);
end of do-file
r(603);
7. I then closed other programs that I had running (Outlook and File
Manager) and re-ran the do-file but it crashed at the -reshape- again.
.
. save "c:\A1\ttt", replace
file c:\A1\ttt.dta saved
.
. reshape long diagcode, i(sequence order) j(diag)
(note: j = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
file C:\Users\Kieran\AppData\Local\Temp\ST_00000002.tmp could not be
opened
r(603);
end of do-file
r(603);
8. Finally, I closed UltraEdit and ran the do-file from the Stata Editor
but it again crashed at the -reshape-.
. save "c:\A1\ttt", replace
file c:\A1\ttt.dta saved
.
. reshape long diagcode, i(sequence order) j(diag)
(note: j = 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22)
file C:\Users\Kieran\AppData\Local\Temp\ST_00000002.tmp could not be
opened
r(603);
end of do-file
r(603);
______________________________________________
Kieran McCaul MPH PhD
WA Centre for Health & Ageing (M577)
University of Western Australia
Level 6, Ainslie House
48 Murray St
Perth 6000
Phone: (08) 9224-2701
Fax: (08) 9224 8009
email: Kieran.McCaul@uwa.edu.au
http://myprofile.cos.com/mccaul
http://www.researcherid.com/rid/B-8751-2008
"One can be beaten up in school simply by referring to oneself as one" -
Sheldon
________________________________________________________________________
__
IMPORTANT: This email is intended for the use of the individual
addressee(s) named above and may
contain information that is confidential, privileged or unsuitable for
overly sensitive persons with low
self-esteem, no sense of humour or irrational religious beliefs. If you
are not the intended recipient,
any dissemination, distribution or copying of this email is not
authorised (either explicitly or implicitly)
and constitutes an irritating social faux pas.
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/