Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Is anyone aware of the more efficient way to use reshape command


From   "Michael I. Lichter" <mlichter@buffalo.edu>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: Is anyone aware of the more efficient way to use reshape command
Date   Wed, 08 Jul 2009 11:05:42 -0400

Dmitriy, a general but not necessarily palatable solution is to use SAS PROC TRANSPOSE if Stata isn't up to the task. You can use -fdasave- and -fdause- to export and reimport your data.

To help the list troubleshoot your particular problem, however, I suspect that you will need to supply more information about the structure of your dataset (how many variables and how many observations), the specific -reshape- command you're using, and what you see when you -query memory-.

Two possibly helpful suggestions, though:

1. Drop all unnecessary variables before doing the -reshape-.
2. Consider whether the reshaping can be done "manually" through some set of steps. Since you're trying to go from long to wide, you presumably have a set of repeated measurements, e.g., weight1 weight2 weight3. You can turn these weights into three records using a loop like this:

forvalues i = 1 to 3 {
   use wide, clear
   gen t = `i'
   gen weight = weight`i'
   keep t weight
   tempfile long`i'
   save `long``i''
}
use `long1'
forvalues i = 2 to 3 {
  append using `long``i''
}

I haven't checked the code for correctness, but hopefully I'm getting the idea across.

-ml

Dmitriy Krichevskiy wrote:
Hello all,

I have a relatively large panel data set 0.6GB and I am trying to
reshape it from wide to long using   reshape. Allocating 7GB of memory
to Stata leads to 'not enough memory' message even though it is more
than 10 times the dataset. Allocating 12-15GB leads to Stata not
responding in Windows. I do have the latest PC with 64bit Vista and
64bit Stata 10, -compress- does not help either.
Any help is greatly appreciated.   Thanks

Dmitriy Krichevskiy
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

--
Michael I. Lichter, Ph.D. <mlichter@buffalo.edu>
Research Assistant Professor & NRSA Fellow
UB Department of Family Medicine / Primary Care Research Institute
UB Clinical Center, 462 Grider Street, Buffalo, NY 14215
Office: CC 126 / Phone: 716-898-4751 / FAX: 716-898-3536

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index