Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Reshape limit


From   "Joseph Coveney" <jcoveney@bigplanet.com>
To   "Statalist" <statalist@hsphsun2.harvard.edu>
Subject   Re: st: Reshape limit
Date   Fri, 7 Nov 2008 01:00:56 +0900

Ashim Kapoor wrote:

Re: st: Reshape limit

--------------------------------------------------------------------------------

Ashim Kapoor wrote:

I have 3 variables value  , j1 and j2.

j1 goes from 1 to 3
j2 goes from 1 to 10032

and the data is in long form.

When I say reshape wide value,i(j1) j(j2)

Stata says that I have too many values for j2.

saying -help limits- or -help reshape- does not enlighten me.

can someone point me in the right direction ?

--------------------------------------------------------------------------------

If you haven't done so already, be sure to -set maxvars- to what you'll
need.

I don't know whether -reshape- has a limit smaller than 32767 variables, but
if so then you can do the reshape in batches, each of which individually is
small enough for -reshape- to handle, assembling the pieces with -merge-.
The do-file below shows how it's done.  Pardon the use of magic numbers.

Joseph Coveney

clear *
set more off

set maxvar 32767


// Create dummy dataset for illustration
set obs 10032
generate byte j1 = .
generate int j2 = _n
generate float value = runiform()

tempfile dataset
save `dataset', emptyok

replace j1 = 1
forvalues j1 = 2/3 {
   append using `dataset'
   replace j1 = `j1' if missing(j1)
}


// Reshape in small batches
sort j2
save  `dataset', replace

tempfile Accumulator
local last_N 0
local first_pass 1
quietly forvalues this_N = 1881(1881)30096 {

   use `dataset'
   keep in `=`last_N' + 1' / `this_N'
   local last_N `this_N'

   reshape wide value, i(j1) j(j2)

   if (`first_pass') {
       save `Accumulator'
       local first_pass 0
   }
   else {
       sort j1
       merge j1 using `Accumulator'
       assert _merge == 3
       drop _merge
       sort j1
      save `Accumulator', replace
   }

}

use `Accumulator', clear
exit


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index