Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Reshape limit


From   "Friedrich Huebler" <fhuebler@gmail.com>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: Reshape limit
Date   Thu, 6 Nov 2008 11:07:50 -0500

Joseph,

Could you please explain the meaning of "magic numbers"?

Thanks,

Friedrich

On Thu, Nov 6, 2008 at 11:00 AM, Joseph Coveney <jcoveney@bigplanet.com> wrote:
> Ashim Kapoor wrote:
>
> Re: st: Reshape limit
>
> --------------------------------------------------------------------------------
>
> Ashim Kapoor wrote:
>
> I have 3 variables value  , j1 and j2.
>
> j1 goes from 1 to 3
> j2 goes from 1 to 10032
>
> and the data is in long form.
>
> When I say reshape wide value,i(j1) j(j2)
>
> Stata says that I have too many values for j2.
>
> saying -help limits- or -help reshape- does not enlighten me.
>
> can someone point me in the right direction ?
>
> --------------------------------------------------------------------------------
>
> If you haven't done so already, be sure to -set maxvars- to what you'll
> need.
>
> I don't know whether -reshape- has a limit smaller than 32767 variables, but
> if so then you can do the reshape in batches, each of which individually is
> small enough for -reshape- to handle, assembling the pieces with -merge-.
> The do-file below shows how it's done.  Pardon the use of magic numbers.
>
> Joseph Coveney
>
> clear *
> set more off
>
> set maxvar 32767
>
>
> // Create dummy dataset for illustration
> set obs 10032
> generate byte j1 = .
> generate int j2 = _n
> generate float value = runiform()
>
> tempfile dataset
> save `dataset', emptyok
>
> replace j1 = 1
> forvalues j1 = 2/3 {
>   append using `dataset'
>   replace j1 = `j1' if missing(j1)
> }
>
>
> // Reshape in small batches
> sort j2
> save  `dataset', replace
>
> tempfile Accumulator
> local last_N 0
> local first_pass 1
> quietly forvalues this_N = 1881(1881)30096 {
>
>   use `dataset'
>   keep in `=`last_N' + 1' / `this_N'
>   local last_N `this_N'
>
>   reshape wide value, i(j1) j(j2)
>
>   if (`first_pass') {
>       save `Accumulator'
>       local first_pass 0
>   }
>   else {
>       sort j1
>       merge j1 using `Accumulator'
>       assert _merge == 3
>       drop _merge
>       sort j1
>      save `Accumulator', replace
>   }
>
> }
>
> use `Accumulator', clear
> exit
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index