Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: st: U.S. Census Data


From   "Martin Weiss" <martin.weiss1@gmx.de>
To   <statalist@hsphsun2.harvard.edu>
Subject   RE: st: U.S. Census Data
Date   Thu, 6 May 2010 21:25:13 +0200

<>

" If you need to -reshape- your data set
this way or that way, it may also be problematic, although I don't
know specifically what the memory requirements are."


-reshape long- in this example requires about 72% of memory to be free. You
can play with the # of obs in this code: 2.000.000 is the maximum that runs
with 200m allocated.

***********
clear*
set mem 200m
set obs 2000000

gen id=_n
gen byte sex=runiform()<.5

foreach var of newlist ue80-ue83{
	gen byte `var'=runiform()<.1
}

foreach var of newlist inc80-inc83{
	gen `var'=rnormal(7500,10000)
}

d,s

reshape long inc ue, i(id) j(year)

d,s
***********


HTH
Martin

-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu
[mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Stas Kolenikov
Sent: Donnerstag, 6. Mai 2010 20:16
To: statalist@hsphsun2.harvard.edu
Subject: Re: st: U.S. Census Data

On Thu, May 6, 2010 at 10:03 AM, Martin Weiss <martin.weiss1@gmx.de> wrote:
> But careful with such examples: They do not say much about Nate`s problem
as
> you are creating the default data type after -gen- which is "float". It
> occupies 4 bytes, as in
http://www.stata.com/support/faqs/data/howbig.html.
> Strings in particular could change the picture.

As you data set is so thin, you also need to worry about the overheads
of statistical and data management procedures. If a particular -ml-
routine wants to create 10 temp variables, and you request -robust-
option that needs scores for every equation, you may see your memory
requirements double or triple. If you need to -reshape- your data set
this way or that way, it may also be problematic, although I don't
know specifically what the memory requirements are.

-- 
Stas Kolenikov, also found at http://stas.kolenikov.name
Small print: I use this email account for mailing lists only.
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index