Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, is already up and running.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Machine spec for 70GB data

From   Yuval Arbel <>
Subject   Re: st: Machine spec for 70GB data
Date   Sat, 22 Oct 2011 13:52:28 +0200


Are you sure the data file is 70GB? I'm using Windows operating system
and I recently succeded to run a file of  1.29 GB that includes above
4 million observations. Here are the few raws from the do file. Just
make sure to use the "set memory" command:

. do "D:\kingston\public_housing\"

. clear

. clear matrix

. set memory 12500m

. insheet using "D:\kingston\public_housing\survivalindexed27-Jun-2011.csv"
(56 vars, 4086490 obs)

. sort time_index

. stset time_index, id(appt) failure(fail==1)

                id:  appt
     failure event:  fail == 1
obs. time interval:  (time_index[_n-1], time_index]
 exit on or before:  failure

  4086490  total obs.
    32731  obs. end on or before enter()
  4053759  obs. remaining, representing
    49650  subjects
     8582  failures in single failure-per-subject data
  5084887  total analysis time at risk, at risk from t =         0
                             earliest observed entry t =         0
                                  last observed exit t =       114

On Sat, Oct 22, 2011 at 1:00 PM, Gindo Tampubolon
<> wrote:
> Dear all,
> I need to process a large data file [70GB; a few millions obs] with Stata 12 MP8. Mainly to do cross-random effects,individuals and hospitals, where the outcome is length of stay [controlling for no more than a handful of covariates to begin with]. As an approximation, the outcome is treated as continuous i.e. linear mixed models.
> What kind of machine spec would be needed? Any ideas, information, experience? Would operating system make any difference? I'm open to consider Windows, Linux, OS X.
> Many thanks,
> Gindo
> University of Manchester
> *
> *   For searches and help try:
> *
> *
> *

Dr. Yuval Arbel
School of Business
Carmel Academic Center
4 Shaar Palmer Street, Haifa, Israel

*   For searches and help try:

© Copyright 1996–2015 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index