Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

st: RE: may I run -lowess- in a faster way?


From   "Nick Cox" <n.j.cox@durham.ac.uk>
To   <statalist@hsphsun2.harvard.edu>
Subject   st: RE: may I run -lowess- in a faster way?
Date   Thu, 18 Sep 2008 19:06:09 +0100

Don't do that then! 

<http://catb.org/jargon/html/D/Don-t-do-that-then-.html> 

. u auto
(1978 Automobile Data)

. set rmsg on
r; t=0.00 18:50:22

. set more off
r; t=0.02 18:50:35

. expand 1000
(73926 observations created)
r; t=0.06 18:50:40

. lowess mpg weight
r; t=659.56 19:01:47

. rcspline mpg weight
r; t=7.58 19:02:17

i.e. -rcspline- is ~100 times faster, in one experiment. 

Install -rcspline- from -ssc-. Requires version 10. 

Nick 
n.j.cox@durham.ac.uk 

josemaria

I have a large data set and used the command -lowess-, which took 9.5 
minutes to run using art of the data. I am using a notebook with Windows
XP 
Professional, a processor Intel Pentium 1.73 GHz, 1.99 GB RAM.
For the specifications bellow, is it possible to get more speed, mainly
to 
analyse a  less restrict piece of data within the dataset? By the way, I

know that -lowess- is time consuming.

set mem 75m
(76800k)
. use "C:\ROSANGELA\doutorado\trabalho2.dta"
. des
Contains data from C:\ROSANGELA\doutorado\trabalho2.dta
  obs:       272,704
 vars:            32                          14 Sep 2008 20:26
 size:    41,723,712 (46.9% of memory free)


memory
                                                  bytes
--------------------------------------------------------------------
Details of set memory usage
    overhead (pointers)                       1,090,816        1.39%
    data                                            40,632,896
51.67%
                                        ----------------------------
    data + overhead                          41,723,712       53.05%
    free                                             36,919,480
46.95%
                                        ----------------------------
    Total allocated                             78,643,192      100.00%
--------------------------------------------------------------------
Other memory usage
    system overhead                           1,600,258
    set matsize usage                              337,600
    programs, saved results, etc.               17,351
                                        ---------------
    Total                                             1,955,209
-------------------------------------------------------
Grand total                                      80,598,401

count if sexo01==1 & idad_pesm<=12 & ordempesagem<=5
46355

lowess  est_wazw idad_pesm if sexo01==1 & idad_pesm<=12 &
ordempesagem<=5
r; t=570.88 13:50:31

. dis 570/60
9.5


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index