Thanks a lot for this suggestion. I am not sure whether I need equally
spaced intervals for the density estimate (this seems to be standard). I
ended up doing
kdensity x, n(1000) gen(grid dens)
sort grid
gen density =.
forvalues i = 2/19426 {
qui count if grid < x[`i']
qui replace density = (dens[r(N)] + dens[r(N)+1])/2 in `i'
}
While this approach works, it turns out that it takes nearly as long as
computing the densitiy for all observations in the first place. In the
meantime, I tried this in EViews (with exactly the same data, bandwidth
and N) and found that density estimation and interpolation take about 3
seconds (!) in EViews, while Stata has about 10 Minutes overall. I was
very surprised by this huge difference in speed.
Thanks again and best wishes,
Eva
Nichols, Austin wrote:
> You could
> . sort x
> . gen y=x if mod(_n,20)==0 | _n==1 | _n==_N
> . kdensity x, at(y) gen(xdens)
> . ipolate xdens x, gen(f)
>
> for example.
>
> -----Original Message-----
> From: Eva Poen [mailto:[email protected]]
> Sent: Tuesday, November 16, 2004 10:55 AM
> To: [email protected]
> Subject: st: Kernel density estimation in a large dataset
>
>
> Dear all,
>
> I want to do Kernel density estimation and local polynomial regression
> on a dataset with 20'000 observations using Stata 8.2. Computations
> using all
> observations as a grid, like in
>
> - kdensity x, at(x) gen(xdens) -
>
> take quite a long time (between 10 and 15 minutes each). So I would
> like to use a grid of, say, 1000 points, but still have density
> estimates for all my observations. That is, I want to have a variable
> xdens which contains in observation i
>
> - the exact estimated density if x[i] happens to be a grid point
> - the linear interpolation of the two densities estimated at the the
> closest grid points to the left and right of x[i]
>
> for all 20'000 observations. I was told that this is the default
> behaviour in EViews, but I have really no clue how to best implement
> this in Stata.
>
> Thanks a lot for any suggestions.
> Best regards,
>
> Eva Poen
>
> *
> * For searches and help try:
> * http://www.stata.com/support/faqs/res/findit.html
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
