Re: st: new package -fastxtile- available in SSC

[David Muller <davidmull@gmail.com>]

Hi Michael,

This looks great, and it is certainly much faster than built in
-xtile- when operating on a lot of observations!

One thing to note is that -fastxtile- does not necessarily produce
identical results to -xtile-. This seems to occur for values that are
essentially equal to a quantile cutpoint:

**************************************
clear
set seed 300
set obs 10
gen x = rnormal()
fastxtile fxt = x, nq(6)
xtile xt = x, nq(6)
assert xt == fxt
list if xt != fxt

// And a larger example
clear
set obs 10000000
gen x = rnormal()
fastxtile fxt = x, nq(6)
xtile xt = x, nq(6)
assert xt == fxt
list if xt != fxt
**************************************


All the best,
David

On 6 October 2013 23:02, Michael Stepner <stepner@mit.edu> wrote:
> -fastxtile- is a Stata routine to create a variable of quantile
> categories.  It is now available in the SSC, with thanks to Kit Baum.
>
> fastxtile is a drop in replacement for the built-in Stata program
> xtile. It has the same syntax and produces identical results, but the
> process has been altered to be more computationally efficient.  The
> difference in running time is substantial in large datasets.
>
> fastxtile also has a few added features.  It supports computing the
> quantile boundaries using a random sample of the data, which further
> increases the speed, but generates approximate quantiles due to
> sampling error.  fastxtile can also create categories based on a
> user-specified numlist, rather than computing the quantile boundaries
> itself.
>
> For anyone currently using -xtile- with large datasets, -fastxtile- is
> worth checking out.  It has no downside, and runs significantly
> faster.
>
> If you're interested, you can install the program via -ssc install fastxtile-.
>
> Best regards,
> Michael
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/