Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: slow execution of duplicates drop varlist, force


From   Nick Cox <njcoxstata@gmail.com>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: slow execution of duplicates drop varlist, force
Date   Tue, 26 Jun 2012 16:30:56 +0100

You can see for yourself what -duplicates- does by looking at the code.

. viewsource duplicates.ado

There is some scaffolding and some extra stuff that supports other
things that -duplicates- can do. It is not optimised for the case in
which you know that you want to -drop- in the way that you do; in
particular it is designed to protect people losing information by
checking what's what.

I am surprised at "much faster", however.

Nick

On Tue, Jun 26, 2012 at 4:16 PM, Kristoffer Milonas
<Kristoffer.Milonas@hhs.se> wrote:

> I wonder why the syntax duplicates drop varlist,force takes so long time on large datasets compared to doing it manually.
>
> Suppose for instance that varlist is groupvar1 groupvar2 ... groupvark. It seems to me that I can then drop duplicates manually by typing
> sort groupvar1 ... groupvark
> keep if groupvar1!=groupvar1[_n-1] | ... | groupvark!=groupvark[_n-1].
>
> The latter is much faster when I try (using both versions 10 and 11 of Stata). Am I missing some finer points that the duplicates drop command takes into account?
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index