Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: st: identifying duplicate records


From   Nick Cox <n.j.cox@durham.ac.uk>
To   "'statalist@hsphsun2.harvard.edu'" <statalist@hsphsun2.harvard.edu>
Subject   RE: st: identifying duplicate records
Date   Fri, 10 Feb 2012 16:49:50 +0000

Let's be clear on this: 

-duplicates- will fail to find "duplicates" that are not in fact duplicates. It nowhere and in no way is designed to find _approximate_ duplicates. 

Nick 
n.j.cox@durham.ac.uk 

Dimitriy V. Masterov

I assume you've already tried the duplicates command on various
combination of the id variables and that did not work.

I would create a combination id that concatenates the dob, NHS # and
surname. Then use the user-written strgroup on this variable. This
approach will still require a bit of manual work.

A non-Stata approach is to use Google Refine.

On Fri, Feb 10, 2012 at 9:08 AM, raoul reulen <r.c.reulen@gmail.com> wrote:

> Just wondering if I could get some advice. I have a large database
> with around 300,000 records of individuals. There can be more than one
> record per individual.  Now, how do I identify individuals? I assume
> that it is the same indivual if:
>
> Date of birth and NHS number are the same  OR
> date of birth and surname are the same OR
> surname and NHS number are the same.
>
> So there are various combinations possible. A date of birth could have
> typos in it; but if the NHS number and the surname are the same then I
> assume it is the same person. The NHS number can have typos, but if
> the date of birth and the surname are the same I will assume it is the
> same person.
>
>  What is the best way to approach this?  I want to end up with an
> id-number that identifies the individual.  Many thanks for your help.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index