# RE: st: Interesting numerical accuracy/collinearity issue

 From "Schaffer, Mark E" To Subject RE: st: Interesting numerical accuracy/collinearity issue Date Thu, 13 Apr 2006 13:00:22 +0100

More good info ... Thanks!

--Mark

> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu
> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of
> Rodrigo A. Alfaro
> Sent: 13 April 2006 02:08
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: Interesting numerical accuracy/collinearity issue
>
> Just in case: Numerical Recipes book is available on-line
>
> http://www.numerical-recipes.com/nronline_switcher.html
>
> Rodrigo.
>
>
> ----- Original Message -----
> From: "Stas Kolenikov" <skolenik@gmail.com>
> To: <statalist@hsphsun2.harvard.edu>
> Sent: Wednesday, April 12, 2006 5:45 PM
> Subject: Re: st: Interesting numerical accuracy/collinearity issue
>
>
> At your leisure time (probably a few years after retirement???...),
> you might want to check a reference like Numerical Recipes or Demmel's
> Applied Numerical Linear Algebra book from SIAM. The finite
> arithemtics space can be formally set up, and it is really strange;
> the series \sum 1/n converges in that space, for instance, according
> to the difference in partial sums criterion. So yes, you can think
> about linear algebra augmented by finite precision. Stata is generally
> aware of strange properties of that space, so commands like -_rmcoll-
> or  -issymmetric()- or -diag0cnt()- count the differences of matrix
> entries or eigenvalues from zero up to that finite precision. If there
> is a roundoff error introduced at some steps in shifting and scaling,
> -_rmcoll- would be still able to tell if there is collinearity, up to
> numerical accuracy of the X'X matrix.
>
> The general principles of finite precision arithmetics are generally
> based on condition numbers, which is, roughly, the largest possible
> change in the answer due to infinitesimal change in the inputs of the
> procedure. Invoking appropriate infinitesimal (mathematical rather
> than computer!) calculus, the condition numbers for many matrix
> operations like matrix inversions or determinants or linear systems
> can be shown to be the ratios of the largest to the smallest
> eigenvalues. Suppose this ratio for a your particular matrix is 10^4
> (which is not that huge; in -reg pri wei for trunk disp- with auto
> data, the condition number of the covariance matrix is 4e8). Then by
> taking your variables to the fourth power, you'll get that number to
> 10^16 (not quite so, but you can think of this as the worst case
> scenario), and that is already beyond the double arithmetics routinely
> employed (may be in the guts of Stata there is also quad arithmetics;
> I was coming across it a couple of times in Mata): epsdouble() =
> 2e-16, so a single blurp of that order leads to the change
> epsdouble()*condition# approx= order of one: you cannot trust even the
> variables are bad; and why you should center your variables; and why
> integrated processes lead to weird distributions... oops, that's
> another story, sorry :))
>
> On 4/12/06, Schaffer, Mark E <M.E.Schaffer@hw.ac.uk> wrote:
> > My follow-up question is simple: why does the shifting and
> scaling used by
> > Stata's
> > ‑ovtest‑ introduce greater accuracy rather than, say,
> greater rounding
> > error?  (Either
> > accuracy or error would remove the numerical collinearity.)
>  The algebra
> > doesn't help
> > me here, since all three methods are algebraically equivalent.  I'm
> > guessing that there's
> > probably a general principle about how best to maintain numerical
> > precision, but I don't
> > know what it might be.
>
>
> --
> Stas Kolenikov
> http://stas.kolenikov.name
>
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
>

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/