Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.

Re: st: Guidance on matrix inversion for OLS in mata

 From Thomas Jacobs To statalist@hsphsun2.harvard.edu Subject Re: st: Guidance on matrix inversion for OLS in mata Date Tue, 27 Apr 2010 11:31:52 -0500

```Well I totally screwed this up.  I should be doing invsym[X'X].  No
wonder I can't get any results.  While I am sorry for the bother, I am
not sure I would have caught this had I not taken the time to write

Tom

On Tue, Apr 27, 2010 at 11:17 AM, Thomas Jacobs <thomasjacobs@gmail.com> wrote:
> Austin,
>
> Thanks for the reply.  I guess I have no choice but to do as you
> suggest or else rewrite the program entirely in Stata.  I don't have
> the days it always takes me to decipher how to do the stata mata data
> transfer for this project at present.  Every time I sit down to do
> something like this I always feel it a roll of the dice.  Do I go the
> mata route and discover what should have been done in stata or go the
> stata route and discover the opposite!
>
> Here is a short excerpt of the problem.  I am simply trying to compute
> x'x inverse for a 100 trading day time series of index returns and use
> it to solve for betahat for a similar time series of firm returns
> where the latter often has one or more missing values.
>
> Here are the initial steps I take in mata and will use an example with
> no missing values:
>
> X=J(100,2,1)
> X[.,2]=LnBAATreasSprd[800::899]
> : X
>                    1              2
>      +-------------------------------+
>    1 |             1    -.005838091  |
>    2 |             1   -.0063325767  |
>    3 |             1    .0100309728  |
>    4 |             1    .0040479107  |
>    5 |             1     .000136131  |
>    6 |             1      .00040839  |
>    7 |             1   -.0148075884  |
>    8 |             1   -.0107393684  |
>    9 |             1    -.002842935  |
>   10 |             1    .0066983248  |
>   11 |             1   -.0093153818  |
>   12 |             1   -.0502348617  |
>   13 |             1   -.0178726967  |
>   14 |             1    .0145704737  |
>   15 |             1   -.0073333359  |
>   16 |             1   -.0029384219  |
>   17 |             1   -.0095219389  |
>   18 |             1   -.0128216762  |
>   19 |             1   -.0150571838  |
>   20 |             1     .005833914  |
>   21 |             1    .0069765295  |
>   22 |             1    .0038266596  |
>   23 |             1   -.0067447214  |
>   24 |             1   -.0082368711  |
>   25 |             1    .0079292208  |
>   26 |             1   -.0185827948  |
>   27 |             1   -.0081839114  |
>   28 |             1    .0103758555  |
>   29 |             1    -.007115229  |
>   30 |             1   -.0043146866  |
>   31 |             1    .0007379653  |
>   32 |             1   -.0010016384  |
>   33 |             1    .0068858997  |
>   34 |             1   -.0044625248  |
>   35 |             1    .0031520503  |
>   36 |             1   -.0554844476  |
>   37 |             1    .0113020828  |
>   38 |             1    .0031746107  |
>   39 |             1    .0117888227  |
>   40 |             1   -.0010267079  |
>   41 |             1    .0095236665  |
>   42 |             1    .0261100251  |
>   43 |             1   -.0126527818  |
>   44 |             1   -.0114778923  |
>   45 |             1    .0066052717  |
>   46 |             1   -.0084775724  |
>   47 |             1   -.0032715374  |
>   48 |             1   -.0015591006  |
>   49 |             1    .0142634539  |
>   50 |             1    .0003712231  |
>   51 |             1   -.0077179475  |
>   52 |             1    .0082479911  |
>   53 |             1    .0071813553  |
>   54 |             1    .0058756177  |
>   55 |             1   -.0082988022  |
>   56 |             1   -.0003692649  |
>   57 |             1   -.0046538925  |
>   58 |             1   -.0049419519  |
>   59 |             1    .0040403828  |
>   60 |             1   -.0099174296  |
>   61 |             1    .0231989622  |
>   62 |             1   -.0097859399  |
>   63 |             1   -.0028060512  |
>   64 |             1   -.0028140107  |
>   65 |             1    .0092088645  |
>   66 |             1    .0032609063  |
>   67 |             1    .0080019441  |
>   68 |             1    .0002604046  |
>   69 |             1    .0027563504  |
>   70 |             1   -.0134371053  |
>   71 |             1    .0051452187  |
>   72 |             1    .0188821666  |
>   73 |             1    .0129679879  |
>   74 |             1    .0097926175  |
>   75 |             1    .0015559109  |
>   76 |             1   -.0006521898  |
>   77 |             1    .0055551799  |
>   78 |             1    .0016954662  |
>   79 |             1              0  |
>   80 |             1   -.0011965502  |
>   81 |             1   -.0123975417  |
>   82 |             1    .0008581794  |
>   83 |             1    .0113398973  |
>   84 |             1   -.0044504236  |
>   85 |             1   -.0101748193  |
>   86 |             1    .0006073291  |
>   87 |             1    .0028294155  |
>   88 |             1   -.0016158026  |
>   89 |             1    -.009393123  |
>   90 |             1   -.0073215333  |
>   91 |             1    .0056367237  |
>   92 |             1   -.0044041635  |
>   93 |             1    .0046085101  |
>   94 |             1    .0162160564  |
>   95 |             1    .0047139199  |
>   96 |             1   -.0005505134  |
>   97 |             1    .0033981025  |
>   98 |             1     .034180887  |
>   99 |             1   -.0102245966  |
>  100 |             1   -.0060580331  |
>      +-------------------------------+
>
> and the inverse step with a partial excerpt (all values are zero save
> for three cells):
> : invsym(X*X')
> [symmetric]
>                    1              2              3              4
>         5
>      +----------------------------------------------------------------------------
>    1 |             0
>    2 |             0              0
>    3 |             0              0              0
>    4 |             0              0              0              0
>    5 |             0              0              0              0
>         0
>    6 |             0              0              0              0
>         0
>    7 |             0              0              0              0
>         0
>    8 |             0              0              0              0
>         0
>    9 |             0              0              0              0
>         0
>   10 |             0              0              0              0
>         0
>   11 |             0              0              0              0
>         0
>   12 |             0              0              0              0
>         0
>   13 |             0              0              0              0
>         0
>   14 |             0              0              0              0
>         0
>   15 |             0              0              0              0
>         0
>   16 |             0              0              0              0
>         0
>   17 |             0              0              0              0
>         0
>   18 |             0              0              0              0
>         0
>   19 |             0              0              0              0
>         0
>   20 |             0              0              0              0
>         0
>   21 |             0              0              0              0
>         0
>   22 |             0              0              0              0
>         0
>   23 |             0              0              0              0
>         0
>   24 |             0              0              0              0
>         0
>   25 |             0              0              0              0
>         0
>   26 |             0              0              0              0
>         0
>   27 |             0              0              0              0
>         0
>   28 |             0              0              0              0
>         0
>   29 |             0              0              0              0
>         0
>   30 |             0              0              0              0
>         0
>   31 |             0              0              0              0
>         0
>   32 |             0              0              0              0
>         0
>   33 |             0              0              0              0
>         0
>   34 |             0              0              0              0
>         0
>   35 |             0              0              0              0
>         0
>   36 |             0              0              0              0
>         0
>   37 |             0              0              0              0
>         0
>   38 |             0              0              0              0
>         0
>   39 |             0              0              0              0
>         0
>   40 |             0              0              0              0
>         0
>
> qrinv gives something similar with a few more non-zero values while
> cholinv and luinv produce nulls.  pinv does work in this case.
> Suffice it to say I have no problem regressing a similar single firm
> return series on this index for the time period in question using
> regress.
>
> If I am missing something, please let me know.  Otherwise, I guess I
> need to go to stata or do a better job of reproducing regress!  Thanks
> again.
>
> Tom
> On Tue, Apr 27, 2010 at 10:23 AM, Austin Nichols
> <austinnichols@gmail.com> wrote:
>>
>> Thomas Jacobs <thomasjacobs@gmail.com>:
>> Zero is not a problem, but you should expunge the missings first;
>> however you seem to be trying to rewrite -regress- as you go, which is
>> far from a good idea. Why *not* export your vectors to Stata and run
>> -regress- and let Stata handle the sample selection and matrix
>> inversion for you?
>>
>> Maybe if you give us a simple example with real numbers, the problem
>> will be clearer and you can get better guidance...
>>
>> On Tue, Apr 27, 2010 at 12:45 AM, Thomas Jacobs <thomasjacobs@gmail.com> wrote:
>> > Hi,
>> >
>> > I am trying to perform a lengthy series of simulations to examine some
>> > event study methodologies.  I have moved to mata for the bulk of the
>> > work but find that for those cases where I wish to use a market model
>> > approach requiring an OLS regression to establish abnormal returns I
>> > am unable to generate an inverse for x'x in seeking to solve for beta
>> > hat.  I am typically working with vectors that have 1. missing values,
>> > 2. zero values, and 3. very small values close to zero (within a
>> > couple of decimal places such as -.01 or .005).  I have tried mata's
>> > cholinv, invsym, pinv, luinv, and qrinv (I realize that some of these
>> > are probably inappropriate for my problem but I am no expert) and
>> > generally get an inverse matrix of missing values or bizarre results
>> > like a single populated row.
>> >
>> > I would prefer not to go back and forth between stata and mata to use
>> > the stata regress function unless that is the only way to accomplish
>> > this effort.
>> >
>> > Can anyone offer general guidance on how to proceed here?  Thanks.
>> >
>> > Tom
>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>
>
>
> --
> Thomas Jacobs
>

--
Thomas Jacobs

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```