Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: change the format of the pseudo log-likelihood in gb2fit


From   Nick Cox <njcoxstata@gmail.com>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: change the format of the pseudo log-likelihood in gb2fit
Date   Fri, 6 Jul 2012 22:38:07 +0100

Thanks for clarifying what you want. On your questions others may have
answers; here are mine.

1. My guess is No. This concerns not programs like -lognfit- but -ml-
that does the work. This is just a guess.

2. Michal has in effect already addressed this. You need to write your
own code to get what you want.

3. No comment.

Nick

On Fri, Jul 6, 2012 at 9:47 PM, Lucia R.Latino
<Latino@economia.uniroma2.it> wrote:
> Dear Nick,
>
> I am sorry if I was not clear in my previous email. I will try to explain
> here better.
>
> I need to choose the best fit for my data on consumption and I am using
> survey data.
>
> I have been using the programs written by Stephen Jenkins (e.g. gb2fit,
> smfit, dagumfit, lognfit) using the option svy.
>
> Stata shows the log pseudolikelihood with the scientific notation and it
> does not store the loglikelihood in e(ll).
>
> I used the code kindly suggested from Michal Brzezinski to recover the log
> pseudolikelihood for the gb2fit and I tried your suggestion of using
> -display- with a different format on r(sum). It worked.
>
> In order to get the full likelihood also for the other distributions (e.g.
> lognormal, Singh-Maddala, Dagum, Fisk, beta2) following the procedure above,
> I would need, as you said, to write a code for the log of each distribution
> functions. It's a little bit complicated (for me, at least).
>
> So, I was wondering if:
> 1.  there is a way to change the format of the output of -lognfit-, -smfit-,
> -dagumfit-, -fisk-, -beta2- in such way that the iterations of the log
> pseudolikelihood  are shown not in scientific notation; or
> 2. there is an easier way to recover the log pseudolikelihood  and store it
> after I run each command to fit the distribution; or
> 3. there is someone who can help me writing the code for the log of each
> distribution functions (which I understand it may be just asking too much),
> similarly to what Michal Brzezinski did for the gb2.
>
> Best,
> Lucia
>
>
>
>
>
> -----Messaggio originale-----
> Da: owner-statalist@hsphsun2.harvard.edu
> [mailto:owner-statalist@hsphsun2.harvard.edu] Per conto di Nick Cox
> Inviato: venerdì 6 luglio 2012 20:19
> A: statalist@hsphsun2.harvard.edu
> Oggetto: Re: st: change the format of the pseudo log-likelihood in gb2fit
>
> The last bit involves taking logarithms of some slightly messy expressions.
> That's all.
>
> On your first point, I don't understand what you are missing. If the final
> log-likelihoods for different models are the same to 4 significant figures,
> I suggest that they are to be regarded as identical in practice.
> Alternatively, I guess that least one of the following statements is true.
>
> 1. You didn't try my suggestion of using -display- with a different format
> on r(sum).
>
> 2. You did try it, but ran into some problem, in which case you need to
> spell out what that problem is.
>
> 3. You want something else, but again you need to spell out what that is.
> That said, it seems very unlikely that anyone will rewrite these programs
> for you.
>
> Nick
>
> On Fri, Jul 6, 2012 at 6:59 PM, Lucia R.Latino <Latino@economia.uniroma2.it>
> wrote:
>> Dear Nick,
>>
>> thanks for your hint.
>>
>> I am trying to compare different nested distributions and  I would
>> love to compare the log likelihood. With the scientific notation I
>> cannot see any difference. That is the reason why I was trying to
>> obtain the full log likelihood.
>>
>> Combine Michael's code and -display- I am able to get the full log
>> likelihood. But I haven't been able to change how stata display the
>> result after gb2fit or similar command.
>>
>> Does anyone can help me in writing the explicit expression for
>> log-likelihood for lognfit, dagumfit, smfit, fisk and Beta2 similar to
>> what Michal wrote for the gb2fit?
>>
>> Thanks,
>> Lucia
>>
>> -----Messaggio originale-----
>> Da: owner-statalist@hsphsun2.harvard.edu
>> [mailto:owner-statalist@hsphsun2.harvard.edu] Per conto di Nick Cox
>> Inviato: venerdì 6 luglio 2012 11:58
>> A: statalist@hsphsun2.harvard.edu
>> Oggetto: Re: st: change the format of the pseudo log-likelihood in
>> gb2fit
>>
>> -gb2fit- is a user-written program from SSC written by Stephen Jenkins.
>>
>> -display- takes a format which can be used to change how a result is
>> displayed. Look at the help for -display- and for -format-. However,
>> it is difficult to see that further decimal places could be of either
>> use or interest.
>>
>> Nick
>>
>> On Fri, Jul 6, 2012 at 10:41 AM, Lucia R.Latino
>> <Latino@economia.uniroma2.it> wrote:
>>> Dear Michal and Stephen,
>>>
>>> Thanks for your answer. I tried Michal's code and it actually
>>> recovers the log pseudolikelihood, however stata shows the value
>>> using again the scientific notation. Do you have any suggestion?
>>> Furthermore, I am not sure I am able to write the same code for the
>>> other distributions I am using (lognfit, smfit, dagumfit, fisk,
>>> beta2). Do you think there is a way to simply change the format of
>>> the Iteration so to have more precision?
>>>
>>> If it can helps I copied the result I obtained.
>>>
>>> Thanks,
>>> Lucia
>>>
>>> -----------------------------------------------------------start
>>> output
>>> ---------------------------------------------------------------------
>>> -
>>> ------
>>> ------------
>>> gb2fit dec_ae, cdf(gb2cdf) pdf(gb2pdf) svy
>>>
>>> initial:       log pseudolikelihood =     -<inf>  (could not be
> evaluated)
>>> feasible:      log pseudolikelihood = -1.469e+08
>>> rescale:       log pseudolikelihood = -1.469e+08
>>> rescale eq:    log pseudolikelihood = -1.075e+08
>>> Iteration 0:   log pseudolikelihood = -1.075e+08  (not concave)
>>> Iteration 1:   log pseudolikelihood = -1.064e+08  (not concave)
>>> Iteration 2:   log pseudolikelihood = -1.063e+08
>>> Iteration 3:   log pseudolikelihood = -1.060e+08  (not concave)
>>> Iteration 4:   log pseudolikelihood = -1.060e+08
>>> Iteration 5:   log pseudolikelihood = -1.059e+08
>>> Iteration 6:   log pseudolikelihood = -1.059e+08
>>> Iteration 7:   log pseudolikelihood = -1.059e+08
>>> Iteration 8:   log pseudolikelihood = -1.059e+08
>>> Iteration 9:   log pseudolikelihood = -1.059e+08
>>> Iteration 10:  log pseudolikelihood = -1.059e+08 Iteration 11:  log
>>> pseudolikelihood = -1.059e+08 Iteration 12:  log pseudolikelihood =
>>> -1.059e+08
>>>
>>> ML fit of GB2 distribution
>>>
>>> pweight:  iwght                                   Number of obs    =
>>> 11183
>>> Strata:   <one>                                   Number of strata =
>>> 30
>>> PSU:      <observations>                          Number of PSUs   =
>>> 564
>>>                                                   Population size  =
>>> 12272397
>>>                                                   F(   0,    535)  =
>>> .
>>>                                                   Prob > F         =
>>> .
>>>
>>> ---------------------------------------------------------------------
>>> -
>>> ------
>>> --
>>>       dec_ae |      Coef.   Std. Err.      t    P>|t|     [95% Conf.
>>> Interval]
>>> -------------+-------------------------------------------------------
>>> -------------+-
>>> -------------+------
>>> --
>>> a            |
>>>        _cons |   2.327935   .2782045     8.37   0.000     1.781426
>>> 2.874444
>>> -------------+-------------------------------------------------------
>>> -------------+-
>>> -------------+------
>>> --
>>> b            |
>>>        _cons |   3997.361   250.2252    15.98   0.000     3505.814
>>> 4488.907
>>> -------------+-------------------------------------------------------
>>> -------------+-
>>> -------------+------
>>> --
>>> p            |
>>>        _cons |   1.603659   .2993526     5.36   0.000     1.015606
>>> 2.191712
>>> -------------+-------------------------------------------------------
>>> -------------+-
>>> -------------+------
>>> --
>>> q            |
>>>        _cons |   2.983855   .6731986     4.43   0.000     1.661412
>>> 4.306297
>>> ---------------------------------------------------------------------
>>> -
>>> ------
>>> --
>>>
>>>  tempvar ll
>>> local wv= "`e(wvar)'"
>>>
>>>  gen `ll' = lngamma(e(bp)+e(bq)) + log(e(ba)) + (e(ba)*e(bp)-1)*
>>> log(dec_ae) ///
>>>          - e(ba)*e(bp)*log(e(bb)) - lngamma(e(bp)) - lngamma(e(bq)) - ///
>>>          (e(bp)+e(bq))*log(1+(dec_ae/e(bb))^e(ba))
>>>
>>>  qui sum `ll' [aw=`wv'] if e(sample),meanonly di r(sum)
>>>
>>> -1.059e+08
>>> -----------------------------------------------------------end output
>>> ---------------------------------------------------------------------
>>> -
>>> ------
>>> ------------
>>>
>>> Lucia
>>>
>>>
>>> -----Messaggio originale-----
>>> Da: owner-statalist@hsphsun2.harvard.edu
>>> [mailto:owner-statalist@hsphsun2.harvard.edu] Per conto di Michal
>>> Brzezinski
>>> Inviato: giovedì 5 luglio 2012 23:15
>>> A: statalist@hsphsun2.harvard.edu
>>> Oggetto: Re: st: change the format of the pseudo log-likelihood in
>>> gb2fit
>>>
>>> You can recover pseudo log-likelihood writing an explicit expression
>>> for log-likelihood for a given model.
>>> For example, in case of GB2 model you could try the following code:
>>>
>>> gb2fit dec, cdf(gb2cdf) pdf(gb2pdf) svy tempvar ll local wv=
>>> "`e(wvar)'"
>>>
>>> gen `ll' = lngamma(e(bp)+e(bq)) + log(e(ba)) + (e(ba)*e(bp)-1)*
>>> log(dec)
>> ///
>>>         - e(ba)*e(bp)*log(e(bb)) - lngamma(e(bp)) - lngamma(e(bq)) - ///
>>>         (e(bp)+e(bq))*log(1+(dec/e(bb))^e(ba))
>>> qui sum `ll' [aw=`wv'] if e(sample),meanonly di r(sum)
>>>
>>> ----------
>>> Hope this helps,
>>> Michal
>>>
>>> 2012/7/5 Lucia Latino <Latino@economia.uniroma2.it>:
>>>> Dear Statalist,
>>>>
>>>> After running - gb2fit - smfit - lognfit - Stata reports the pseudo
>>>> log-likelihood with the scientific notation, but I want to view the
>>>> full likelihood.
>>>> Usually, I can obtain the full log-likelihood by using - display
>>>> e(ll)
>>>> -
>>>>
>>>> However, I need to add the option svy for the estimation (e.g. -
>>>> gb2fit dec, stats cdf(gb2cdf) pdf(gb2pdf) svy - ) and after adding
>>>> the option 'svy', Stata doesn't store anymore the pseudo
>>>> log-likelihood in
>>> e(ll).
>>>>
>>>> Is there any way to do change the format of the pseudo
>>>> log-likelihood in the iterations or a way to obtain it after the
> estimation?

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index