Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: change the format of the pseudo log-likelihood in gb2fit
From
Nick Cox <[email protected]>
To
[email protected]
Subject
Re: st: change the format of the pseudo log-likelihood in gb2fit
Date
Fri, 6 Jul 2012 22:38:07 +0100
Thanks for clarifying what you want. On your questions others may have
answers; here are mine.
1. My guess is No. This concerns not programs like -lognfit- but -ml-
that does the work. This is just a guess.
2. Michal has in effect already addressed this. You need to write your
own code to get what you want.
3. No comment.
Nick
On Fri, Jul 6, 2012 at 9:47 PM, Lucia R.Latino
<[email protected]> wrote:
> Dear Nick,
>
> I am sorry if I was not clear in my previous email. I will try to explain
> here better.
>
> I need to choose the best fit for my data on consumption and I am using
> survey data.
>
> I have been using the programs written by Stephen Jenkins (e.g. gb2fit,
> smfit, dagumfit, lognfit) using the option svy.
>
> Stata shows the log pseudolikelihood with the scientific notation and it
> does not store the loglikelihood in e(ll).
>
> I used the code kindly suggested from Michal Brzezinski to recover the log
> pseudolikelihood for the gb2fit and I tried your suggestion of using
> -display- with a different format on r(sum). It worked.
>
> In order to get the full likelihood also for the other distributions (e.g.
> lognormal, Singh-Maddala, Dagum, Fisk, beta2) following the procedure above,
> I would need, as you said, to write a code for the log of each distribution
> functions. It's a little bit complicated (for me, at least).
>
> So, I was wondering if:
> 1. there is a way to change the format of the output of -lognfit-, -smfit-,
> -dagumfit-, -fisk-, -beta2- in such way that the iterations of the log
> pseudolikelihood are shown not in scientific notation; or
> 2. there is an easier way to recover the log pseudolikelihood and store it
> after I run each command to fit the distribution; or
> 3. there is someone who can help me writing the code for the log of each
> distribution functions (which I understand it may be just asking too much),
> similarly to what Michal Brzezinski did for the gb2.
>
> Best,
> Lucia
>
>
>
>
>
> -----Messaggio originale-----
> Da: [email protected]
> [mailto:[email protected]] Per conto di Nick Cox
> Inviato: venerdì 6 luglio 2012 20:19
> A: [email protected]
> Oggetto: Re: st: change the format of the pseudo log-likelihood in gb2fit
>
> The last bit involves taking logarithms of some slightly messy expressions.
> That's all.
>
> On your first point, I don't understand what you are missing. If the final
> log-likelihoods for different models are the same to 4 significant figures,
> I suggest that they are to be regarded as identical in practice.
> Alternatively, I guess that least one of the following statements is true.
>
> 1. You didn't try my suggestion of using -display- with a different format
> on r(sum).
>
> 2. You did try it, but ran into some problem, in which case you need to
> spell out what that problem is.
>
> 3. You want something else, but again you need to spell out what that is.
> That said, it seems very unlikely that anyone will rewrite these programs
> for you.
>
> Nick
>
> On Fri, Jul 6, 2012 at 6:59 PM, Lucia R.Latino <[email protected]>
> wrote:
>> Dear Nick,
>>
>> thanks for your hint.
>>
>> I am trying to compare different nested distributions and I would
>> love to compare the log likelihood. With the scientific notation I
>> cannot see any difference. That is the reason why I was trying to
>> obtain the full log likelihood.
>>
>> Combine Michael's code and -display- I am able to get the full log
>> likelihood. But I haven't been able to change how stata display the
>> result after gb2fit or similar command.
>>
>> Does anyone can help me in writing the explicit expression for
>> log-likelihood for lognfit, dagumfit, smfit, fisk and Beta2 similar to
>> what Michal wrote for the gb2fit?
>>
>> Thanks,
>> Lucia
>>
>> -----Messaggio originale-----
>> Da: [email protected]
>> [mailto:[email protected]] Per conto di Nick Cox
>> Inviato: venerdì 6 luglio 2012 11:58
>> A: [email protected]
>> Oggetto: Re: st: change the format of the pseudo log-likelihood in
>> gb2fit
>>
>> -gb2fit- is a user-written program from SSC written by Stephen Jenkins.
>>
>> -display- takes a format which can be used to change how a result is
>> displayed. Look at the help for -display- and for -format-. However,
>> it is difficult to see that further decimal places could be of either
>> use or interest.
>>
>> Nick
>>
>> On Fri, Jul 6, 2012 at 10:41 AM, Lucia R.Latino
>> <[email protected]> wrote:
>>> Dear Michal and Stephen,
>>>
>>> Thanks for your answer. I tried Michal's code and it actually
>>> recovers the log pseudolikelihood, however stata shows the value
>>> using again the scientific notation. Do you have any suggestion?
>>> Furthermore, I am not sure I am able to write the same code for the
>>> other distributions I am using (lognfit, smfit, dagumfit, fisk,
>>> beta2). Do you think there is a way to simply change the format of
>>> the Iteration so to have more precision?
>>>
>>> If it can helps I copied the result I obtained.
>>>
>>> Thanks,
>>> Lucia
>>>
>>> -----------------------------------------------------------start
>>> output
>>> ---------------------------------------------------------------------
>>> -
>>> ------
>>> ------------
>>> gb2fit dec_ae, cdf(gb2cdf) pdf(gb2pdf) svy
>>>
>>> initial: log pseudolikelihood = -<inf> (could not be
> evaluated)
>>> feasible: log pseudolikelihood = -1.469e+08
>>> rescale: log pseudolikelihood = -1.469e+08
>>> rescale eq: log pseudolikelihood = -1.075e+08
>>> Iteration 0: log pseudolikelihood = -1.075e+08 (not concave)
>>> Iteration 1: log pseudolikelihood = -1.064e+08 (not concave)
>>> Iteration 2: log pseudolikelihood = -1.063e+08
>>> Iteration 3: log pseudolikelihood = -1.060e+08 (not concave)
>>> Iteration 4: log pseudolikelihood = -1.060e+08
>>> Iteration 5: log pseudolikelihood = -1.059e+08
>>> Iteration 6: log pseudolikelihood = -1.059e+08
>>> Iteration 7: log pseudolikelihood = -1.059e+08
>>> Iteration 8: log pseudolikelihood = -1.059e+08
>>> Iteration 9: log pseudolikelihood = -1.059e+08
>>> Iteration 10: log pseudolikelihood = -1.059e+08 Iteration 11: log
>>> pseudolikelihood = -1.059e+08 Iteration 12: log pseudolikelihood =
>>> -1.059e+08
>>>
>>> ML fit of GB2 distribution
>>>
>>> pweight: iwght Number of obs =
>>> 11183
>>> Strata: <one> Number of strata =
>>> 30
>>> PSU: <observations> Number of PSUs =
>>> 564
>>> Population size =
>>> 12272397
>>> F( 0, 535) =
>>> .
>>> Prob > F =
>>> .
>>>
>>> ---------------------------------------------------------------------
>>> -
>>> ------
>>> --
>>> dec_ae | Coef. Std. Err. t P>|t| [95% Conf.
>>> Interval]
>>> -------------+-------------------------------------------------------
>>> -------------+-
>>> -------------+------
>>> --
>>> a |
>>> _cons | 2.327935 .2782045 8.37 0.000 1.781426
>>> 2.874444
>>> -------------+-------------------------------------------------------
>>> -------------+-
>>> -------------+------
>>> --
>>> b |
>>> _cons | 3997.361 250.2252 15.98 0.000 3505.814
>>> 4488.907
>>> -------------+-------------------------------------------------------
>>> -------------+-
>>> -------------+------
>>> --
>>> p |
>>> _cons | 1.603659 .2993526 5.36 0.000 1.015606
>>> 2.191712
>>> -------------+-------------------------------------------------------
>>> -------------+-
>>> -------------+------
>>> --
>>> q |
>>> _cons | 2.983855 .6731986 4.43 0.000 1.661412
>>> 4.306297
>>> ---------------------------------------------------------------------
>>> -
>>> ------
>>> --
>>>
>>> tempvar ll
>>> local wv= "`e(wvar)'"
>>>
>>> gen `ll' = lngamma(e(bp)+e(bq)) + log(e(ba)) + (e(ba)*e(bp)-1)*
>>> log(dec_ae) ///
>>> - e(ba)*e(bp)*log(e(bb)) - lngamma(e(bp)) - lngamma(e(bq)) - ///
>>> (e(bp)+e(bq))*log(1+(dec_ae/e(bb))^e(ba))
>>>
>>> qui sum `ll' [aw=`wv'] if e(sample),meanonly di r(sum)
>>>
>>> -1.059e+08
>>> -----------------------------------------------------------end output
>>> ---------------------------------------------------------------------
>>> -
>>> ------
>>> ------------
>>>
>>> Lucia
>>>
>>>
>>> -----Messaggio originale-----
>>> Da: [email protected]
>>> [mailto:[email protected]] Per conto di Michal
>>> Brzezinski
>>> Inviato: giovedì 5 luglio 2012 23:15
>>> A: [email protected]
>>> Oggetto: Re: st: change the format of the pseudo log-likelihood in
>>> gb2fit
>>>
>>> You can recover pseudo log-likelihood writing an explicit expression
>>> for log-likelihood for a given model.
>>> For example, in case of GB2 model you could try the following code:
>>>
>>> gb2fit dec, cdf(gb2cdf) pdf(gb2pdf) svy tempvar ll local wv=
>>> "`e(wvar)'"
>>>
>>> gen `ll' = lngamma(e(bp)+e(bq)) + log(e(ba)) + (e(ba)*e(bp)-1)*
>>> log(dec)
>> ///
>>> - e(ba)*e(bp)*log(e(bb)) - lngamma(e(bp)) - lngamma(e(bq)) - ///
>>> (e(bp)+e(bq))*log(1+(dec/e(bb))^e(ba))
>>> qui sum `ll' [aw=`wv'] if e(sample),meanonly di r(sum)
>>>
>>> ----------
>>> Hope this helps,
>>> Michal
>>>
>>> 2012/7/5 Lucia Latino <[email protected]>:
>>>> Dear Statalist,
>>>>
>>>> After running - gb2fit - smfit - lognfit - Stata reports the pseudo
>>>> log-likelihood with the scientific notation, but I want to view the
>>>> full likelihood.
>>>> Usually, I can obtain the full log-likelihood by using - display
>>>> e(ll)
>>>> -
>>>>
>>>> However, I need to add the option svy for the estimation (e.g. -
>>>> gb2fit dec, stats cdf(gb2cdf) pdf(gb2pdf) svy - ) and after adding
>>>> the option 'svy', Stata doesn't store anymore the pseudo
>>>> log-likelihood in
>>> e(ll).
>>>>
>>>> Is there any way to do change the format of the pseudo
>>>> log-likelihood in the iterations or a way to obtain it after the
> estimation?
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/