Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, statalist.org is already up and running.

# Re: st: xtlogit: panel data transformation's recast to double makes model incomputable

 From Nick Cox To "statalist@hsphsun2.harvard.edu" Subject Re: st: xtlogit: panel data transformation's recast to double makes model incomputable Date Wed, 3 Apr 2013 17:10:28 +0100

So, it is not a problem with zero or negative values. I don't have an

I don't think anyone needs a test to choose a transformation with 2
million values that are manifestly highly skew.

Your -summarize- results give min, max and nine quantiles in between.
Transformations of interest are here all monotonic, so preserve order,
so it is sufficient just to fire up Mata to look at what quantiles
become, e.g.

: y
1
+----------+
1 |     .02  |
2 |     .62  |
3 |    1.66  |
4 |    3.09  |
5 |    8.66  |
6 |   18.68  |
7 |   33.48  |
8 |   57.38  |
9 |   74.08  |
10 |  123.49  |
11 |  785.37  |
+----------+

: log10(y)
1
+----------------+
1 |  -1.698970004  |
2 |  -.2076083105  |
3 |    .220108088  |
4 |   .4899584794  |
5 |    .937517892  |
6 |   1.271376872  |
7 |   1.524785449  |
8 |   1.758760544  |
9 |   1.869700974  |
10 |   2.091631791  |
11 |   2.895074308  |
+----------------+

: sqrt(y)
1
+---------------+
1 |  .1414213562  |
2 |  .7874007874  |
3 |  1.288409873  |
4 |  1.757839583  |
5 |  2.942787794  |
6 |  4.322036557  |
7 |  5.786190457  |
8 |  7.574958746  |
9 |  8.606973917  |
10 |  11.11260546  |
11 |  28.02445361  |
+---------------+

A glance shows that roots are not strong enough, but logarithms do
much better, which matches a lot in economic thinking. However, it's
one thing to make a distribution more symmetric; the question of how
it behaves in a model remains.

Nick

On 3 April 2013 16:09, Tom <tommedema@gmail.com> wrote:
> "close" is a price, and therefore it is always positive. -sum close,
> detail- follows (after dropping some outliers in which I am not
> interested):
> http://pastebin.com/25gm94i0
>
> close- issue. It turns out that when I supply -noadjust-, i.e. execute
> http://pastebin.com/m8Z4YmKd
>
> -ladder-. The help file says that this is what it does:
>
> (1991) to the overall chi-squared and its significance level and
> presents the unaltered test as described by D'Agostino, Balanger, and
> D'Agostino (1990)."
>
> But does anyone have a clue why this adjustment would prevent ladder
> from returning anything? Also see the results of -sktest close- versus
> http://pastebin.com/tPUf3xE6
>
> Tom
>
> On Wed, Apr 3, 2013 at 4:54 PM, Nick Cox <njcoxstata@gmail.com> wrote:
>> The transformations that -ladder- tries are typically only defined for
>> positive variables. As I recall, many of your variables are variously
>> positive and negative.
>>
>> The usual transformations for right-skewed variables, such as square
>> root and logarithmic, are thus no use in your case.
>>
>> This is precisely why cube root and inverse hyperbolic functions were
>> suggested earlier/
>>
>> On the face of it that is not an explanation of why all the
>> transformations fail, but I'd not be inclined to think that -ladder-
>>
>> I'm afraid that you have keep reminding us of basics -- e.g. what is
>> -close- and what are its descriptive statistics -- as even those
>> following your threads can't be expected to keep track of all earlier
>> posts.
>>
>> Nick
>>
>> On 3 April 2013 15:33, Tom <tommedema@gmail.com> wrote:
>>
>>> Anyway, I think I may have a very important pointer that could help in
>>> finding the cause of this issue:
>>> when I run -ladder close- it returns absolutely nothing, i.e.:
>>>
>>>
>>> Transformation         formula               chi2(2)       P(chi2)
>>> ------------------------------------------------------------------
>>> cubic                  close^3                    .            .
>>> square                 close^2                    .            .
>>> identity               close                      .            .
>>> square root            sqrt(close)                .            .
>>> log                    log(close)                 .            .
>>> 1/(square root)        1/sqrt(close)              .            .
>>> inverse                1/close                    .            .
>>> 1/square               1/(close^2)                .            .
>>> 1/cubic                1/(close^3)                .            .
>>>
>>> I believe this is very much related to the problems I am having with
>>> this variable, close. If I can determine why ladder refuses to give me
>>> any transformations, then I may be able to figure out what is the
>>> source of this issue.
>>>
>>> I am not just trying to get an answer, I am interested in why ladder
>>> is not returning any transformations because this may give me
>>> information on the root cause of the problem.
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/faqs/resources/statalist-faq/
>> *   http://www.ats.ucla.edu/stat/stata/
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/

--
Nick
njcoxstata@gmail.com
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/