Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: tips on reading large matrix from ASCII file?


From   Jeph Herrin <jeph.herrin@yale.edu>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: tips on reading large matrix from ASCII file?
Date   Tue, 02 Dec 2008 15:59:19 -0500

Two things:

1. I ran the empty model (no predictors) using -xtmelogit-
    and it was still going after 24 hours. I have dozens of models
    to run; HLM6 did them all in less than 20 minutes total.
    Not sure if that counts as functionality.

2. As for my code, check to see what "exactly"

   mat input A = (3 -5)
   mat li A

produces. Not quite what you expected; the -input- qualifier
obviates the commas.

cheers,
Jeph



Stas Kolenikov wrote:
> Uhm... just an idea: re-run the analysis using -xtmixed- and -post- to
> work out any summaries of any particular analysis that you need as you
> go?.. Or is there some functionality not present in -xtmixed- that
> only HLM offers?
> 
> As for your code, check to see what exactly
> 
> mat A = (3 -5)
> mat li A
> 
> produces. Not quite what you expected; you'd need to put commas
> between the matrix values.
> 
> On 12/2/08, Jeph Herrin <junk@spandrel.net> wrote:
>>  Solved, but there may be better ways:
>>
>>  I used -file- to read the ASCII file a line at a time
>>  and write out a -do- file which contained the necessary
>>  lines to -matrix input- the values. This works beautifully
>>  in that the process is entirely automated. I did have to
>>  create row vectors and combine them with "\", as the whole
>>  matrix was too big to input at once. Here's the top of the
>>  automatically generated -do- file; I then just run the do
>>  file to create the vector and VC matrices:
>>
>>
>>  #delimit ;
>>  matrix input b = (
>>  -1.9488158      -0.0874626       0.1785559       0.0304908 -0.0104287
>> -0.0703224      -0.1402648      -0.1590217 -0.1926564      -0.3406200
>> -0.4178385       0.0257852 -0.0065606       0.0570689      -0.0954522
>> 0.1280117 0.0992062       0.0927456       0.0836843       0.0516530
>> 0.0455983       0.0205244      -0.0223471      -0.0603116 -0.0759259
>> -0.0762618      -0.0953297      -0.1421571 -0.1698843       0.3499212
>> 0.2285282       0.2955863 0.0175301       0.7239078       0.2099990
>> 0.2993378 0.3490949       1.0163712       0.2478369       0.3167427
>> -0.3050866      -0.2854428       1.6223277      -0.0351432 0.1986832
>> 0.2789393       0.2540214      -0.0226724 -0.2104767      -0.0402748
>> 0.0700727       0.0355059 0.0453931      -0.2231259       0.0576488
>> 0.0559482 0.1819322       0.4569752       0.4768460       1.7106584
>>  2.3341741       2.0369928      -0.0271400       0.0421754      -0.0549200
>>  ) ;
>>
>>  And so on.
>>
>>  cheers,
>>  Jeph
>>
>>
>>
>>
>>  Jeph Herrin wrote:
>>
>>> I've used HLM6 to run a large number of mixed effects models,
>>> each model producing an ASCII file which contains the fixed
>>> effects and variance-covariance matrix. There are 60+ variables,
>>> but for whatever reason HLM6 only writes 60 values per line,
>>> so one ASCII file (with 65 terms) looks like this
>>>
>>>
>>>  F1   F2   .. ....   ...  F60
>>>  F61  F62  F63  F64  F65          // 65 coeffs on two rows
>>>  V11  V12   .. ....   ...  V160
>>>  V161 V162 V163 V164 V165         // then 65x65 VC entries on
>>>  V21  V22   .. ....   ...  V260   //  2 x 65 lines
>>>  V261 V262 V263 V264 V265
>>>  .
>>>  .
>>>  .
>>>  V651  V652   .. ....   ... V6560  // last row of VC matrix on
>>>  V6561 V6562 V6563 V6564 V6565     //   two lines
>>>
>>> (where the ASCII file doesn't have the comments).
>>>
>>> Since this is a fairly rigid format that depends only on
>>> N, the number of covariates, I thought it would be a small
>>> matter to -infile- this with a -dct- file and store it as
>>> a matrix. However, -infile- requires me to write out every
>>> single variable name, and to modify the number of variables
>>> in the -dct- file according to the number of covariates.
>>>
>>> In the past, I have used PERL to parse these files, but
>>> I'm doing this on a new box and figured instead of reinstalling
>>> PERL I'd try to sort it out in Stata. Is there an easier way
>>> convert this file to a matrix (actually a vector for the first
>>> two lines and a matrix for the remainder)?
>>>
>>> Thanks for any suggestions.
>>>
>>> Jeph
>>> *
>>> *   For searches and help try:
>>> *   http://www.stata.com/help.cgi?search
>>> *   http://www.stata.com/support/statalist/faq
>>> *   http://www.ats.ucla.edu/stat/stata/
>>>
>>>
>>  *
>>  *   For searches and help try:
>>  *   http://www.stata.com/help.cgi?search
>>  *   http://www.stata.com/support/statalist/faq
>>  *   http://www.ats.ucla.edu/stat/stata/
>>
> 
> 

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index