[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

From |
"Sergiy Radyakin" <serjradyakin@gmail.com> |

To |
statalist@hsphsun2.harvard.edu |

Subject |
Re: st: SV: Re: Memory problems when creating a spatial weight matrix |

Date |
Wed, 26 Sep 2007 13:05:42 +0200 |

Hi Lina, type: viewsource spatwmat.ado in Stata's command prompt to see the source code Also you can: set trace on before calling spatwmat and see where it breaks down. Best regards, Sergiy On 9/26/07, Lina Jonsson <lina.jonsson@vti.se> wrote: > Hi again. > > > Thank you for the quick reply Sergiy! > > There is no problem running the code you gave me. I belive that the spatwmat-command is creating something temporarily that takes a lot of memory. I have only been using Stata for a couple of months so I have never looked into a source code before - how do I do that? > > Does anyone else have experience in using the spatwmat-command on "large" (more than 800 observations) datasets? > > Best Regards > > /Lina Jonsson > > > -----Ursprungligt meddelande----- > > Från: owner-statalist@hsphsun2.harvard.edu > > [mailto:owner-statalist@hsphsun2.harvard.edu] För Sergiy Radyakin > > Skickat: den 26 september 2007 12:34 > > Till: statalist@hsphsun2.harvard.edu > > Ämne: Re: st: SV: Re: Memory problems when creating a spatial > > weight matrix > > > > Hi Lina, > > > > may be I am missing something, but a matrix 500x500 at 4 > > bytes per cell occupies 1MB of memory. At 8 bytes per cell > > 2MB respectively. How does this relate to the 500/900MB of > > memory limit? As I understand, your dataset is 800 > > observations and -spatwmat- produces a matrix 800x800. > > > > Could you run the following and tell us what happens? > > > > clear > > set mem 100m <-- or whatever your maximum is > > set matsize 800 > > matrix Z=J(800,800,0) > > matrix dir > > matrix A=J(800,800,0) > > matrix dir > > matrix B=J(800,800,0) > > matrix dir > > matrix C=J(800,800,0) > > matrix dir > > use "mydata" <--- whatever your dataset is > > > > If it works fine, than it is not a memory issue, but rather > > you'd have to look into the source code of -spatwmat- to find > > if the memory is used there efficiently (e.g. it does not > > generate 800 temporary matrices). > > > > From the first look, it might be relatively easy to port the > > code into matlab. However then you need matlab, why not Stata > > SE? If you don't have constraints with software -- buy Stata > > 10 SE and this might resolve the issue. If you do have > > constraints, than what are they (which software is available > > to you)? There might be just a handful of people working with > > specialized commands like this, so please explain us what > > this command does in plain English. > > > > Best regards, > > Sergiy Radyakin > > > > > > > > > > > > On 9/26/07, Lina Jonsson <lina.jonsson@vti.se> wrote: > > > Dear statalister. > > > > > > Sorry I haven´t had time to follow up my question until now. > > > > > > The memory problem occurs only when using the command for > > creating the spatial weight matrix (spatwmat). The dataset > > I´m using consists of only 28 variables - all numerical and > > dummies (0,1). According to an earlier question concering > > memory problems when using the spatwmat-command; "-spatwmat- > > will reset the number of observations to (N/2)*N-1" which > > means about 320 000 if I´m using a dataset with 800 > > observations. The advice that is given in the former question > > is to set memory to a higher value. That is the advice that > > also is given for my error message "no room to add more > > variables". But the problem is that I cannot set memory to > > more then 940m. > > > > > > I am using Windows XP and have just increased my RAM-memory > > to 2 GB which had no effect at all on the maximum memory I > > can set within Stata. As I understand it; the hotfix Michel > > mentioned is only for Stata 10 and I`m using Intercooled Stata 9.2. > > > > > > Is anyone having any suggestions on how this problem can be > > solved or is it not possible to use the spatwmat-command on > > datasets with 800 observation using "a normal computer". In > > fact I whould like to use up to 1738 observations but for > > that I whould need to switch to Stata SE instead of > > Intercooled Stata due to the fact that Intercooled Stata can > > set matsize to a maximum of 800. But there is no point in > > changing to Stata SE if the available memory sets the limit > > to less than 800 observations... > > > > > > If Stata can´t handle spatial weight matrices for large > > datasets, do any of you have any suggestions of other > > statistical software that can? > > > > > > Best regards. > > > > > > /Lina Jonsson > > > > > > > > > > > > > > > > Från: owner-statalist@hsphsun2.harvard.edu > > > > [mailto:owner-statalist@hsphsun2.harvard.edu] För Michael Blasnik > > > > Skickat: den 21 september 2007 14:49 > > > > Till: statalist@hsphsun2.harvard.edu > > > > Ämne: st: Re: Memory problems when creating a spatial > > weight matrix > > > > > > > > You need to give us more information about your dataset, > > operating > > > > system, etc. > > > > The 940m should be plenty of room for a 500 observation > > dataset and > > > > an 800x800 matrix. Can you describe your dataset in more > > detail? > > > > Do you have hundreds of long string variables? Is there > > a way for > > > > you to encode any long strings and compress other variables? In > > > > terms of accessing more memory -- what OS are you using? If MS > > > > Windows XP, have you tried the memory hotfix described here > > > > http://www.stata.com/support/faqs/win/winmemory.html? > > > > > > > > Michael Blasnik > > > > > > > > ----- Original Message ----- > > > > From: "Lina Jonsson" <lina.jonsson@vti.se> > > > > To: <statalist@hsphsun2.harvard.edu> > > > > Sent: Friday, September 21, 2007 5:25 AM > > > > Subject: st: Memory problems when creating a spatial weight matrix > > > > > > > > > > > > > Dear statalisters. > > > > > > > > > > I am working with a hedonic price study where I whould like > > > > to check for > > > > > and correct for spatial autocorrelation. For this I > > whould like to > > > > > create a spatial weight matrix based on the x- and > > y-coordinates > > > > > for each house using the spatwmat-command in spdep. Since I > > > > have Intercooled > > > > > Stata the maximum matsize that I can set is 800, which > > > > means that I can > > > > > use no more than 800 observations. But since I cannot set > > > > memory to more > > > > > than 940m, the actual numbers of observations that I can use is > > > > > less than 500. This is to few. > > > > > > > > > > After reading some old questions concerning memory problems > > > > I realize > > > > > that this might be a problem that is hard to solve. > > > > > Does any of you have any suggestions on how I can increase > > > > the memory in > > > > > Stata. Can increasing the RAM-memory on my computer > > help? Or is it > > > > > unrealistic to think that it will be possible to create a > > > > matrix whith > > > > > the spatwmat-command that will be almost 8000x800 on a > > "standard" > > > > > computer? Should I change to some other software and what > > > > in that case? > > > > > > > > > > I am greatful for any suggestions! > > > > > > > > > > /Lina Jonsson > > > > > > > > * > > > > * For searches and help try: > > > > * http://www.stata.com/support/faqs/res/findit.html > > > > * http://www.stata.com/support/statalist/faq > > > > * http://www.ats.ucla.edu/stat/stata/ > > > > > > > > > > * > > > * For searches and help try: > > > * http://www.stata.com/support/faqs/res/findit.html > > > * http://www.stata.com/support/statalist/faq > > > * http://www.ats.ucla.edu/stat/stata/ > > > > > > > * > > * For searches and help try: > > * http://www.stata.com/support/faqs/res/findit.html > > * http://www.stata.com/support/statalist/faq > > * http://www.ats.ucla.edu/stat/stata/ > > > > * > * For searches and help try: > * http://www.stata.com/support/faqs/res/findit.html > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ > * * For searches and help try: * http://www.stata.com/support/faqs/res/findit.html * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**References**:**st: bug in ivtobit***From:*"David Roodman (DRoodman@cgdev.org)" <DRoodman@CGDEV.ORG>

**st: Memory problems when creating a spatial weight matrix***From:*"Lina Jonsson" <lina.jonsson@vti.se>

**st: Re: Memory problems when creating a spatial weight matrix***From:*"Michael Blasnik" <michael.blasnik@verizon.net>

**st: SV: Re: Memory problems when creating a spatial weight matrix***From:*"Lina Jonsson" <lina.jonsson@vti.se>

**Re: st: SV: Re: Memory problems when creating a spatial weight matrix***From:*"Sergiy Radyakin" <serjradyakin@gmail.com>

**SV: st: SV: Re: Memory problems when creating a spatial weight matrix***From:*"Lina Jonsson" <lina.jonsson@vti.se>

- Prev by Date:
**SV: st: SV: Re: Memory problems when creating a spatial weight matrix** - Next by Date:
**st: could you help me how to write stata command about Marginal effects for panel data neg binomial(xtnbreg)** - Previous by thread:
**SV: st: SV: Re: Memory problems when creating a spatial weight matrix** - Next by thread:
**st: Re: SV: Re: Memory problems when creating a spatial weight matrix** - Index(es):

© Copyright 1996–2017 StataCorp LLC | Terms of use | Privacy | Contact us | What's new | Site index |