Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

Re: st: R: memory problem when using odbe to read in large dataset


From   "natalie chan" <natalie.channew@googlemail.com>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: R: memory problem when using odbe to read in large dataset
Date   Sun, 13 Jul 2008 15:32:31 +0100

Dear Manos,

Thanks for your help. I will try that too.

Many thanks.

Have a good day.

Natalie

On Sun, Jul 13, 2008 at 3:23 PM, Mentzakis, Emmanouil
<e.mentzakis@abdn.ac.uk> wrote:
> Maybe you could try loading only the variables you need in the analysis. -odbc- allows you to choose colums.
>
> That would reduce the required memory size. Unless you need all 200 variables.
>
> Cheers
> Manos
>
> ________________________________________
> From: owner-statalist@hsphsun2.harvard.edu [owner-statalist@hsphsun2.harvard.edu] On Behalf Of natalie chan [natalie.channew@googlemail.com]
> Sent: 13 July 2008 14:41
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: R: memory problem when using odbe to read in large dataset
>
> I have also tried to read in the dataset separately using "in":
>
> odbc load in 1/60000, table ("firms")
> odbc load in 60001/120000, table ("firms")
> ..
>
> The first command works, I successfully read in the first 60000
> observations but the second and other commands did not work, that is,
> the Stata only read in the row of the variable names but no
> observation was read in.
>
> Have anyone met similar problem before?
>
> Many thanks
>
> On Sun, Jul 13, 2008 at 2:19 PM, Carlo Lazzaro <carlo.lazzaro@tin.it> wrote:
>>
>> Dear Natalie,
>> I would suggest you to take a look at - help compress -.
>>
>> HTH and Kind Regards,
>>
>> Carlo
>> -----Messaggio originale-----
>> Da: owner-statalist@hsphsun2.harvard.edu
>> [mailto:owner-statalist@hsphsun2.harvard.edu] Per conto di natalie chan
>> Inviato: domenica 13 luglio 2008 14.27
>> A: statalist@hsphsun2.harvard.edu
>> Oggetto: st: memory problem when using odbe to read in large dataset
>>
>> Dear Statalisters,
>>
>> I am trying to read in a dataset with more than 200 variables and 2
>> million observations. the size of the dataset is 2.1G. I am using
>> Stata/MP 10 and Ram of the computer I am using is 1G
>>
>> I set the memory to 500m. the command I used was:
>>
>> odbc load, table ("firms")
>>
>> The stata produced the message "obs must be
>> between 62593 and 62593. As a possible solution, try increasing memory
>> using  set memory ".
>>
>> I increased the memory to 700m, which is the maximum possible memeory
>> I can use, but still it did not work.
>>
>>  Could anyone here give some advice? Thanks a lot.
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/support/faqs/res/findit.html
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>>
>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/support/faqs/res/findit.html
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>>
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
>
> The University of Aberdeen is a charity registered in Scotland, No SC013683.
>
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index