Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

RE: st: R: memory problem when using odbe to read in large dataset


From   "Mentzakis, Emmanouil" <e.mentzakis@abdn.ac.uk>
To   "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
Subject   RE: st: R: memory problem when using odbe to read in large dataset
Date   Sun, 13 Jul 2008 15:23:47 +0100

Maybe you could try loading only the variables you need in the analysis. -odbc- allows you to choose colums.

That would reduce the required memory size. Unless you need all 200 variables.

Cheers
Manos

________________________________________
From: owner-statalist@hsphsun2.harvard.edu [owner-statalist@hsphsun2.harvard.edu] On Behalf Of natalie chan [natalie.channew@googlemail.com]
Sent: 13 July 2008 14:41
To: statalist@hsphsun2.harvard.edu
Subject: Re: st: R: memory problem when using odbe to read in large dataset

I have also tried to read in the dataset separately using "in":

odbc load in 1/60000, table ("firms")
odbc load in 60001/120000, table ("firms")
..

The first command works, I successfully read in the first 60000
observations but the second and other commands did not work, that is,
the Stata only read in the row of the variable names but no
observation was read in.

Have anyone met similar problem before?

Many thanks

On Sun, Jul 13, 2008 at 2:19 PM, Carlo Lazzaro <carlo.lazzaro@tin.it> wrote:
>
> Dear Natalie,
> I would suggest you to take a look at - help compress -.
>
> HTH and Kind Regards,
>
> Carlo
> -----Messaggio originale-----
> Da: owner-statalist@hsphsun2.harvard.edu
> [mailto:owner-statalist@hsphsun2.harvard.edu] Per conto di natalie chan
> Inviato: domenica 13 luglio 2008 14.27
> A: statalist@hsphsun2.harvard.edu
> Oggetto: st: memory problem when using odbe to read in large dataset
>
> Dear Statalisters,
>
> I am trying to read in a dataset with more than 200 variables and 2
> million observations. the size of the dataset is 2.1G. I am using
> Stata/MP 10 and Ram of the computer I am using is 1G
>
> I set the memory to 500m. the command I used was:
>
> odbc load, table ("firms")
>
> The stata produced the message "obs must be
> between 62593 and 62593. As a possible solution, try increasing memory
> using  set memory ".
>
> I increased the memory to 700m, which is the maximum possible memeory
> I can use, but still it did not work.
>
>  Could anyone here give some advice? Thanks a lot.
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


The University of Aberdeen is a charity registered in Scotland, No SC013683.

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index