[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

From |
"Todd D. Kendall" <todddavidkendall@gmail.com> |

To |
statalist@hsphsun2.harvard.edu |

Subject |
Re: st: RE: Use a few observations from a tab-delimited or csv file |

Date |
Wed, 20 Aug 2008 11:15:10 -0500 |

I do have Excel 2007, but I have been unsuccessful in using it to effectively create a fixed-column file. Whenever I save the file as a .prn, I only get about 250 columns. But this is Statalist, not "Excellist".... Thanks to everyone for their suggestions. I am still working on it, but I think one of these methods will likely work for me. On Wed, Aug 20, 2008 at 10:35 AM, Martin Weiss <martin.weiss@uni-tuebingen.de> wrote: > True, I do use Excel 2007 occasionally, so yes I should have given a > caveat... BTW, it makes me wonder which program Todd is using to handle the > file at the monent > > -----Original Message----- > From: owner-statalist@hsphsun2.harvard.edu > [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Sergiy Radyakin > Sent: Wednesday, August 20, 2008 5:20 PM > To: statalist@hsphsun2.harvard.edu > Subject: Re: st: RE: Use a few observations from a tab-delimited or csv file > > Hello Martin > > unless you have Excel 2007 (or newer :) , the file limit is 65535x256, > while in Todd's case the file is 80000x2200, so Excel would have a > hard time. > > Regards, Sergiy > > > > On 8/20/08, Martin Weiss <martin.weiss@uni-tuebingen.de> wrote: >> Well, to create a dataset of summary datasets, use -h collapse-. If you > had >> access to Stat/Transfer, that would facilitate your problem with the size > of >> the file. Excel could probably take care of the conversion as well, but is >> usually frowned upon on the list... >> >> HTH >> Martin >> >> >> -----Original Message----- >> From: owner-statalist@hsphsun2.harvard.edu >> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Todd D. Kendall >> Sent: Wednesday, August 20, 2008 4:41 PM >> To: statalist@hsphsun2.harvard.edu >> Subject: st: Use a few observations from a tab-delimited or csv file >> >> Dear Statlisters, >> >> I have a file that is currently in csv format (or I could easily >> convert it to tab-delimited). It is fairly large: roughly 80,000 >> observations and 2,200 variables. >> >> In fact, it is too large to fit into Stata (I am running Stata 9.2 on >> a Windows XP machine with 1 GB of RAM). The maximum memory I can >> allocate to Stata is -set mem 636m-. When I try to simply insheet the >> file at this setting, I get only 16,276 observations read in -- not >> anywhere close to the whole file, so I don't think there are any easy >> tweaks to make this work. >> >> However, it turns out that, for roughly the last 2,000 variables, I >> really don't need every single variable; instead, I just need a few >> summary statistics calculated over these 2,000 variables (e.g., the >> mean or standard deviation). My idea is to write a simple do file >> that loads in, say, the first 15,000 observations, computes the mean >> and standard deviation of the 2,000 variables, then drops these >> variabes and saves as a .dta file. I would then repeat on the next >> 15,000 observations, and so on. Then I could just append all the >> little files together, and I would assume I could fit this into Stata, >> as it would only have around 200 variables instead of 2,200. >> >> My problem is that insheet doesn't work with "in" -- i.e., I can't >> write -insheet filename.csv in 1/15000-. Alternatively, if I could >> convert the file from csv into a fixed format, I could write a >> dictionary and use infix, but my Google search for how to convert a >> csv file into a fixed-column file has come up pretty dry. >> >> Am I barking up the wrong tree completely here, or am I missing >> something obvious? I greatly appreciate any suggestions. >> * >> * For searches and help try: >> * http://www.stata.com/help.cgi?search >> * http://www.stata.com/support/statalist/faq >> * http://www.ats.ucla.edu/stat/stata/ >> >> >> * >> * For searches and help try: >> * http://www.stata.com/help.cgi?search >> * http://www.stata.com/support/statalist/faq >> * http://www.ats.ucla.edu/stat/stata/ >> > * > * For searches and help try: > * http://www.stata.com/help.cgi?search > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ > > > * > * For searches and help try: > * http://www.stata.com/help.cgi?search > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ > * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**Follow-Ups**:**Re: st: RE: Use a few observations from a tab-delimited or csv file***From:*Richard Williams <Richard.A.Williams.5@ND.edu>

**References**:**st: Use a few observations from a tab-delimited or csv file***From:*"Todd D. Kendall" <todddavidkendall@gmail.com>

**Re: st: RE: Use a few observations from a tab-delimited or csv file***From:*"Sergiy Radyakin" <serjradyakin@gmail.com>

- Prev by Date:
**st: RE: graph command: xline() with by() option** - Next by Date:
**Re: st: RE: Use a few observations from a tab-delimited or csv file** - Previous by thread:
**RE: st: RE: Use a few observations from a tab-delimited or csv file** - Next by thread:
**Re: st: RE: Use a few observations from a tab-delimited or csv file** - Index(es):

© Copyright 1996–2014 StataCorp LP | Terms of use | Privacy | Contact us | What's new | Site index |