Stata The Stata listserver
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

st: Re: compressed files


From   baum <baum@bc.edu>
To   statalist@hsphsun2.harvard.edu
Subject   st: Re: compressed files
Date   Thu, 13 Feb 2003 10:41:57 -0500

--On Thursday, February 13, 2003 2:33 -0500 G A wrote:

I don't really know what it would take, but my guess is that it wouldn't
be too hard to store the files in some sort of compressed format, and
having Stata decompressing them on the fly when opening. I could even
think of having very large datasets (larger than physical memory)
compressed in memory... but that would probably slow things too much.
Since I don't think this will be implemented soon, could someone share
experiences on invoking Windows apps from within stata? I'm thinking
about doing some sort of "usezip" program that would open .zip files...
Some of our users have already dealt with this. They have annual data files, which if all uncompressed at the same time exhaust disk capacity. They have a program that wants to work with each file in turn. So they save the dta files in .gz format (this is Unix) and have a command
!gzip -d -c foo`year'.dta.gz > foo`year'.dta
within a forvalues loop over `year'. At the end of the loop,
!rm foo`year'.dta
is used, since the original .gz file has been preserved with the -c option. This allows the decompression of each file to be done within the Stata program, the file accessed, and then the .dta file discarded so that space constraints are not breached.

Kit

________________________________________________________________________
Christopher F Baum, Boston College Economics, Chestnut Hill MA 02467 USA
baum@bc.edu http://fmwww.bc.edu/ec/baum.php
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/




© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index