st: Re: compressed files

[baum <baum@bc.edu>]

--On Thursday, February 13, 2003 2:33 -0500 G A wrote:

> I don't really know what it would take, but my guess is that it wouldn't
> be too hard to store the files in some sort of compressed format, and
> having Stata decompressing them on the fly when opening. I could even
> think of having very large datasets (larger than physical memory)
> compressed in memory... but that would probably slow things too much.
> Since I don't think this will be implemented soon, could someone share
> experiences on invoking Windows apps from within stata? I'm thinking
> about doing some sort of "usezip" program that would open .zip files...
Some of our users have already dealt with this. They have annual data 
files, which if all uncompressed at the same time exhaust disk capacity. 
They have a program that wants to work with each file in turn. So they save 
the dta files in .gz format (this is Unix) and have a command
!gzip -d -c foo`year'.dta.gz > foo`year'.dta
within a forvalues loop over `year'. At the end of the loop,
!rm foo`year'.dta
is used, since the original .gz file has been preserved with the -c option. 
This allows the decompression of each file to be done within the Stata 
program, the file accessed, and then the .dta file discarded so that space 
constraints are not breached.

Kit

________________________________________________________________________
Christopher F Baum, Boston College Economics, Chestnut Hill MA 02467 USA
baum@bc.edu                              http://fmwww.bc.edu/ec/baum.php
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/