How do I read data from a pipe?

Some users may wish to read data from a pipe. This is typically useful if you have large amounts of ASCII data that has been compressed and if you do not wish to uncompress the data to a file before reading it in.

You can arrange for Stata to read from a pipe, although it will only work if Stata reads the file in a sequential manner and does not try to seek backward in it.

First, you need to write a shell script that creates a special type of file called a FIFO or a pipe. This shell script also should uncompress the compressed file, sending its output through the pipe.

Here is a sample script:

 --------------myprog---------CUT HERE-----------------------------------
 #!/bin/sh
 
 fname=$1
 rm -f mypipe.pip
 mknod mypipe.pip p
 zcat $fname > mypipe.pip &
 --------------myprog---------CUT HERE-----------------------------------

Then you can have a do-file shell out to call this script, passing it the name of the compressed file that you want Stata to read. The next command in Stata is a command to read from the FIFO:

 --------------mytest.do------CUT HERE-----------------------------------
 !myprog testfile.Z >& /dev/null < /dev/null
 infile a b c using mypipe.pip
 --------------mytest.do------CUT HERE-----------------------------------

Stata shells out to myprog, passing it the name of the file it needs to uncompress. The example above uses csh-style redirection to send any output of myprog to /dev/null. If you use sh, you should write

        !myprog testfile.Z > /dev/null 2>&1 < /dev/null

The & (ampersand) at the end of the zcat command in myprog directs Unix to put that process in the background, allowing myprog to return immediately to Stata. While the zcat command uncompresses testfile.Z and sends the uncompressed data to the pipe (mypipe.pip) in the background, Stata’s infile command reads the uncompressed data from that pipe.