I would also like to thank Mr Gould for the detailed explanation. I
still can not pinpoint the problem. The tempfiles are created inside
one procedure and passed to a subroutine which is to read them
(datasets and sersets are saved). After that the main program
continues. This should be an allowed way of handling the tempfiles.
After ",replace" was added the program continues to work as expected,
so I guess the main reason was that Stata has shut down several times
abnormally before that and some files left in the temp directory.
Anyways, I would think that checking if a file exists before returning
it as a candidate might be an improvement in reliability without a
major loss in compatibility (unless someone objects).
2Jeph: base34: 0,1,2,3....9,a,b,c,....x
Best regards, Sergiy Radyakin
On 8/29/07, Jeph Herrin <[email protected]> wrote:
> Thanks for this highly informative response.
>
> One oddity: base-34?
>
>
> William Gould, StataCorp LP wrote:
> > Sergiy Radyakin <[email protected]> asks a series of questions
> > concerning Stata's tempfiles.
> >
> >> Can anyone confirm that Stata does or does nothing to check if the
> >> 1) temp candidate file already exists
> >> 2) temp candidate file is open by Stata or another application (Stata
> >> or not Stata)
> >> 3) temp folder is writable
> >> 4) another Stata session is attempting to allocate tempfiles too
> >> (syncronization to avoid overlaps)
> >
> >
> > 1. whether temp candidate file already exists
> > ----------------------------------------------
> >
> > Under Windows, a tempfile name looks like
> >
> > <path>\ST_<instance><seqno>.tmp
> >
> > where <instance> := 2-digit, base-34 instance number
> > <seqno> := 6-digit, base-34 sequence number
> >
> > Sergiy gave the example,
> > <seqno>
> > __|___
> > c:\Documents and Settings\mylogin\Local Settings\Temp\ST_010000jf.tmp
> > ----------------------------------------------------- --
> > | |
> > <path> <instance>
> >
> > The instance number has to do with which Stata is running if multiple
> > Stata's are running and will be unique for each simultaneously running
> > Stata.
> >
> > The sequence number will 000001 to begin, followed by 000002, etc.,
> > counting in base 34 for which the top digit is "x". You count 1, 2, ...,
> > 9, a, b, ..., w, x, 10, 11, ...
> >
> > So, to answer the question, "Does Stata check whether a tempfile already
> > exists?", the answer is No, not at the time the -tempfile- command is given.
> > At startup time, Stata sets <instance>. When Stata comes up, it creates a
> > special temporary file (not to be confused with tempfile) named
> > SQ_<instance>000000.tmp. When Stata shutdown cleanly, it erases that file.
> > Thus, when Stata comes up, it looks for SQ_<instance>000000.tmp files, find
> > the first one that does not exist, and that determines this Stata's
> > <instance>. Stata immediately creates SQ_<instance>000000.tmp.
> >
> > By this logic, files of the form ST_<instance><seqno>.tmp should not
> > exist and therefore it is not necessary to check whether they do.
> >
> >
> > 2. Whether temp candidate file is open by Stata or another application
> > -----------------------------------------------------------------------
> >
> > Stata does not check for this. See (1) above.
> >
> >
> >
> > 3. Whether temp folder is writable
> > -----------------------------------
> >
> > This is checked when Stata comes up. It is NOT checked everytime
> > a tempfile is generated.
> >
> >
> > 4. Another Stata session is attempting to allocate tempfiles too
> > -----------------------------------------------------------------
> >
> > See (1) above. It is not necessary to check because any other
> > running Stata will have a different <instance>.
> >
> >
> > Sergiy goes on to ask,
> >
> >> [...] what might be an explanation for the following:
> >>
> >> tempfile tfile
> >> save `"`tfile'"'
> >> file c:\Documents and Settings\mylogin\Local
> >> Settings\Temp\ST_010000jf.tmp already exists
> >> r(602);
> >>
> >> The situation occurs not always, but with a non-zero probability.
> >> (Stata 9.2 up-to-date, Windows XP). The temp folder indeed contains
> >> dozens of abandoned temporary files, but please do not just suggest to
> >> clean it up regularly.
> >
> > There are three possiblities:
> >
> > 1. You are using -tempfile- interactively (which is to say, outside of
> > do- and ado-files). In this case, it is your responsibility
> > to erase tempfiles you create.
> >
> > 2. You are passing a tempfile name out of a program for use by
> > a calling program. (That is inappropriate use of -tempfile-.
> > Each program, do-file, and ado-file knows only to erase
> > tempfiles that it itself obtained from -tempfile-).
> >
> > 3. Stata previously crashed. In this case, you should clean up.
> >
> > Concerning (1), due to Sergiy's posting we are now considering changing
> > Stata's logic to erase ALL temporary files on exit. Is anyone taking
> > advantage of (1) as it currently works and passing temporary files from
> > one Stata session to the next? If so, we need to hear about it.
> >
> > Concerning (3), cleaning up is easy. With no Stata's running, erase all
> > ST_*.tmp files in the temporary directory, which in Sergiy's case is
> >
> > c:\Documents and Settings\mylogin\Local Settings\Temp\
> >
> > We recommend Sergiy do this. If Sergiy has found some way to crash
> > Stata repeatedly, he should contact Technical Services because there
> > are other files in the temporary directory that Sergiy may need to
> > erase. For most users, the need to erase temporary files never arises.
> >
> > Finally, Sergiy says,
> >
> >> So far I have to enforce ",replace" everywhere in the program. This
> >> doesn't seem to be a good idea, [...]
> >
> > We agree; do not use -, replace-. Clean up the temporary directory.
> >
> >
> > -- Bill
> > [email protected]
> > *
> > * For searches and help try:
> > * http://www.stata.com/support/faqs/res/findit.html
> > * http://www.stata.com/support/statalist/faq
> > * http://www.ats.ucla.edu/stat/stata/
> >
> >
> *
> * For searches and help try:
> * http://www.stata.com/support/faqs/res/findit.html
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
