[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

From |
"Radu Ban" <raduban@gmail.com> |

To |
statalist@hsphsun2.harvard.edu |

Subject |
Re: st: dofile tripping over itself |

Date |
Tue, 22 Apr 2008 00:43:33 +0100 |

Aside from this problem, it's not good practice to -save, replace- over the original file. Especially if you want to run this do file more than once. Best, Radu 2008/4/22, Sergiy Radyakin <serjradyakin@gmail.com>: > Here is a demonstration illustrating ~2048 files limit (Windows): > > // ---------------------------------------------------------- > set more off > forval i=1/2050 { > tempname file`i' > display "Opening file `i'" > file open `file`i'' using c:\temp\delme`i', write text replace > } > file close _all > // ---------------------------------------------------------- > > After several runs, I am getting ~2045 as the failure value (2044 > successful open files). Since the .do file remains open while running, > this brings the total open files to 2045 (if I run the loop from the > Stata's prompt, I get precisely one more open file), plus > Stata-executable remains opened during the whole session, so total > reaches 2046. There is one more file created in the Windows\Temp\ > directory for each Stata session (STXXXX), so we have 2047 opened > files. Does Stata open anything else? (to bring the total to declared > 2048) Note that all commands in the benchmark are built-in, so no .ado > files remain open during execution. > > Could somebody please try it on other OSes and say how many files can > be simultaneousely opened by Stata there? > > Best regards, > Sergiy Radyakin > > > On 4/21/08, Sergiy Radyakin <serjradyakin@gmail.com> wrote: > > Dear Dan, > > > > one of the possible explanations could be that you are opening a file > > for output ("file open ....") and don't close it within iterations. > > Than Stata will ultimately run out of handlers (perhaps somebody from > > the Stata, Corp could tell more precisely how many files can be opened > > simultaneously). > > > > The following article from the M$ knowledgebase gives some support and > > an approximate value of how big (small) this limit can be depending on > > the version of OS: http://support.microsoft.com/kb/163202 > > Though they mention quite old OSes, there probably is a limit in the > > new ones too. There could also be a limit imposed by the compiler, > > used to compile Stata. Microsoft's compiler has it's restrictions > > described here: > > http://msdn2.microsoft.com/en-us/library/6e3b887c(VS.80).aspx > > > > ************* Quote ***************************************** > > Up to 2,048 files can be open simultaneously at the lowio level (that > > is, opened and accessed by means of the _open, _read, _write, and so > > forth family of I/O functions). Up to 512 files can be open > > simultaneously at the stdio level (that is, opened and accessed by > > means of the fopen, fgetc, fputc, and so forth family of functions). > > The limit of 512 open files at the stdio level can be increased to a > > maximum of 2,048 by means of the _setmaxstdio function. > > > > Because stdio-level functions, such as fopen, are built on top of the > > lowio functions, the maximum of 2,048 is a hard upper limit for the > > number of simultaneously open files accessed through the C run-time > > library. > > > > ************* End of quote ***************************************** > > > > Hope this helps, > > Sergiy Radyakin > > > > > > > > > > On 4/21/08, Dan Weitzenfeld <dan.weitzenfeld@emsense.com> wrote: > > > Hi Folks, > > > I'm working on a dofile that loops through a process thousands of > > > times. Part of that process includes the following: > > > > > > ----begin code > > > > > > use "$directory/REACH_RESULTS.dta", clear > > > * random stuff > > > save "$directory/REACH_RESULTS.dta", replace > > > > > > ----end code > > > > > > I am repeatedly getting the following error > > > > > > file C:\Documents and Settings\Dan > > > Weitzenfeld\Desktop\REACH_RESULTS.dta could not be opened > > > r(603); > > > > > > ...but when this error occurs, Stata has that very file open! > > > Moreover, the error is occuring at seemingly random times. That is, > > > sometimes it will occur on the 124th run through the loop, while > > > others it won't occur until the 865th run through the loop. > > > I have had similarly random errors with a different > > > computationally-intensive do file. It's as if Stata starts going so > > > fast it trips over itself. > > > Having done a good deal of coding, I am well aware of how suspicious > > > "randomly occurring error" sounds. But trust me, I am changing > > > nothing in the do file, there is no randomness within the file, but > > > the error occurs at random loop iterations. > > > I am using Stata 10 on a Windows XP Professional SP2 on a Dell > > > Dimension DXP061 Core2, 6600@2.40HHz, 2GB ram > > > > > > Has anyone else had this problem? > > > > > > Thanks in advance, > > > Dan > > > * > > > * For searches and help try: > > > * http://www.stata.com/support/faqs/res/findit.html > > > * http://www.stata.com/support/statalist/faq > > > * http://www.ats.ucla.edu/stat/stata/ > > > > > > * > * For searches and help try: > * http://www.stata.com/support/faqs/res/findit.html > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ > -- picasaweb.google.com/raduban * * For searches and help try: * http://www.stata.com/support/faqs/res/findit.html * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**References**:**st: dofile tripping over itself***From:*"Dan Weitzenfeld" <dan.weitzenfeld@emsense.com>

**Re: st: dofile tripping over itself***From:*"Sergiy Radyakin" <serjradyakin@gmail.com>

**Re: st: dofile tripping over itself***From:*"Sergiy Radyakin" <serjradyakin@gmail.com>

- Prev by Date:
**Re: st: dofile tripping over itself** - Next by Date:
**st: dofile tripping over itself** - Previous by thread:
**Re: st: dofile tripping over itself** - Next by thread:
**st: dofile tripping over itself** - Index(es):

© Copyright 1996–2016 StataCorp LP | Terms of use | Privacy | Contact us | What's new | Site index |