# Re: st: RE: Programming advice on generating loops

 From "Le Wang" <[email protected]> To [email protected] Subject Re: st: RE: Programming advice on generating loops Date Wed, 2 Apr 2008 18:39:53 -0500

```Hi All. Many thanks for your help! I will try them and see if they
work in my case.

Le

On Wed, Apr 2, 2008 at 3:41 PM, philippe van kerm
<[email protected]> wrote:
> I had started drafting this response before seeing further details on the particular problem at hand, so I'm not exactly spot on the problem. But still... I was thinking that taking advantage of the recursive programming facility offered by Stata would be a solution here.
>
>  Recursive programming means that you can call a command ABC in the code for the command ABC itself.
>  (See http://en.wikipedia.org/wiki/Recursion_(computer_science))
>
>  This problem shows an instance where this feature is useful.
>
>  The following example loops through all possible combinations of values taken by an arbitrary number of variables (assume here that the values taken by the variables define points in a particular dimension on a grid).
>
>  * ---
>  pr def recloop
>   syntax [varlist(default=none)] [, upvars(varlist) upvalues(numlist) ]
>   if (`: word count `varlist''==0) {
>     di "... do whatever you need here (knowing that {`upvars'} are equal to {`upvalues'}) respectively"
>   }
>   else {
>     gettoken v varlist : varlist
>     levelsof `v' , local(levels)
>     foreach level of local levels {
>       di "`v' == `level'"
>       recloop `varlist' , upvars(`upvars' `v') upvalues(`upvalues' `level')
>     }
>   }
>  end
>
>  sysuse auto, clear
>  recloop foreign
>  recloop foreign rep78
>  recloop foreign rep78 price
>  * ---
>
>  In the examples, the command loops through all combinations of foreign (0,1), all combinations of rep78 and foreign, all combinations of foreign, rep78 and price. Of course, this code needs to be adapted to your particular problem (here it does nothing within the loops) but it gives you an idea of a potential solution with recursive programming.
>
>  The principle is that you recursively call 'recloop'. At each call,
>  - you remove one variable from its initial arguments (the initial varlist),
>  - you process it: call -levelsof- and loop through all levels, storing the variable name and particular value in 'upvars' and 'upvalues',
>  - then call recloop again.
>  You take action (the inside part of the embedded loops) whenever the list of arguments of recloop is empty. You can take action by using the content of 'upvars' and 'upvalues'.
>
>  Recursive programming is particularly useful in that sort of problem, I believe. But implementation can be somewhat challenging. So perhaps seeking an alternative solution along the lines suggested by Nick or Austin is a more practical solution.
>
>
>  Philippe
>
>
>  Philippe Van Kerm
>  http://ideas.repec.org/e/pva19.html
>
>
>
>
>
>  -----Original Message-----
>  From: [email protected] [mailto:[email protected]] On Behalf Of Austin Nichols
>  Sent: Wednesday, April 02, 2008 8:49 PM
>  To: [email protected]
>  Subject: Re: st: RE: Programming advice on generating loops
>
>  Le Wang <[email protected]>:
>  This may be an ugly-looking approach, but you can write out datasets
>  (tempfiles with one variable each) of the values for each dimension of
>  the grid, then -cross- them, then loop over observations in the
>  crossed data (one loop for any size problem, or depending on what goes
>  inside the loop, no loop at all). This approach is unlikely to hit
>  system limits even given big problems that may tax the limits of local
>  macros.
>
>  On Wed, Apr 2, 2008 at 2:31 PM, Le Wang <[email protected]> wrote:
>  > Nick, thanks for your comments. I tried your solution of writing a
>  > program to write a program..but haven't really figured how.
>  >
>  > Let me try to elaborate my problem a bit more.
>  >
>  > I am doing grid search for N-dimension (where N is the number of
>  > variables) for a minimization problem. For each variable, the program
>  > calculates the widths of the grids and increments (calculated before
>  > the grid search but not given by the user); these widths and
>  > increments could be different. This asymmetry creates some problem
>  > which may not be solved easily.
>  >
>  > Example 1.
>  >
>  > Suppose that we have two variables. For variable X, the grid is in [0
>  > 1] with an increment of .1 (11 values), for variable Y, the grid is in
>  >  [-3 3] with an increment of .01 (601 values). Then I need to generate
>  > a nested loop, for
>  >
>  > forvalues i=1/11{
>  >    forvalues j=1/601{
>  >
>  >        .... (codes to get a value, say, g_i_j)
>  >       (fill in the values in g[i,j] = g_i_j)
>  >    }
>  > }
>  >
>  > then find out (i,j) that has the minimum in matrix g.
>  >
>  >
>  > Example 2.
>  >
>  > Suppose that we have three variables. For variable X, the grid is in
>  > [0 1] with an increment of .1 (11 values), for variable Y, the grid is
>  > in  [-3 3] with an increment of .01 (601 values), for variable Z, the
>  > grid is in [100 200] with an increment of 20 (6 values). Then I need
>  > to generate a nested loop, again
>  >
>  > forvalues i=1/11{
>  >    forvalues j=1/601{
>  >        forvalues k=1/6{
>  >
>  >                .... (codes to get a value, say, g_i_j_k)
>  >               (fill in the values in g[i,j,k] = g_i_j_k)
>  >        }
>  >    }
>  > }
>  >
>  > then find out (i,j,k) that has the minimum in matrix g.
>  >
>  > and so on.
>  >
>  > I am not sure if I have clarified my problem. Let me know if not.
>  >
>  > Nick's solution or similar ones may work, but I haven't succeeded in
>  > my case. Please let me know if there is actually a simple solution to
>  > this. Many thanks!
>  >
>  > Best,
>  > Le
>  >
>  >
>  >
>  >
>  >
>  >
>  > On Wed, Apr 2, 2008 at 10:44 AM, Nick Cox <[email protected]> wrote:
>  > > Tiago Pereira seconded this question.
>  > >
>  > >  Neither Le Wang nor Tiago gave any underlying context or details of what
>  > >  else they want to do.
>  > >
>  > >  Your question, on the face of it, would be answered by writing a program
>  > >  to write a program, given the variables concerned. Writing something to
>  > >  emit scaffolding (i.e. the
>  > >  -forval- loops and the closing braces) would be much easier than writing
>  > >  something that generates code to fit around the scaffolding (any
>  > >  initialisations and wrap-up code, and what goes inside each loop). It
>  > >  would probably be _much_ easier just to write the program directly.
>  > >
>  > >  I doubt very much, however, that you really need what you are asking
>  > >  for. The key here is that you don't know in advance how many variables
>  > >  you have to do with. That suggests to me that there is no special
>  > >  structure to your problem(s), but just a desire to crunch through all
>  > >  the distinct cross-combinations of a set of variables. If that is so,
>  > >  your problem(s) may well yield to -by:- and/or looping over composite
>  > >  groups. There is technique at
>  > >
>  > >  FAQ     . . . . . . . . . . Making foreach go through all values of a
>  > >  variable
>  > >         8/05    Is there a way to tell Stata to try all values of a
>  > >                 particular variable in a foreach statement without
>  > >                 specifying them?
>  > >                 http://www.stata.com/support/faqs/data/foreach.html
>  > >
>  > >  For example,
>  > >
>  > >  egen group = group(varlist)
>  > >  su group, meanonly
>  > >  forval i = 1/`r(max)' {
>  > >         ...
>  > >  }
>  > >
>  > >  applies regardless of the number of variables in varlist.
>  > >
>  > >  That said, there have been attempts to set up more general tools in this
>  > >  territory.
>  > >  Those by myself predate or do not supersede the direct use of -forval-
>  > >  or -foreach- (e.g. the -cp- tool in STB-45).
>  > >
>  > >  I would rather know more than the real problem(s) than try to make
>  > >  further guesses.
>  > >
>  > >  Nick
>  > >  [email protected]
>  > >
>  > >  Le Wang
>  > >
>  > >  I need to generate loops according to the number of variables (unknown
>  > >  beforehand) in the ado program.
>  > >
>  > >  For example, if one uses two variables, then I want to do the following
>  > >  loop
>  > >
>  > >  forvalues i=1/10{
>  > >       forvalues j=1/32{
>  > >
>  > >       .......
>  > >
>  > >       }
>  > >  }
>  > >
>  > >
>  > >  if one specifies three variables, then the loop looks like this
>  > >
>  > >  forvalues i=1/10{
>  > >       forvalues j=1/32{
>  > >             forvalues k=1/47{
>  > >
>  > >             .......
>  > >             }
>  > >       }
>  > >  }
>  > >
>  > >  and so on....
>  > >
>  > >  How can I do this? Thanks in advance.
>  > >
>  > >  *
>  > >  *   For searches and help try:
>  > >  *   http://www.stata.com/support/faqs/res/findit.html
>  > >  *   http://www.stata.com/support/statalist/faq
>  > >  *   http://www.ats.ucla.edu/stat/stata/
>  > >
>  >
>  >
>  >
>  > --
>  > ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>  > Le Wang, Ph.D.
>  > Minnesota Population Center
>  > University of Minnesota
>  > (o) 612-624-5818
>  >
>  > *
>  > *   For searches and help try:
>  > *   http://www.stata.com/support/faqs/res/findit.html
>  > *   http://www.stata.com/support/statalist/faq
>  > *   http://www.ats.ucla.edu/stat/stata/
>  >
>  *
>  *   For searches and help try:
>  *   http://www.stata.com/support/faqs/res/findit.html
>  *   http://www.stata.com/support/statalist/faq
>  *   http://www.ats.ucla.edu/stat/stata/
>
>  **********************************************************************
>  This email and any files transmitted with it are confidential and
>  intended solely for the use of the individual or entity to whom they
>  are addressed. If you have received this email in error please notify
>  the system manager.
>
>  This footnote also confirms that this email message has been swept
>  **********************************************************************
>
>
>
>
>
>  *
>  *   For searches and help try:
>  *   http://www.stata.com/support/faqs/res/findit.html
>  *   http://www.stata.com/support/statalist/faq
>  *   http://www.ats.ucla.edu/stat/stata/
>

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Le Wang, Ph.D.
Minnesota Population Center
University of Minnesota
(o) 612-624-5818
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```