# RE: st: RE: Programming advice on generating loops

 From philippe van kerm To "statalist@hsphsun2.harvard.edu" Subject RE: st: RE: Programming advice on generating loops Date Wed, 2 Apr 2008 22:41:35 +0200

```I had started drafting this response before seeing further details on the particular problem at hand, so I'm not exactly spot on the problem. But still... I was thinking that taking advantage of the recursive programming facility offered by Stata would be a solution here.

Recursive programming means that you can call a command ABC in the code for the command ABC itself.
(See http://en.wikipedia.org/wiki/Recursion_(computer_science))

This problem shows an instance where this feature is useful.

The following example loops through all possible combinations of values taken by an arbitrary number of variables (assume here that the values taken by the variables define points in a particular dimension on a grid).

* ---
pr def recloop
syntax [varlist(default=none)] [, upvars(varlist) upvalues(numlist) ]
if (`: word count `varlist''==0) {
di "... do whatever you need here (knowing that {`upvars'} are equal to {`upvalues'}) respectively"
}
else {
gettoken v varlist : varlist
levelsof `v' , local(levels)
foreach level of local levels {
di "`v' == `level'"
recloop `varlist' , upvars(`upvars' `v') upvalues(`upvalues' `level')
}
}
end

sysuse auto, clear
recloop foreign
recloop foreign rep78
recloop foreign rep78 price
* ---

In the examples, the command loops through all combinations of foreign (0,1), all combinations of rep78 and foreign, all combinations of foreign, rep78 and price. Of course, this code needs to be adapted to your particular problem (here it does nothing within the loops) but it gives you an idea of a potential solution with recursive programming.

The principle is that you recursively call 'recloop'. At each call,
- you remove one variable from its initial arguments (the initial varlist),
- you process it: call -levelsof- and loop through all levels, storing the variable name and particular value in 'upvars' and 'upvalues',
- then call recloop again.
You take action (the inside part of the embedded loops) whenever the list of arguments of recloop is empty. You can take action by using the content of 'upvars' and 'upvalues'.

Recursive programming is particularly useful in that sort of problem, I believe. But implementation can be somewhat challenging. So perhaps seeking an alternative solution along the lines suggested by Nick or Austin is a more practical solution.

Philippe

Philippe Van Kerm
http://ideas.repec.org/e/pva19.html

-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Austin Nichols
Sent: Wednesday, April 02, 2008 8:49 PM
To: statalist@hsphsun2.harvard.edu
Subject: Re: st: RE: Programming advice on generating loops

Le Wang <statauser@gmail.com>:
This may be an ugly-looking approach, but you can write out datasets
(tempfiles with one variable each) of the values for each dimension of
the grid, then -cross- them, then loop over observations in the
crossed data (one loop for any size problem, or depending on what goes
inside the loop, no loop at all). This approach is unlikely to hit
system limits even given big problems that may tax the limits of local
macros.

On Wed, Apr 2, 2008 at 2:31 PM, Le Wang <statauser@gmail.com> wrote:
> program to write a program..but haven't really figured how.
>
> Let me try to elaborate my problem a bit more.
>
> I am doing grid search for N-dimension (where N is the number of
> variables) for a minimization problem. For each variable, the program
> calculates the widths of the grids and increments (calculated before
> the grid search but not given by the user); these widths and
> increments could be different. This asymmetry creates some problem
> which may not be solved easily.
>
> Example 1.
>
> Suppose that we have two variables. For variable X, the grid is in [0
> 1] with an increment of .1 (11 values), for variable Y, the grid is in
>  [-3 3] with an increment of .01 (601 values). Then I need to generate
> a nested loop, for
>
> forvalues i=1/11{
>    forvalues j=1/601{
>
>        .... (codes to get a value, say, g_i_j)
>       (fill in the values in g[i,j] = g_i_j)
>    }
> }
>
> then find out (i,j) that has the minimum in matrix g.
>
>
> Example 2.
>
> Suppose that we have three variables. For variable X, the grid is in
> [0 1] with an increment of .1 (11 values), for variable Y, the grid is
> in  [-3 3] with an increment of .01 (601 values), for variable Z, the
> grid is in [100 200] with an increment of 20 (6 values). Then I need
> to generate a nested loop, again
>
> forvalues i=1/11{
>    forvalues j=1/601{
>        forvalues k=1/6{
>
>                .... (codes to get a value, say, g_i_j_k)
>               (fill in the values in g[i,j,k] = g_i_j_k)
>        }
>    }
> }
>
> then find out (i,j,k) that has the minimum in matrix g.
>
> and so on.
>
> I am not sure if I have clarified my problem. Let me know if not.
>
> Nick's solution or similar ones may work, but I haven't succeeded in
> my case. Please let me know if there is actually a simple solution to
> this. Many thanks!
>
> Best,
> Le
>
>
>
>
>
>
> On Wed, Apr 2, 2008 at 10:44 AM, Nick Cox <n.j.cox@durham.ac.uk> wrote:
> > Tiago Pereira seconded this question.
> >
> >  Neither Le Wang nor Tiago gave any underlying context or details of what
> >  else they want to do.
> >
> >  Your question, on the face of it, would be answered by writing a program
> >  to write a program, given the variables concerned. Writing something to
> >  emit scaffolding (i.e. the
> >  -forval- loops and the closing braces) would be much easier than writing
> >  something that generates code to fit around the scaffolding (any
> >  initialisations and wrap-up code, and what goes inside each loop). It
> >  would probably be _much_ easier just to write the program directly.
> >
> >  I doubt very much, however, that you really need what you are asking
> >  for. The key here is that you don't know in advance how many variables
> >  you have to do with. That suggests to me that there is no special
> >  structure to your problem(s), but just a desire to crunch through all
> >  the distinct cross-combinations of a set of variables. If that is so,
> >  your problem(s) may well yield to -by:- and/or looping over composite
> >  groups. There is technique at
> >
> >  FAQ     . . . . . . . . . . Making foreach go through all values of a
> >  variable
> >         8/05    Is there a way to tell Stata to try all values of a
> >                 particular variable in a foreach statement without
> >                 specifying them?
> >                 http://www.stata.com/support/faqs/data/foreach.html
> >
> >  For example,
> >
> >  egen group = group(varlist)
> >  su group, meanonly
> >  forval i = 1/`r(max)' {
> >         ...
> >  }
> >
> >  applies regardless of the number of variables in varlist.
> >
> >  That said, there have been attempts to set up more general tools in this
> >  territory.
> >  Those by myself predate or do not supersede the direct use of -forval-
> >  or -foreach- (e.g. the -cp- tool in STB-45).
> >
> >  I would rather know more than the real problem(s) than try to make
> >  further guesses.
> >
> >  Nick
> >  n.j.cox@durham.ac.uk
> >
> >  Le Wang
> >
> >  I need to generate loops according to the number of variables (unknown
> >  beforehand) in the ado program.
> >
> >  For example, if one uses two variables, then I want to do the following
> >  loop
> >
> >  forvalues i=1/10{
> >       forvalues j=1/32{
> >
> >       .......
> >
> >       }
> >  }
> >
> >
> >  if one specifies three variables, then the loop looks like this
> >
> >  forvalues i=1/10{
> >       forvalues j=1/32{
> >             forvalues k=1/47{
> >
> >             .......
> >             }
> >       }
> >  }
> >
> >  and so on....
> >
> >  How can I do this? Thanks in advance.
> >
> >  *
> >  *   For searches and help try:
> >  *   http://www.stata.com/support/faqs/res/findit.html
> >  *   http://www.stata.com/support/statalist/faq
> >  *   http://www.ats.ucla.edu/stat/stata/
> >
>
>
>
> --
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Le Wang, Ph.D.
> Minnesota Population Center
> University of Minnesota
> (o) 612-624-5818
>
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
the system manager.

This footnote also confirms that this email message has been swept
**********************************************************************

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```