Moving limits options (#681)

* added moving limits options in Quasi-Newton solver

* options for the Newton solver

* updated user manual

doc/src/hiop.bib

@@ -12,7 +12,7 @@ @Article{ipopt_impl
 }
 
 @article{petra2019memory,
-  title={A memory-distributed quasi-Newton solver for nonlinear programming problems with a small number of general constraints},
+  title={A memory-distributed quasi-{N}ewton solver for nonlinear programming problems with a small number of general constraints},
   author={Petra, Cosmin G},
   journal={Journal of Parallel and Distributed Computing},
   volume={133},
@@ -21,14 +21,3 @@ @article{petra2019memory
   publisher={Elsevier}
 }
 
-@TECHREPORT{petra_hiop,
-  author = {Cosmin G. Petra},
-  title = {A memory-distributed quasi-{N}ewton solver for
-nonlinear programming problems with a small number
-of general constraints},
-  year = {2017},
-  number = {LLNL-JRNL-739001},
-  month = {October},
-  institution = {Lawrence Livermore National Laboratory}
-}
-

doc/src/petra.bib

@@ -3213,7 +3213,7 @@ @article{ByrdKhalfanSchnabel_96_SR1
 
 @Article{ByrdNocedalSchnabel_94_quasiNewtonRepres,
 author="Byrd, Richard H. and Nocedal, Jorge and Schnabel, Robert B.",
-title="Representations of quasi-Newton matrices and their use in limited memory methods",
+title="Representations of quasi-{N}ewton matrices and their use in limited memory methods",
 journal="Mathematical Programming",
 year="1994",
 volume="63",

doc/src/sections/solver_options.tex

@@ -131,7 +131,13 @@ \subsubsection{Line search and step computation}
 \noindent \textbf{tau\_min}: fraction-to-the-boundary parameter used in the line-search to back-off from the boundary (eqn. (8) in~\cite{waecther_05_ipopt0}). Double values in $[0.9, 0.99999]$. Default value: $0.99$. 
 \medskip
 
-\noindent \textbf{accept\_every\_trial\_stepduals}: disable the line-search and take the close-to-boundary step. String values: ``no'' (default) and ``yes''.
+\noindent \textbf{accept\_every\_trial\_step}: Disables the filter line-search and take the close-to-boundary step. This step can be further controlled (while the filter line-search remains disabled) using options \textit{moving\_lim\_abs} and \textit{moving\_lim\_abs}. This option takes the string values: ``no'' (default) and ``yes''.
+\medskip
+
+\noindent \textbf{moving\_lim\_abs}: Controls the maximum allowed update of the $x$ primal variables during the line-search. Primal step-length may be reduced to ensure that the inf norm of the ``$\Delta x$'' update/direction is less than or equal with the option's value. It can be used with \textit{accept\_every\_trial\_step}. Numerical range is $[0,10^8]$. A zero value will disable this option. Default value: 0. 
+\medskip
+
+\noindent \textbf{moving\_lim\_rel}: Controls the maximum allowed update of the primal variables \textit{relative} to close-to-boundary step. With this option the line-search will be started using the close-to-boundary step \textit{scaled} by the option's value. Default value: 0 (disabled). It can be used with \textit{accept\_every\_trial\_step}. Numerical range is $[0,1]$. A zero value will disable this option. Default value: 0. 
 \medskip
 
 \noindent \textbf{duals\_init}: type of the update for the initialization of Lagrange multipliers corresponding to the equality constraints. Possible values one of the the strings ``lsq'' (least-square (LSQ) solve initialization) and ``zero'' (multipliers are set identically to zero). Default value: is ``lsq''.

doc/src/techrep_main.tex

@@ -133,7 +133,7 @@
 \vspace{3cm}
 
   {\huge\bfseries \Hi\ -- User Guide} \\[14pt]
-  {\large\bfseries version 1.0}
+  {\large\bfseries version 1.03}
 
 \vspace{3cm}
 
@@ -155,7 +155,7 @@
 \vspace{4.75cm}
 
  \textcolor{violet}{{\large\bfseries Oct 15, 2017} \\
-{\large\bfseries Updated Sep 7, 2023}}
+{\large\bfseries Updated Feb 5, 2024}}
 
 \vspace{0.75cm}
 
@@ -242,7 +242,7 @@ \section{Introduction}
 \end{itemize}
 
 This document includes instructions on how to obtain and
-build \Hi and a description of its interface, user options, and use as an optimization library. Guidelines on how is best to use the solver for parallel computations are also provided. The document generally targets users of \Hi, but also contains information relevant to potential developers or advanced users; these are strongly encouraged to also read the paper on the computational approach implemented in \Hi~\cite{petra_hiop}.
+build \Hi and a description of its interface, user options, and use as an optimization library. Guidelines on how is best to use the solver for parallel computations are also provided. The document generally targets users of \Hi, but also contains information relevant to potential developers or advanced users; these are strongly encouraged to also read the paper on the computational approach implemented in \Hi~\cite{petra2019memory}.
 
 
 While the MPI quasi-Newton solver of \texttt{Hiop} targets DAE- and PDE-constrained optimization problems formulated in a ``reduced-space'' approach, it can be used for general nonconvex nonlinear optimization as well. For efficiency considerations, it is recommended to \textit{use quasi-Newton \texttt{Hiop} for NLPs that have a relatively small number of general constraints}, say less than $100$; note that there are no restrictions on the number of bounds constraints, \textit{e.g.}, one can specify simple bounds on any, and potentially all the decision variables without affecting the computational efficiency. The minimizers computed by \Hi satisfies \textit{local} first-order optimality conditions.

src/Optimization/hiopAlgFilterIPM.cpp

@@ -1192,7 +1192,12 @@ hiopSolveStatus hiopAlgFilterIPMQuasiNewton::run()
     nlp->runStats.tmSolverInternal.start();
 
     //maximum  step
-    bret = it_curr->fractionToTheBdry(*dir, _tau, _alpha_primal, _alpha_dual); assert(bret);
+    bret = it_curr->fractionToTheBdry(*dir, _tau, _alpha_primal, _alpha_dual);
+    assert(bret);
+    //Step `_alpha_primal` may be reduced when option 'moving_lim_abs' or 'moving_lim_rel' is active.
+    //Returned bool indicates if reduction was done or not.
+    bret = ensure_moving_lims(*it_curr, *dir, _alpha_primal);
+
     double theta = onenorm_pr_curr_ = resid->get_theta(); //at it_curr
     double theta_trial;
     nlp->runStats.tmSolverInternal.stop();
@@ -2164,6 +2169,10 @@ hiopSolveStatus hiopAlgFilterIPMNewton::run()
 
       //maximum  step
       bret = it_curr->fractionToTheBdry(*dir, _tau, _alpha_primal, _alpha_dual); assert(bret);
+      //Step `_alpha_primal` may be reduced when option 'moving_lim_abs' or 'moving_lim_rel' is active.
+      //Returned bool indicates if reduction was done or not.
+      bret = ensure_moving_lims(*it_curr, *dir, _alpha_primal);
+
       double theta = onenorm_pr_curr_ = resid->get_theta(); //at it_curr
       double theta_trial;
       nlp->runStats.tmSolverInternal.stop();
@@ -2496,6 +2505,43 @@ void hiopAlgFilterIPMNewton::outputIteration(int lsStatus, int lsNum, int use_so
   }
 }
 
+bool hiopAlgFilterIPMBase::ensure_moving_lims(const hiopIterate& it, const hiopIterate& dir, double& alpha_pr)
+{
+  auto moving_lim_rel = nlp->options->GetNumeric("moving_lim_rel");
+  if(moving_lim_rel>0) {
+    const auto alpha_pr_in = alpha_pr;
+    alpha_pr = moving_lim_rel * alpha_pr;
+    nlp->log->printf(hovLinesearch,
+                     "Moving lim (relative, [%7.3e]): step reduced: %7.3e -> %7.3e.\n",
+                     moving_lim_rel,
+                     alpha_pr_in,
+                     alpha_pr);
+    return true;
+  }
+
+  auto moving_lim_abs = nlp->options->GetNumeric("moving_lim_abs");
+  if(moving_lim_abs>0) {
+    const auto alpha_pr_in = alpha_pr;
+    auto x_nrm = dir.get_x()->infnorm();
+    auto step_nrm = alpha_pr*x_nrm;
+    if(step_nrm>moving_lim_abs) {
+      alpha_pr = moving_lim_abs / step_nrm;
+      nlp->log->printf(hovLinesearch,
+                       "Moving lim (absolute, [%7.3e]): step reduced: %7.3e -> %7.3e.\n",
+                       moving_lim_abs,
+                       alpha_pr_in,
+                       alpha_pr);
+      return true;
+    } else {
+      nlp->log->printf(hovLinesearch,
+                       "Moving lim (absolute, [%7.3e]) satisfied, step (norm %7.3e) not reduced.\n",
+                       moving_lim_abs,
+                       step_nrm);    
+    }
+  } // end of moving lim abs 
+  return false;
+}
+
 
 int hiopAlgFilterIPMBase::accept_line_search_conditions(const double theta_curr,
                                                         const double theta_trial,

src/Optimization/hiopAlgFilterIPM.hpp

@@ -183,7 +183,7 @@ class hiopAlgFilterIPMBase {
                                  const bool& elastic_mode_on,
                                  double& mu_new,
                                  double& tau_new);
-
+protected:
   // second order correction
   virtual int apply_second_order_correction(hiopKKTLinSys* kkt,
                                             const double theta_curr,
@@ -198,7 +198,9 @@ class hiopAlgFilterIPMBase {
                                             const double alpha_primal,
                                             bool &grad_phi_dx_computed,
                                             double &grad_phi_dx);
-
+  /// @brief Step-length `alpha_pr` may be reduced when option 'moving_lim_abs' or 'moving_lim_rel' is active.
+  bool ensure_moving_lims(const hiopIterate& it, const hiopIterate& dir, double& alpha_pr);
+public:
   /// @brief do feasibility restoration
   virtual bool apply_feasibility_restoration(hiopKKTLinSys* kkt);
   virtual bool solve_soft_feasibility_restoration(hiopKKTLinSys* kkt);

src/Utils/hiopLogger.cpp

@@ -155,8 +155,14 @@ void hiopLogger::printf_error(hiopOutVerbosity v, const char* format, ...)
   char buff[4096];
   va_list args;
   va_start (args, format);
-  vsnprintf(buff,4096,format, args);
-  fprintf(stderr,"%s",buff);
+  vsnprintf(buff, 4096, format, args);
+  if(v<=hovError) {
+    fprintf(stderr, "ERROR: %s", buff);
+  } else if(v<=hovWarning) {
+    fprintf(stderr, "WARNING: %s", buff);
+  } else {
+    fprintf(stderr, "%s", buff);
+  }
   va_end (args);
 };
 

src/Utils/hiopOptions.cpp

@@ -677,6 +677,7 @@ void hiopOptionsNLP::register_options()
                       0.,
                       1e+7,
                       "Initial value of the initial multiplier of the identity in the secant approximation (default 1.0)");
+  //step length controls
   {
     vector<string> range(2); range[0] = "no"; range[1] = "yes";
     register_str_option("accept_every_trial_step", "no", range, "Disable line-search and take close-to-boundary step");
@@ -688,18 +689,27 @@ void hiopOptionsNLP::register_options()
                         "Minimum step size allowed in line-search (default 1e-16). If step size is less than this number, " 
                         "feasibility restoration problem is activated.");
 
+    auto d_abs = "Max allowed update of the 'x' primal variables during the line-search. Primal step-length may be reduced "
+                 "so that the inf norm of the 'x' update is less than or equal with the option's value. Default value: "
+                 "0 (disabled).";
+    register_num_option("moving_lim_abs", 0., 0., 1e+8, d_abs);
+
+    auto d_rel = "Max allowed update of the primal variables relative to fraction-to-boundary (FTB) step. Line-search "
+                 "will be started using the FTB step scaled by the option's value. Default value: 0 (disabled).";
+    register_num_option("moving_lim_rel", 0., 0., 1., d_rel);
+
     register_num_option("theta_max_fact",
                         1e+4,
                         0.0,
                         1e+7,
-                        "Maximum constraint violation (theta_max) is scaled by this fact before using in the fileter line-search "
+                        "Maximum constraint violation (theta_max) is scaled by this factor before using in the filter line-search "
                         "algorithm (default 1e+4). (eqn (21) in Filt-IPM paper)");
 
     register_num_option("theta_min_fact",
                         1e-4,
                         0.0,
                         1e+7,
-                        "Minimum constraint violation (theta_min) is scaled by this fact before using in the fileter line-search "
+                        "Minimum constraint violation (theta_min) is scaled by this factor before using in the filter line-search "
                         "algorithm (default 1e-4). (eqn (21) in Filt-IPM paper)");
   }
   {
@@ -1532,6 +1542,15 @@ void hiopOptionsNLP::ensure_consistence()
       set_val("fact_acceptor", "inertia_free");
     }
   }
+
+  if(0 != GetNumeric("moving_lim_rel") * GetNumeric("moving_lim_abs")) {
+    if(is_user_defined("moving_lim_rel") || is_user_defined("moving_lim_abs")) {
+      log_printf(hovWarning,
+                 "Options 'moving_lim_rel' and 'moving_lim_rel' are both active, which is not supported. "
+                 "Option 'moving_lim_rel' will be disabled.\n");
+    }
+    set_val("moving_lim_rel", 0.);
+  }
 }
 
 /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////