PyBADS Example 2: Non-box constraints#

In this example, we will show how to set more complex constraints in PyBADS, besides a simple bounded box.

This notebook is Part 2 of a series of notebooks in which we present various example usages for BADS with the PyBADS package. The code used in this example is available as a script here.

import sys

if "google.colab" in sys.modules:  # Colab lacks PyBADS: install it
    %pip install "pybads>=1.5.1"

import numpy as np
from pybads import BADS

0. Constrained optimization#

PyBADS naturally supports box constraints lower_bounds and upper_bounds, as we saw in the previous example. However, some optimization problems might have more complex constraints over the variables. Formally, we may wish to solve the problem

\[ \mathbf{x}^\star = \arg\min_{\mathbf{x} \in \mathcal{X}} f(\mathbf{x}) \]

where \(\mathcal{X} \subseteq \mathbb{R}^D\) is the admissible region for the optimization.

We can do this in PyBADS by providing a function non_box_cons that defines constraints violation, that is a function \(g(\mathbf{x})\) which returns True if \(\mathbf{x} \notin \mathcal{X}\) (and False otherwise), as demonstrated below.

1. Problem setup#

We optimize Rosenbrock’s banana function in 2D as in the previous example, but here we force the input to stay within a circle with unit radius.

Since we know the optimization region, we set tight box bounds lower_bounds and upper_bounds around the circle. This step is not necessary, but it is recommended as it further helps the search.

The function passed to non_box_cons:

takes as input an array \(\mathbf{x}_1, \ldots, \mathbf{x}_M\) with shape (M, D), where each \(\mathbf{x}_m \in \mathbb{R}^D\);
outputs a bool array with shape (M,), where the \(m\)-th value is True if \(\mathbf{x}_m\) violates the constraint, False otherwise;

where \(M\) is an arbitrary number of inputs.

def rosenbrocks_fcn(x):
    """Rosenbrock's 'banana' function in any dimension."""
    x_2d = np.atleast_2d(x)
    return np.sum(
        100 * (x_2d[:, 0:-1] ** 2 - x_2d[:, 1:]) ** 2
        + (x_2d[:, 0:-1] - 1) ** 2,
        axis=1,
    )


x0 = np.array([0, 0])  # Starting point
lower_bounds = np.array([-1, -1])
upper_bounds = np.array([1, 1])


def circle_constr(x):
    """Return constraints violation outside the unit circle."""
    x_2d = np.atleast_2d(x)
    # non_box_cons receives an (M, D) array, one point per row
    return np.sum(x_2d**2, axis=1) > 1.0

2. Run the optimization#

We initialize bads with the non-box constraints defined by non_box_cons. Note that we also still specify standard box constraints lower_bounds and upper_bounds, as this will help the search.

Here we do not specify the plausible bounds: in their absence, BADS takes the lower/upper bounds as plausible bounds. As a general rule, it is strongly recommended to specify the plausible bounds.

As in the previous example, we fix the random seed of the run with random_seed, so that the example is reproducible.

bads = BADS(
    rosenbrocks_fcn,
    x0,
    lower_bounds,
    upper_bounds,
    non_box_cons=circle_constr,
    options={"random_seed": 0},  # Makes the run reproducible
)
optimize_result = bads.optimize()

Plausible lower/upper bounds not specified. Using hard upper/lower bounds instead.
Beginning optimization of a DETERMINISTIC objective function

Tip: The message at the end of the run, also in optimize_result['message'], says why it ended. If it used up its budget (max_fun_evals, 500 * D by default, D the number of parameters, or max_iter) before settling, raise it; if it converged but the solution seems imprecise, the FAQ lists options that make it search longer.
https://acerbilab.github.io/pybads/faq.html#faq-on-some-problems-pybads-seems-to-get-stuck-and-stop-too-early-is-there-a-way-to-tune-pybads-to-optimize-towards-a-higher-precision-result-or-to-have-it-optimize-for-longer

 Iteration    f-count         f(x)           MeshScale          Method             Actions
         2               1               1                                 Uncertainty test
         5               1               1         Initial mesh            Initial points
         9               1             0.5         Refine grid             Train
        17               1            0.25         Refine grid             Train
        25        0.953125           0.125         Refine grid             
        26        0.358103           0.125     Successful search (ES-wcm)        
        27        0.169571           0.125     Successful search (ES-wcm)        
        28        0.078532           0.125     Successful search (ES-wcm)        
        29       0.0479325           0.125     Incremental search (ES-wcm)        
        30        0.046686           0.125     Incremental search (ES-ell)        
        31       0.0457222           0.125     Incremental search (ES-wcm)        
        35       0.0457222          0.0625         Refine grid             
        41       0.0457222         0.03125         Refine grid             
        47       0.0457222        0.015625         Refine grid             Train
        48        0.045715        0.015625     Incremental search (ES-ell)        
        50        0.045689        0.015625     Incremental search (ES-ell)        
        51       0.0456798        0.015625     Incremental search (ES-wcm)        
        53       0.0456798      0.00390625         Refine grid             
        59       0.0456798     0.000976562         Refine grid             
Optimization terminated: change in the function value less than options['tol_fun'].
Function value at minimum: 0.045679774134013466

3. Results and conclusions#

x_min = optimize_result["x"]
fval = optimize_result["fval"]

print(f"BADS minimum at: x_min = {x_min.flatten()}, fval = {fval:.4g}")
print(
    f"total f-count: {optimize_result['func_count']}, time: {round(optimize_result['total_time'], 2)} s"
)
print(f"Problem type: {optimize_result['problem_type']}")

BADS minimum at: x_min = [0.78642297 0.61765718], fval = 0.04568
total f-count: 60, time: 0.69 s
Problem type: non-box constraints

The true global minimum of the Rosenbrock function under these constraints is at \(\textbf{x}^\star = [0.786,0.618]\), where \(f^\star = 0.0457\).

Remarks#

While in theory non_box_cons can receive any arbitrary constraints, in practice PyBADS will likely work well only within relatively simple domains (e.g., simple convex regions), as the current version of (Py)BADS uses a simple heuristic to reject samples outside the admissible region.
In particular, PyBADS does not support equality constraints (e.g., of the form \(x_1 + x_2 + x_3 = 1\)).

PyBADS Example 2: Non-box constraints

Contents