PyBADS Example 3: Noisy objective function#
In this example, we will show how to run PyBADS on a noisy target.
This notebook is Part 3 of a series of notebooks in which we present various example usages for BADS with the PyBADS package. The code used in this example is available as a script here.
import sys
if "google.colab" in sys.modules: # Colab lacks PyBADS: install it
%pip install "pybads>=1.5.1"
import numpy as np
from pybads import BADS
0. Noisy optimization#
PyBADS is also able to optimize noisy objective functions. A noisy (or stochastic) objective function is an objective that will return different results if evaluated twice at the same point \(\mathbf{x}\). Conversely, a non-noisy objective function is known as noiseless or deterministic. For example, noisy objectives are common in model fitting when the model is evaluated through simulation (e.g., via sampling aka Monte Carlo methods).
For a noisy objective, PyBADS aims to minimize the expected value of \(f(\mathbf{x})\):
1. Problem setup#
For this example, we take as target a quadratic function and we add i.i.d. Gaussian noise to it (noisy sphere). In a real case, the noise would arise from some stochastic process in the calculation of the target; here it comes from the target’s own random number generator, noise_rng.
We also set here several options for the optimization:
We tell
badsthat the target is noisy by activating theuncertainty_handlingoption. This is not strictly needed, asbadscan automatically detect if a target is noisy, but it is good practice to specify.We also limit the number of function evaluations with
max_fun_evals, knowing that this is a simple example. Generally,badswill tend to run for longer on noisy problems to better explore the noisy landscape.We ask
badsto re-evaluate the target at the returned solution with 100 samples vianoise_final_samples(by default,noise_final_samples = 10, but since our function is inexpensive we can use more evaluations). These evaluations count towards the total budget of function evaluationsmax_fun_evals.Finally, we fix the random seed of the run with
random_seed, so that the example is reproducible.random_seeddoes not seed the target, so we givenoise_rnga fixed seed too, when we create it right before the run (the PyBADS FAQ says how to make a run reproducible).
def noisy_sphere(x, sigma=1.0):
"""Simple quadratic function with added noise."""
x_2d = np.atleast_2d(x)
f = np.sum(x_2d**2, axis=1)
noise = sigma * noise_rng.normal(size=x_2d.shape[0])
return f + noise
x0 = np.array([-3, -3]) # Starting point
lower_bounds = np.array([-5, -5])
upper_bounds = np.array([5, 5])
plausible_lower_bounds = np.array([-2, -2])
plausible_upper_bounds = np.array([2, 2])
options = {
"uncertainty_handling": True,
"max_fun_evals": 300,
"noise_final_samples": 100,
"random_seed": 0, # Makes the run reproducible
}
2. Run the optimization#
We create the target’s random number generator, noise_rng, and run bads with the user-defined options; running this cell again repeats the run.
noise_rng = np.random.default_rng(8) # Makes the target's noise reproducible
bads = BADS(
noisy_sphere,
x0,
lower_bounds,
upper_bounds,
plausible_lower_bounds,
plausible_upper_bounds,
options=options,
)
optimize_result = bads.optimize()
Beginning optimization of a STOCHASTIC objective function
Tip: To hold a parameter at a known value, set its four bounds (lb, ub, plb and pub) to that value, and its entry of x0 to the same value or NaN. PyBADS optimizes only the other parameters and still passes all of them to your objective, so your code needs no change.
https://acerbilab.github.io/pybads/faq.html#faq-can-i-set-lb-ub-for-some-variable-to-fix-it-to-a-given-value
Iteration f-count E[f(x)] SD[f(x)] MeshScale Method Actions
0 1 16.2617 nan 1
0 33 -0.240306 nan 1 Initial mesh Initial points
1 37 -0.240306 1 0.5 Refine grid Train
2 39 -0.34589 0.283683 0.5 Incremental search (ES-wcm)
2 40 -0.427625 0.270467 0.5 Incremental search (ES-wcm)
2 45 -0.427625 0.270467 0.25 Refine grid Train
3 53 -0.516785 0.289222 0.125 Refine grid
4 54 -0.42842 0.215627 0.125 Incremental search (ES-ell)
4 61 -0.42842 0.215627 0.0625 Refine grid
5 62 -0.122886 0.18489 0.0625 Successful search (ES-ell)
5 73 -0.122886 0.18489 0.03125 Refine grid
6 81 -0.00407971 0.155754 0.015625 Refine grid
7 89 0.0818933 0.148229 0.0078125 Refine grid
8 90 0.124748 0.143148 0.0078125 Successful search (ES-ell)
8 91 0.11678 0.141836 0.0078125 Successful search (ES-ell)
8 92 0.088315 0.145141 0.0078125 Successful search (ES-wcm)
8 94 0.0834316 0.143771 0.0078125 Successful search (ES-wcm)
8 95 0.0660064 0.149626 0.0078125 Successful search (ES-wcm)
8 100 0.0564396 0.145342 0.0078125 Successful search (ES-wcm)
8 102 0.0526485 0.147265 0.0078125 Successful search (ES-ell)
8 104 0.0482 0.145731 0.0078125 Successful search (ES-ell)
8 107 0.0380541 0.144446 0.0078125 Successful search (ES-ell)
8 117 0.0155354 0.128363 0.015625 Successful poll Train
9 125 0.00257922 0.111075 0.0078125 Refine grid Train
10 133 0.0355806 0.105881 0.00390625 Refine grid
11 141 0.0300042 0.101277 0.00195312 Refine grid
12 145 0.0264208 0.09541 0.00195312 Successful search (ES-wcm)
12 146 0.0259126 0.0943858 0.00195312 Incremental search (ES-wcm)
12 147 0.0103826 0.0930927 0.00195312 Successful search (ES-wcm)
12 148 0.00128948 0.092135 0.00195312 Successful search (ES-wcm)
12 149 -0.0143467 0.0933086 0.00195312 Successful search (ES-wcm)
12 157 -0.0143467 0.0933086 0.000976562 Refine grid Train
13 165 0.00562774 0.090948 0.000488281 Refine grid Train
14 166 0.0276749 0.089581 0.000488281 Successful search (ES-wcm)
14 168 0.0191498 0.0888796 0.000488281 Successful search (ES-ell)
14 177 0.0191498 0.0888796 0.000244141 Refine grid Train
15 178 0.0165261 0.0861189 0.000244141 Successful search (ES-ell)
15 189 0.0125952 0.0841009 0.000488281 Successful poll
16 196 0.0125952 0.0841009 0.000244141 Refine grid Train
17 197 0.0507351 0.0818407 0.000244141 Successful search (ES-wcm)
Optimization terminated: reached maximum number of function evaluations options['max_fun_evals'].
Estimated function value at minimum: -0.008937589566970619 ± 0.10285176296994747 (mean ± SEM from 100 samples)
3. Results and conclusions#
First, note that in this case optimize_result['fval'] is the estimated function value at optimize_result['x'], obtained by taking the mean of options['noise_final_samples'] target evaluations (noise_final_samples = 10 by default, but here we used 100). The uncertainty of the value of the function at the returned solution, optimize_result['fsd'], is the standard error of the mean.
If needed, the final samples used to estimate fval and fsd can be found in optimize_result['yval_vec'].
x_min = optimize_result["x"]
fval = optimize_result["fval"]
fsd = optimize_result["fsd"]
print(
f"BADS minimum at: x_min = {x_min.flatten()}, fval (estimated) = {fval:.4g} +/- {fsd:.2g}"
)
print(
f"total f-count: {optimize_result['func_count']}, time: {round(optimize_result['total_time'], 2)} s"
)
print(f"final evaluations (shape): {optimize_result['yval_vec'].shape}")
BADS minimum at: x_min = [ 0.14642758 -0.02136089], fval (estimated) = -0.008938 +/- 0.1
total f-count: 300, time: 2.76 s
final evaluations (shape): (100,)
We can also check the ground-truth value of the target function at the returned point once we remove the noise:
print(
f"The true, noiseless value of f(x_min) is {noisy_sphere(x_min,sigma=0)[0]:.3g}."
)
The true, noiseless value of f(x_min) is 0.0219.
Compare this to the true global minimum of the sphere function at \(\textbf{x}^\star = [0,0]\), where \(f^\star = 0\).
Remarks#
While PyBADS can handle noisy targets, it cannot handle arbitrarily large noise: the PyBADS FAQ says how much noise it handles, and what to do when the noise is larger.
In this example, we assumed the amount of noise in each target evaluation is unknown. If instead you can estimate the magnitude of the noise for each evaluation, see the next example notebook.
For more information on optimizing noisy objective functions, see the PyBADS FAQ.