BADS

Note

The BADS class implements the Bayesian Adaptive Direct Search (BADS) algorithm.

BADS attempts to solve an unbounded, bounded or nonlinearly constrained optimization (minimization) problem, and is compatible with both noiseless and noisy target functions.

To perform the optimization, first initialize a BADS object and then call bads.optimize() on the instance.

See below for more details on the BADS class methods and interface. The primary entry-points for users are the BADS class, which initializes the algorithm, and the OptimizeResult class, which represents the returned optimization solution. The Basic options may also be useful.

class pybads.bads.BADS(fun: callable, x0: ndarray = None, lower_bounds: ndarray = None, upper_bounds: ndarray = None, plausible_lower_bounds: ndarray = None, plausible_upper_bounds: ndarray = None, non_box_cons: callable = None, gamma_uncertain_interval=None, options: dict = None)

BADS Constrained optimization using Bayesian Adaptive Direct Search.

BADS attempts to solve problems of the form:

\(\mathtt{argmin}_x f(x)\) subject to: lower_bounds \(<= x <=\) upper_bounds, and optionally \(C(x) <= 0\)

Initialize a PyBADS object to set up the optimization problem, then run optimize(). See the examples for more details under the examples directory.

Parameters:

funcallable

A given target fun. fun accepts input x and returns a scalar function value of the target evaluated at x and the noise if provided. In case the target function fun requires additional data/parameters, they can be handled using an anonymous function. For example: fun_for_pybads = lambda x: fun(x, data, extra_params), where fun is the function to optimize, and data and extra_params are given in the outer scope.

x0np.ndarray, optional

Starting point for the optimization. If not specified or None, the starting point x0 is uniformly randomly drawn inside the plausible box between plausible_lower_bounds and plausible_upper_bounds (see below).

lower_bounds, upper_boundsnp.ndarray, optional

lower_bounds (lb) and upper_bounds (ub) define a set of strict lower and upper bounds for the coordinate vector, x, so that the unknown function has support on lb <= x <= ub. If scalars, the bound is replicated in each dimension. Use None for lb and ub if no bounds exist. Set lb[i] = -inf and ub [i] = inf if the i-th coordinate is unbounded (while other coordinates may be bounded). Note that if lb and ub contain unbounded variables, the respective values of plb and pub need to be specified (see below). By default None.

plausible_lower_bounds, plausible_upper_boundsnp.ndarray, optional

Specifies a set of plausible_lower_bounds (plb) and plausible_upper_bounds (pub) such that lb <= plb < pub <= ub. Both plb and pub need to be finite. plb and pub represent a plausible range, which should denote a region where the global minimum is expected to be found. As a rule of thumb, set plausible_lower_bounds and plausible_upper_bounds such that there is > 90% probability that the minimum is found within the box (where in doubt, just set plb = lb and pub = ub).

non_box_cons: callable, optional

A given non-box constraints function that specifies constraint violations, e.g : lambda x: np.sum(x.^2,1)>1

optionsdict, optional

Additional options can be passed as a dict. Please refer to the BADS options page for the default options. If no options are passed, the default options are used. To run BADS on a noisy (stochastic) objective function, set options['uncertainty_handling'] = True. You can help BADS by providing an estimate of the noise. options['noise_size'] = sigma provides a global estimate of the SD of the noise in your problem in a good region of the parameter space. (If not specified, default sigma = 1.0). Alternatively, you can specify the target noise at each location with options['specify_target_noise'] = True. In this case, fun is expected to return two values, the estimate of the target at x and an estimate of the SD of the noise at x (see the examples). If options['uncertainty_handling'] is not specified, BADS will determine at runtime if the objective function is noisy. To obtain reproducible results of the optimization, set options['random_seed'] to a fixed integer (see rng below).

Attributes:

rngnumpy.random.Generator

The generator of every random draw of the run, including the random x0. It is created with the BADS object from options['random_seed'], which takes what numpy.random.default_rng takes, such as an integer or a SeedSequence, or a Generator, which is used as given; a float that is a whole number is converted to an integer. A change of the option after the object is created has no effect. If the option is None (default), the generator is derived from NumPy’s global random state, so that np.random.seed before creating the BADS object fixes the run; deriving it advances that state by four draws. Apart from those draws, the run neither draws from nor seeds NumPy’s global random state; a target that draws from it is not fixed by random_seed. Draws from rng before optimize() change the run.

Raises:

ValueError

When neither x0 or (plausible_lower_bounds and plausible_upper_bounds) are specified.

ValueError

When various checks for the bounds (lower_bounds, upper_bounds, plausible_lower_bounds, plausible_upper_bounds) of BADS fail.

TypeError

When options['random_seed'] is a float that is not a whole number, a string, or another value that numpy.random.default_rng does not take.

References

[1]

Singh, S. G. & Acerbi, L. (2024). “PyBADS: Fast and robust black-box optimization in Python”. Journal of Open Source Software, 9(94), 5694, https://doi.org/10.21105/joss.05694.

[2]

Acerbi, L. & Ma, W. J. (2017). “Practical Bayesian Optimization for Model Fitting with Bayesian Adaptive Direct Search”. In Advances in Neural Information Processing Systems 30, pages 1834-1844. (arXiv preprint: https://arxiv.org/abs/1705.04405).

Examples

For BADS usage examples, please look up the Jupyter notebook tutorials in the PyBADS documentation: https://acerbilab.github.io/pybads/examples.html

optimize()

Run the optimization on an initialized PyBADS object.

BADS starts at X0 and finds a local minimum X of the target function ‘fun’.

A history of the optimization problem can be found at the self.iteration_history variable of the PyBADS object.

Returns:

optimize_result: OptimizeResult

Dictionary containing the result of the optimization. See the documentation of the OptimizeResult class for more details. For example, retrieve the final solution with the following attributes:

• optimize_result.x

• optimize_result.fval