BADS options

The options are divided into two groups:

• Basic options: Commonly adjusted settings, such as evaluation budgets, display and noise handling.

• Advanced options: Specialized features and settings that control the optimization algorithm.

You can find the default options for both groups below. D in the options below is the number of variables that BADS optimizes: all of them but the fixed ones, whose four bounds are equal.

Basic options

We expect these options to be routinely changed by many users.

[BasicOptions]
# Level of display ("iter", "notify", "final", "off", or "full", which also shows debug messages)
display = "iter"
# Show an occasional tip, with a link to the documentation, at the start of a run; False also turns off the reminder to check for updates when the installed release is more than a year old
show_tips = True
# Max number of iterations
max_iter = 200*D
# Max number of target fcn evals
max_fun_evals = 500 * D
# Automatic nonlinear rescaling of variables
nonlinear_scaling = True
# Complete polling around the current iterate
complete_poll = False
# Accelerate mesh contraction
accelerate_mesh = True
# Explicit noise handling (if None, determine at runtime; on with specify_target_noise)
uncertainty_handling = None
# Base observation noise magnitude (SD), e.g. noise_size = 1.0, or a pair [SD, SD of the prior over log SD]; None is 1.0 in a noisy run and sqrt(tol_fun) in a deterministic one; ignored with specify_target_noise
noise_size = None
# Target function returns noise estimate (SD) as second output
specify_target_noise = False
# Samples to estimate FVAL at the end (for noisy objectives; taken from max_fun_evals)
noise_final_samples = 10
# Seed of the run's random draws (integer, SeedSequence or Generator); if None, np.random.seed before creating BADS fixes the run
random_seed = None

Advanced options

This group includes periodic_vars for periodic variables and output_fcn for monitoring or stopping a run. Set these options when your problem calls for them.

Most other options control the search, polling and Gaussian process model. Keep their defaults unless a documented recommendation or a specific algorithmic requirement calls for a change.

[AdvancedOptions]
# Show optimization plots (no effect: not available in PyBADS)
plot = False

# Tolerance and termination conditions
# Tolerance on mesh size
tol_mesh = 1e-6
# Min significant change of objective fcn (a positive number, at most e^6, about 403)
tol_fun = 1e-3
# Max iterations with no significant change (doubled under uncertainty)
tol_stall_iters = int(4 + np.floor(D/2))
# Min variability for a fcn to be considered noisy
tol_noise = np.sqrt(np.spacing(1.0)) * self.get("tol_fun")

#Initialization
# Initial design ('init_sobol', the only one available)
init_fun = "init_sobol"
# Number of restarts after termination (no effect: not available in PyBADS)
restarts = 0
# Initial size of the cache for storing fcn evaluations (it grows as needed)
cache_size = 500
# Size of the initial Sobol design, rounded up to a power of two and doubled when that equals D (at least 20 for a noisy target), then cut to the evaluations that max_fun_evals leaves
fun_eval_start = D
# Earlier fcn evaluations (not supported: pass them to BADS as precomputed_evaluations)
fun_values = {}
# Indices (from 0, counting the fixed variables) of the periodic variables, such as angles, e.g. periodic_vars = [2, 3]; each wraps around its hard bounds, which need to be finite and are its period
periodic_vars = None

# Poll Options
# Poll function (unused: the poll always uses poll_mads_2n)
poll_method                              = 'poll_mads_2n'
# Number of random candidate directions of an alternative poll method (unused)
n_basis                                  = 200 * D
# Mesh multiplicative factor between iterations
poll_mesh_multiplier                      = 2.0
# Force poll vectors to be on mesh
force_poll_mesh                           = False
# Maximum poll integer
max_poll_grid_number                       = 0
# Use alternative incumbent offset
alternative_incumbent                    = False
# Adaptive multiplier to incumbent uncertainty
adaptive_incumbent_shift                  = False
# Power of the GP's length scales in the scaling of the ES-ell search (it does not change the poll, despite its name)
gp_rescale_poll                           = 1.0
# Threshold probability of improvement (PoI) for stopping a poll early (complete_poll makes every poll complete)
tol_poi                                  = 1e-6/D
# Skip polling if PoI below threshold, even with no success (unused)
skip_poll                                = True
# Allow consecutive incomplete polls
consecutive_skipping                     = True
# Skip polling after successful search
skip_poll_after_search                     = True
# Number of failed fcn evaluations before skipping is allowed
min_failed_poll_steps                      = np.inf
# Accelerate mesh after this number of stalled iterations (a positive integer)
accelerate_mesh_steps                     = 3
# Move incumbent even after insufficient improvement, and floor the sufficient improvement at tol_fun
sloppy_improvement                       = True
# Deprecated along with stobads: under Sto-BADS, move the incumbent on an uncertain improvement
opp_stobads                             = True
# Deprecated along with stobads: power of the mesh size in the Sto-BADS interval gamma * SD * mesh_size**power: at 2, a difference of a fraction of its SD counts as certain at a small mesh; 0 makes the rule a z-test
stobads_frame_size_scaling_power        = 2
# Threshold # mesh overflows for warning (doubled under uncertainty)
mesh_overflow_warning                    = 2 + D/2
# Initial mesh size (power value)
init_mesh_size_integer                  = 0

# Deprecated, may be removed in a future release: Sto-BADS, an experimental success rule not found to improve noisy runs; not the setting for a noisy target, which takes uncertainty_handling = True
stobads = False

# Improvement parameters
# Minimum significant improvement at unit mesh size
tol_improvement          = 1
# Exponent of forcing function
forcing_exponent         = 3/2
# Multiplier to incumbent uncertainty for acquisition functions
incumbent_sigma_multiplier = 0.1
# Quantile when computing improvement, greater than 0 and less than 1 (<0.5 for conservative improvement)
improvement_quantile     = 0.5
# Top quantile when choosing final iteration
final_quantile           = 1e-3

# Search properties
# Number of candidate search points (a positive integer)
n_search                 = 2**12
# Number of optimization iterations for search (a positive integer, at most n_search; each generation has n_search / n_search_iter candidates, rounded down)
n_search_iter             = 2
# Multiplier in ES
es_beta                  = 1
# Starting scale value in ES
es_start                 = 0.25
# Fraction of candidate search points with (slower) improved estimate (unused)
search_improve_frac       = 0
# Search radius expansion factor for successful search
search_scale_success      = np.sqrt(2)
# Search radius expansion factor for incremental search
search_scale_incremental  = 2
# Search radius contraction factor for failed search
search_scale_failure      = np.sqrt(0.5)
# Lower bound on the search radius factor, which failed searches contract
search_factor_min         = 0.5
# Search function(s): a non-empty list of pairs (name, sum-rule flag), each name 'ES-wcm' or 'ES-ell'; only ES-wcm reads the flag
search_method            = [('ES-wcm',1), ('ES-ell',1)]
# Iteration scale factor between poll and search
search_grid_number        = 10
# Multiplier integer scale factor between poll and search
search_grid_multiplier    = 2
# Relative search scale factor locked to poll scale factor
search_size_locked       = True
# Number of searches per iteration
search_n_try              = int(np.maximum(D, np.floor(3 + D/2)))
# Search-triggered mesh expansion after this number of successful search rounds
search_mesh_expand        = 0
# Mesh size increment after search-triggered mesh expansion
search_mesh_increment     = 1
# Further optimize acquisition function (no effect: not available in PyBADS)
search_optimize          = False

# Noise parameters
# Treat incumbent as if uncertain regardless of uncertainty handling
uncertain_incumbent      = True
# Contribution to log noise magnitude from log mesh size (0 for noisy functions)
mesh_noise_multiplier      = 0.5

# Gaussian process properties
# Number of training data (minimum 200 under uncertainty)
n_train_max                = 50 + 10*D
# Minimum number of training data (doubled under uncertainty)
n_train_min                = 50
# Max number of training data removed if too far from current point
buffer_ntrain             = 100
# Hyperparameter samples (unused: PyBADS optimizes one set of GP hyperparameters, where MATLAB BADS samples them above 1)
gp_samples                 = 0
# Sampler that use_slice_sampler selects (only 'slicesample')
gp_hyp_sampler = "slicesample"
# Fraction of the training points, those of lowest value, used to initialize the first GP
hpd_frac = 0.8
# Weight of previous trials for a running average of the GP hyperparameter covariance (no effect)
hyp_run_weight = 1
# Minimum fcn evals before refitting the GP
min_refit_time            = 2*D
# Train GP also during poll stage
poll_training            = True
# Always try a second GP fit
double_refit             = False
# GP mean function ('const', a constant, or 'zero')
gp_mean_fun              = 'const'
# Percentile of empirical GP mean
gp_mean_percentile        = 90
# Empirical range of hyperprior over the mean
gp_mean_range_fun          = lambda ym,y: (ym - np.median(y))/5*2
# GP definition fcn (unused: the GP has a rational-quadratic ARD kernel)
gp_def_fcn                = ('gp_def_bads','rq',[1,1])
# GP training set selection method (unused: always the nearest neighbours)
gp_method                = 'nearest'
# Cluster additional points during training (unused)
gp_cluster               = False
# Rotate GP basis (unused)
rotate_gp                = False
# Radius of training set
gp_radius                = 3
# Scale the radius of the training set by the effective radius of the GP's rational-quadratic kernel
use_effective_radius      = True
# Empirical GP hyper-prior over the length scales, 'iso' ('ard' is unsupported)
gp_cov_prior              = 'iso'
# Fix the GP mean at a high percentile of the training targets (incomplete: True keeps the width of the prior over the mean, but does not fix the mean)
gp_fixed_mean             = False
# Fit the likelihood term, the GP noise (False, a fixed noise, is unsupported)
fit_lik                  = True
# Acquisition fcn for poll stage (unused: the poll always ranks by the LCB with the default schedule)
poll_acq_fcn              = ('acq_LCB', None)
# Acquisition fcn for search stage, ('acq_LCB', sqrt_beta): sqrt_beta None (the default schedule), a positive finite number, or a callable sqrt_beta(t, D) that returns one, with t the number of fcn evals so far plus 1
search_acq_fcn            = ('acq_LCB', None)
# Hedge acquisition function (not supported: BADS refuses True)
acq_hedge                = False
# Attempts at performing the Cholesky decomposition (unused)
chol_attempts            = 0
# Increase nudge to noise in case of Cholesky failure
noise_nudge             = np.array([1, 0])
# Start removing training points after this number of failures
remove_points_after_tries  = 1
# SVGD iterations for GP training (unused: the GP hyperparameters are not sampled)
gp_svd_iters              = 200
# Issue warning if GP hyperparameters fit fails
gp_warnings             = False
# Alpha level for normality test of gp predictions
normalpha_level          = 1e-6
# Number of target fcn evals per iteration (no effect: BADS evaluates one point at a time)
fun_evals_per_iter = 1
# Initial design points for GP hyperparameter training
gp_train_n_init = 128
# Final design points for GP hyperparameter training
gp_train_n_init_final = 8
# Initial design method for GP hyperparameter training
gp_train_init_method = "rand"
# Tolerance for optimization of GP hyperparameters
gp_tol_opt = 1e-5
# Floor of the stricter bound on a negative quadratic GP mean (no effect: that mean is not available)
tol_sd = 0.1
# GP output warping (only 0 is supported; another value fails during the run)
warp_func = 0
# Draw the starting point of a second GP fit, or of a retry after a failed refit, by slice sampling rather than from the priors
use_slice_sampler = False

# Adaptive basis (unsupported)
# Update Hessian as you go (unused)
hessian_update           = False
# Hessian update method (unused)
hessian_method           = 'bfgs'

# Hedge heuristic parameters (currently used during the search stage)
# Minimum probability of each search in the hedge's choice of a search, from 0 to 1/n for the n searches of search_method
hedge_gamma              = 0.125
# Inverse temperature of the hedge's softmax choice of a search from the searches' gains, finite and at least 0 (0 is a uniform choice)
hedge_beta               = 1e-3/self.get('tol_fun')
# Decay factor of the hedge's gains at each update, from 0 to 1 (1 is no decay)
hedge_decay              = 0.1**(1/(2*D))

# Output function, called as output_fcn(x, optim_state, state) with state 'init', 'iter' (after each poll) or 'done', and a copy of optim_state; a true return value stops the run
output_fcn = None

# Function values at X0 (not supported: leave it None; pass earlier evaluations to BADS as precomputed_evaluations)
f_vals = None
# Stricter upper bound on a negative quadratic GP mean (no effect: that mean is not available)
gp_quadratic_mean_bound = True
# Heuristic output warping, fitness shaping (no effect: not available in PyBADS)
fitness_shaping = False
# Noise shaping (no effect in PyBADS)
noise_shaping = False