VBMC options

Pass option values through the options argument of pyvbmc.VBMC. Basic options control common aspects of a run. Advanced options expose specialized algorithm settings and are rarely needed.

Basic options

[BasicOptions]
# Level of display ("off", "iter", or "full"); "off" prints warnings only
display = "iter"
# Whether to show occasional runtime tips, and the reminder that the installed release is more than a year old, at the start of new runs
show_tips = True
# Plot marginals of variational posterior at each iteration
plot = False
# Max number of iterations
max_iter = 50*(2+D)
# Maximum function-equivalent evaluation budget, including initialization_cost
max_fun_evals = 50*(2+D)
# Number of target evaluations per iteration
fun_evals_per_iter = 5
# Required stable target evaluations for termination
tol_stable_count = 60
# Number of variational components to refine posterior at termination
min_final_components = 50
# Whether the target log joint function returns a noise estimate (SD) as second output, True or False
specify_target_noise = False
# Target accepts batches with shape (N, D): the initial design is one batch and later calls contain one row; returns one value per row
vectorized_target = False
# Name of file to write logs to (if None, no log file will be written)
log_file_name = None
# Logging level for the log file, which is written whenever log_file_name is set: "off", "iter" or "full", or a level of the logging module, that is a non-negative integer (e.g. logging.DEBUG)
log_file_level = "iter"
# Logging write mode, one of ("w", "w+", or "a")
log_file_mode = "a"
# Whether to re-print column headers for optimization statistics at each iteration. If None, PyVBMC tries to guess based on plotting options.
print_iteration_header = None

Integer-valued variables

Integer variables are experimental. Set integer_vars to either a Boolean mask with one entry per variable or a sequence of zero-based variable indices. Place each hard bound half an integer outside the allowed range. For example, use bounds -0.5 and 10.5 for the integers 0 through 10. A prior passed through prior must cover these hard bounds; the corresponding uniform prior is UniformBox(-0.5, 10.5).

VBMC snaps points found by the active-sampling search to the integer grid. It does not snap the initial design, including x0. The first fun_eval_start evaluations may therefore be off-grid unless the supplied starting points fill the whole initial design. On a grid, the search may also select an input evaluated earlier. For a noiseless target, that call spends an evaluation without adding an observation. For a noisy target, repeated calls can refine the estimate at that input; max_repeated_observations limits consecutive repeated measurements. A repeated measurement at an off-grid point from the initial design stays at that point.

Advanced options

[AdvancedOptions]
# Whether the target log-density is noisy with a noise level VBMC has to infer, True or False; leave it empty to follow specify_target_noise, which instead takes the noise estimate the target returns
uncertainty_handling = []
# Experimental integer variables, given as a Boolean mask or 0-based indices; hard bounds sit half an integer outside the range, and prior= must cover them (e.g. UniformBox(-0.5, 10.5)); see the options documentation
integer_vars = []
# Base observation noise magnitude (standard deviation), a positive finite number, read as tol_gp_noise when smaller; leave it empty ([] or None) for the default; not read with specify_target_noise, whose target returns the noise of each evaluation
noise_size = []
# Max number of consecutive repeated measurements for noisy inputs
max_repeated_observations = 0
# Number of initial target fcn evals
fun_eval_start = 10 * ceil((D + 1) / 10)
# Base step size for stochastic gradient descent
sgd_step_size = 0.005
# Skip active sampling the first iteration after warmup
skip_active_sampling_after_warmup = False
# Use ranking criterion to pick best non-converged solution
rank_criterion = True
# Required stable iterations to switch entropy approximation
tol_stable_entropy_iters = 6
# Use variable component means for variational posterior
variable_means = True
# Use variable mixture weight for variational posterior
variable_weights = True
# Penalty multiplier for small mixture weights
weight_penalty = 0.1
# Run in diagnostics mode to get additional info (not used, the iteration history always keeps a lean GP; record_full_history_details is the nearest knob)
diagnostics = False
# Output function (not used, MATLAB VBMC declares it and never calls it)
output_fcn = []
# Fraction of allowed exceptions when computing iteration stability
tol_stable_excpt_frac = 0.2
# Evaluated log-joint values at X0: with a separate prior (prior or log_prior), the value of the target plus the log prior, where the precomputed_evaluations argument takes the value of the target alone (values only, so not for a target with specify_target_noise, whose observations go through the precomputed_evaluations argument)
f_vals = []
# Weighted proposal fcn for uncertainty search (not used, MATLAB VBMC stores the option and never reads it)
proposal_fcn = None
# Automatic nonlinear rescaling of variables (not used, the parameter transform always rescales)
nonlinear_scaling = True
# Search acquisition fcn(s): a list of objects of the classes of pyvbmc.acquisition_functions, or of strings that name one with literal arguments, such as "AcqFcnVIQR(quantile=0.9)". If length is greater than 1, a function is randomly selected for each new point.
search_acq_fcn = [AcqFcnLog()]
# Samples for fast acquisition fcn eval per new point
ns_search = 2 ** 13
# Total samples for Monte Carlo approx. of the entropy
ns_ent = lambda K : 100 * K ** (2 / 3)
# Total samples for preliminary Monte Carlo approx. of the entropy
ns_ent_fast = 0
# Total samples for refined Monte Carlo approx. of the entropy
ns_ent_fine = lambda K : 2 ** 12 * K
# Whether to do a final boost
do_final_boost = True
# Maximum loss in stored boost ELBO and beta-5 ELCBO scores (None keeps legacy penalized, unguarded behavior)
tol_elcbo_boost = 0.1
# Total samples for Monte Carlo approx. of the entropy (final boost)
ns_ent_boost = lambda K : 200 * K ** (2 / 3)
# Total samples for preliminary Monte Carlo approx. of the entropy (final boost)
ns_ent_fast_boost = []
# Total samples for refined Monte Carlo approx. of the entropy (final boost)
ns_ent_fine_boost = []
# Total samples for Monte Carlo approx. of the entropy (active sampling)
ns_ent_active = lambda K : 20 * K ** (2 / 3)
# Total samples for preliminary Monte Carlo approx. of the entropy (active sampling)
ns_ent_fast_active = 0
# Total samples for refined Monte Carlo approx. of the entropy (active sampling)
ns_ent_fine_active = lambda K : 200 * K
# Samples for fast approximation of the ELBO
ns_elbo = lambda K : 50 * K
# Multiplier to samples for fast approx. of ELBO for incremental iterations
ns_elbo_incr = 0.1
# Starting points to refine optimization of the ELBO
elbo_starts = 2
# Max GP hyperparameter samples (decreases with training points)
ns_gp_max = 80
# Max GP hyperparameter samples during warmup
ns_gp_max_warmup = 8
# Max GP hyperparameter samples during main algorithm
ns_gp_max_main = np.inf
# Cap on the number of GP hyperparameter samples of the GP refits within active sampling (0: MAP fit only; inf: as the main fit)
ns_gp_max_active = np.inf
# Fcn evals without improvement before stopping warmup
warmup_no_impro_threshold = 20 + 5 * D
# Also check for max fcn value improvement before stopping warmup
warmup_check_max = True
# Force stable GP hyperparameter sampling (reduce samples or start optimizing)
stable_gp_sampling = 200 + 10 * D
# Force stable GP hyperparameter sampling after reaching this number of components
stable_gp_vp_k = np.inf
# Number of GP samples when GP is stable (0 = optimize)
stable_gp_samples = 0
# Thinning for GP hyperparameter sampling
gp_sample_thin = 5
# Initial design points for GP hyperparameter training
gp_train_n_init = 1024
# Final design points for GP hyperparameter training
gp_train_n_init_final = 64
# Initial design method for GP hyperparameter training
gp_train_init_method = "rand"
# Tolerance for optimization of GP hyperparameters
gp_tol_opt = 1e-5
# Tolerance for optimization of GP hyperparameters preliminary to MCMC
gp_tol_opt_mcmc = 1e-2
# Tolerance for optimization of GP hyperparameters during active sampling
gp_tol_opt_active = 1e-4
# Tolerance for optimization of GP hyperparameters preliminary to MCMC during active sampling
gp_tol_opt_mcmc_active = 1e-2
# Threshold on GP variance used by regulatized acquisition fcns
tol_gp_var = 1e-4
# Threshold on GP variance, used to stabilize sampling
tol_gp_var_mcmc = 1e-4
# GP mean function ("negquad", "const" or "zero"; the other mean functions of MATLAB VBMC are not ported)
gp_mean_fun = "negquad"
# GP integrated mean function (not used, the integrated mean function of the GP is not ported)
gp_int_mean_fun = 0
# Max variational components as a function of training points
k_fun_max = lambda N : N ** (2 / 3)
# Variational components during warmup
k_warmup = 2
# Added variational components for stable solution (a number, or a function of the current number of components)
adaptive_k = 2
# High Posterior Density region (fraction of training inputs)
hpd_frac = 0.8
# Uncertainty weight on ELCBO for computing lower bound improvement
elcbo_impro_weight = 3
# Minimum fractional length scale
tol_length = 1e-6
# Initial number of rows of the arrays that store the fcn evaluations (they grow as needed)
cache_size = 500
# Fraction of search points from starting cache (if nonempty), in [0, 1]
cache_frac = 0.5
# Stochastic optimizer for variational parameters (only "adam" is implemented; another value raises an error at the first stochastic optimization)
stochastic_optimizer = "adam"
# Stopping threshold for stochastic optimization
tol_fun_stochastic = 1e-3
# Max iterations for stochastic optimization
max_iter_stochastic = 100 * (2 + D)
# Set stochastic optimization stepsize via GP hyperparameters (not used, removed from MATLAB VBMC in 2021 together with the code that read it)
gp_stochastic_step_size = False
# Tolerance on ELBO uncertainty for stopping (if variational posterior is stable)
tol_sd = 0.1
# Stopping threshold on change of variational posterior per training point
tol_skl = 0.01 * np.sqrt(D)
# Number of stable fcn evals for stopping warmup
tol_stable_warmup = 15
# MCMC sampler for variational posteriors (not used, sampling of the variational parameters is not ported)
variational_sampler = "malasample"
# Required ELCBO improvement per fcn eval before termination
tol_improvement = 0.01
# Use Gaussian approximation for symmetrized KL-divergence b\w iters
kl_gauss = True
# True mean of the target density (for debugging)
true_mean = []
# True covariance of the target density (for debugging)
true_cov = []
# Min number of fcn evals
min_fun_evals = 5 * D
# Min number of iterations
min_iter = D
# Fraction of search points from a heavy-tailed variational posterior, in [0, 1]; the five search fractions sum to at most 1, with the remainder drawn from the variational posterior
heavy_tail_search_frac = 0.25
# Fraction of search points from a multivariate normal, in [0, 1]; the five search fractions sum to at most 1, with the remainder drawn from the variational posterior
mvn_search_frac = 0.25
# Fraction of search points from a multivariate normal fitted to HPD points, in [0, 1]; the five search fractions sum to at most 1, with the remainder drawn from the variational posterior
hpd_search_frac = 0
# Fraction of search points from a uniform box based on training inputs, in [0, 1]; the five search fractions sum to at most 1, with the remainder drawn from the variational posterior
box_search_frac = 0.25
# Fraction of search points from previous iterations, in [0, 1]; the five search fractions sum to at most 1, with the remainder drawn from the variational posterior
search_cache_frac = 0
# Always fully refit variational posterior
always_refit_vp = False
# Perform warm-up stage
warmup = True
# Special OPTIONS struct for warmup stage (not used, per-stage option overrides are not ported)
warmup_options = []
# Stop warm-up when ELCBO increase below threshold (per fcn eval)
stop_warmup_thresh = 0.2
# Max log-likelihood difference for points kept after warmup
warmup_keep_threshold = 10 * D
# Max log-likelihood difference for points kept after a false-alarm warmup stop
warmup_keep_threshold_false_alarm = 100 * (D + 2)
# Reliability index required to stop warmup
stop_warmup_reliability = 100
# Local optimizer of the acquisition search: "cmaes" (CMA-ES, replaced by a bounded scalar search where the problem has one variable) or "none" (no local search)
search_optimizer = "cmaes"
# Initialize CMA-ES search SIGMA from variational posterior
search_cmaes_vp_init = True
# Take the best-ever point of the CMA-ES acquisition search (not used, the search always takes it)
search_cmaes_best = False
# Max number of acquisition fcn evaluations during search
search_max_fun_evals = 500 * (D + 2)
# Weight of previous trials (per trial) for running avg of variational posterior moments (the running average is recorded in the optimization state, and nothing reads it)
moments_run_weight = 0.9
# Upper threshold on reliability index for full retraining of GP hyperparameters
gp_retrain_threshold = 1
# Compute full ELCBO also at best midpoint
elcbo_midpoint = True
# Multiplier to widths from previous posterior for GP sampling (Inf = do not use previous widths)
gp_sample_widths = 5
# Weight of previous trials (per trial) for running avg of GP hyperparameter covariance
hyp_run_weight = 0.9
# Use weighted hyperparameter posterior covariance
weighted_hyp_cov = True
# Minimum weight for weighted hyperparameter posterior covariance
tol_cov_weight = 0
# MCMC sampler for GP hyperparameters (only "slicesample" is supported, the other samplers of MATLAB VBMC are not ported)
gp_hyp_sampler = "slicesample"
# Switch to covariance sampling below this threshold of stability index (not used, covariance sampling is not ported)
cov_sample_thresh = 10
# Optimality tolerance for optimization of deterministic entropy
det_entropy_tol_opt = 1e-3
# Switch from deterministic entropy to stochastic entropy when reaching stability
entropy_switch = False
# Force switch to stochastic entropy at this fraction of total fcn evals
entropy_force_switch = 0.8
# Alpha value for lower/upper deterministic entropy interpolation (not used, MATLAB VBMC removed the interpolation in 2021; the value is stored in the optimization state and reaches no computation)
det_entropy_alpha = 0
# Randomize deterministic entropy alpha during active sample updates (no effect on the fit, since the alpha value is not used; with active_sample_vp_update, each such update draws a random number and so advances the run's random stream)
update_random_alpha = False
# Online adaptation of alpha value for lower/upper deterministic entropy interpolation (not used, the adaptation is not ported)
adaptive_entropy_alpha = False
# Start with deterministic entropy only with this number of vars or more
det_entropy_min_d = 5
# Fractional tolerance for constraint violation of variational parameters
tol_con_loss = 0.01
# SD multiplier of ELCBO for computing best variational solution
best_safe_sd = 5
# When computing best solution lacking stability go back up to this fraction of iterations (used only when rank_criterion is False)
best_frac_back = 0.25
# Threshold mixture component weight for pruning
tol_weight = 1e-2
# Multiplier to threshold for pruning mixture weights
pruning_threshold_multiplier = lambda K : 1 / np.sqrt(K)
# Annealing for hyperprior width of GP negative quadratic mean (not used, MATLAB VBMC removed the code that read it in 2021)
annealed_gp_mean = lambda N,NMAX: 0
# Strict hyperprior for GP negative quadratic mean (not used, MATLAB VBMC removed the code that read it in 2021)
constrained_gp_mean = False
# Empirical Bayes prior over some GP hyperparameters (not used, removed from MATLAB VBMC in 2021 together with the code that read it)
empirical_gp_prior = False
# Minimum GP observation noise
tol_gp_noise = np.sqrt(1e-5)
# Prior mean over GP input length scale (in plausible units)
gp_length_prior_mean = np.sqrt(D / 6)
# Prior std over GP input length scale (in plausible units)
gp_length_prior_std = 0.5 * np.log(1e3)
# Upper bound on GP input lengths based on plausible box (0 = ignore)
upper_gp_length_factor = 0
# Initial samples (plausible is uniform in the plausible box)
init_design = "plausible"
# Stricter upper bound on GP negative quadratic mean function
gp_quadratic_mean_bound = True
# Heuristic output warping (not implemented; True only updates an unused threshold and may consume random draws)
fitness_shaping = False
# Output warping starting threshold (not used, the GP output warping of MATLAB VBMC is not ported)
out_warp_thresh_base = 10 * D
# Output warping threshold multiplier when failed sub-threshold check (not used, the GP output warping of MATLAB VBMC is not ported)
out_warp_thresh_mult = 1.25
# Output warping base threshold tolerance (fraction of current threshold) (not used, the GP output warping of MATLAB VBMC is not ported)
out_warp_thresh_tol = 0.8
# Temperature for posterior tempering (allowed values T = 1, 2, 3, 4) (not used, posterior tempering is not ported)
temperature = 1
# Use separate GP with constant mean for active search (not used, a development option of MATLAB VBMC that PyVBMC does not need)
separate_search_gp = False
# Discount observations from extremely low-density regions (must be False, the shaping of MATLAB VBMC's noiseshaping_vbmc.m is not ported)
noise_shaping = False
# Threshold from max observed value to start discounting (not used, noise shaping is not ported)
noise_shaping_threshold = 10 * D
# Proportionality factor of added noise wrt distance from threshold (not used, noise shaping is not ported)
noise_shaping_factor = 0.05
# Hedge on multiple acquisition functions (must be False, the portfolio of MATLAB VBMC's acqhedge_vbmc.m is not ported; one entry of search_acq_fcn is picked at random at each step)
acq_hedge = False
# Past iterations window to judge acquisition fcn improvement (not used, the acquisition-portfolio hedge is not ported)
acq_hedge_iter_window = 4
# Portfolio value decay per function evaluation (not used, the acquisition-portfolio hedge is not ported)
acq_hedge_decay = 0.9
# MCMC variational steps before each active sampling (not used, variational active sampling is not ported)
active_variational_samples = 0
# Apply lower bound on variational components scale during variational sampling (not used, sampling of the variational parameters is not ported)
scale_lower_bound = True
# Perform variational optimization after each active sample
active_sample_vp_update = False
# Perform GP training after each active sample
active_sample_gp_update = False
# # iters past warmup to continue update after each active sample
active_sample_full_update_past_warmup = 2
# Perform full update during active sampling if stability above threshold
active_sample_full_update_threshold = 3
# Use previous variational posteriors to initialize optimization (not used, the repository of previous variational posteriors is not ported)
variational_init_repo = False
# Extra variational components sampled from GP profile (not used, the extra components are not ported)
sample_extra_vp_means = 0
# Uncertainty weight on ELCBO during active sampling (not used, the optimistic bound is not ported)
optimistic_variational_bound = 0
# # importance samples from smoothed variational posterior
active_importance_sampling_vp_samples = 100
# # importance samples from box-uniform centered on training inputs
active_importance_sampling_box_samples = 100
# Noisy-acquisition importance samples, rounded up: VP draws for VIQR (minimum 1), or for IMIQR the MCMC samples kept per GP hyperparameter sample, one in every active_importance_sampling_mcmc_thin steps after a burn-in of half as many steps (0 skips MCMC); a number or function called with K (mixture components), n_vars and D (both variable count) as keywords
active_importance_sampling_mcmc_samples = 100
# Thinning for importance sampling MCMC
active_importance_sampling_mcmc_thin = 1
# Fractional ESS threshold to update GP and VP (not used, the in-loop update is not conditioned on the effective sample size)
active_sample_fess_thresh = 1
# Fractional ESS threshold to do MCMC while active importance sampling (not used, the refinement of the importance samples by an ensemble sampler is not ported)
active_importance_sampling_fess_thresh = 0.9
# Active search bound multiplier
active_search_bound = 2
# Try integrating GP mean function (not used, removed from MATLAB VBMC in 2021 together with the code that read it)
integrate_gp_mean = False
# Tolerance on closeness to bound constraints (fraction of total range)
tol_bound_x = 1e-5
# Recompute LCB max for each iteration based on current GP estimate
recompute_lcb_max = True
# Input transform for bounded variables
bounded_transform = "probit"
# Use double GP (not used, removed from MATLAB VBMC in 2021 together with the code that read it)
double_gp = False
# Warp every this number of iterations
warp_every_iters = 5
# Increase delay between warpings
incremental_warp_delay = True
# Threshold on reliability index to perform warp
warp_tol_reliability = 3
# Rotate and scale input
warp_rotoscaling = True
# Regularization weight towards diagonal covariance matrix in the rotoscaling warp, clamped to [0, 1]: a number, or a function of N, the number of distinct points logged so far (optim_state["N"]), such as lambda N: 10 / N
warp_cov_reg = 0
# Threshold on correlation matrix for roto-scaling
warp_roto_corr_thresh = 0.05
# Min number of variational components to perform warp
warp_min_k = 5
# Immediately undo warp if not improving ELBO
warp_undo_check = True
# Improvement of ELBO required to keep a warp proposal
warp_tol_improvement = 0.1
# Multiplier tolerance of ELBO SD after warp proposal
warp_tol_sd_multiplier = 2
# Base tolerance on ELBO SD after warp proposal
warp_tol_sd_base = 1
# Keep the importance samples of noisy acquisitions in the iteration history (recomputed on resume, not rebuildable from the record); for debugging
record_full_history_details = False
# Fixed numerical chunk profile: "cached" reuses a compatible machine-local calibration, "off" pins historical defaults, or pass a CalibrationProfile
performance_calibration = "cached"