Back

A Tutorial on Bayesian Optimization

Peter I. Frazier

article2018arXiv2,481 citations

Presents a comprehensive guide to optimizing computationally expensive objective functions with Bayesian optimization, covering core acquisition functions, advanced multi-fidelity settings, and a decision-theoretic generalization of expected improvement for noisy evaluations.

Listen

The article addresses the challenge of optimizing expensive black-box objective functions that require minutes or hours per evaluation, lack derivatives, and have no known structure to exploit. Such problems arise in hyperparameter tuning for machine learning models, engineering system design, materials discovery, and laboratory experiments, where only a few hundred evaluations are feasible.

The tutorial sets out to explain how Bayesian optimization solves these problems, detail its core components, and extend the approach to more complex settings. It covers Gaussian process regression for building a probabilistic surrogate model, three main acquisition functions for choosing evaluation points, and a range of advanced variants.

The method begins with an initial set of evaluations, fits a Gaussian process surrogate that quantifies uncertainty, and then repeatedly selects the next point by maximizing an acquisition function. Expected improvement is the most common choice in the basic noise-free case because it balances predicted performance against uncertainty. Knowledge gradient, entropy search, and predictive entropy search provide stronger performance when evaluations are noisy or when value arises from improving beliefs across the entire domain rather than only at the sampled point.

Key extensions allow parallel evaluations, multi-fidelity and multi-information-source queries, expensive constraints, random environmental conditions, multi-task objectives, and derivative observations. In these exotic settings, knowledge gradient and entropy-based methods often outperform expected improvement because they account for how a measurement updates beliefs over the full feasible region.

These techniques matter because they enable reliable global optimization of costly functions that defeat gradient-based or local search methods, reducing the number of evaluations needed and supporting high-stakes decisions in science and engineering. Software packages such as GPyOpt, Spearmint, and Cornell MOE make the methods accessible.

Next steps include developing finite-time performance guarantees, scaling methods to higher dimensions through structure discovery, creating new statistical models for specific applications, and applying the framework to chemistry, materials design, and drug discovery. Limitations include the assumption of moderate input dimensions (typically under 20), the computational cost of optimizing acquisition functions in some variants, and the lack of fully multi-step optimal policies for general problems; readers should verify that problem characteristics match the method's assumptions before deployment.

bayesian methodsdecision-theoreticgaussian processesBayesian OptimizationGaussian ProcessExpected Improvement
arXiv: 1807.02811
Cover for A Tutorial on Bayesian Optimization

Abstract

Bayesian optimization is an approach to optimizing objective functions that take a long time (minutes or hours) to evaluate. It is best-suited for optimization over continuous domains of less than 20 dimensions, and tolerates stochastic noise in function evaluations. It builds a surrogate for the objective and quantifies the uncertainty in that surrogate using a Bayesian machine learning technique, Gaussian process regression, and then uses an acquisition function defined from this surrogate to decide where to sample. In this tutorial, we describe how Bayesian optimization works, including Gaussian process regression and three common acquisition functions: expected improvement, entropy search, and knowledge gradient. We then discuss more advanced techniques, including running multiple function evaluations in parallel, multi-fidelity and multi-information source optimization, expensive-to-evaluate constraints, random environmental conditions, multi-task Bayesian optimization, and the inclusion of derivative information. We conclude with a discussion of Bayesian optimization software and future research directions in the field. Within our tutorial material we provide a generalization of expected improvement to noisy evaluations, beyond the noise-free setting where it is more commonly applied. This generalization is justified by a formal decision-theoretic argument, standing in contrast to previous ad hoc modifications.

Topics

bayesian methodsdecision-theoreticgaussian processesgaussiansoptimizationoptimization problemssoftwarestochasticmachine learning

Keywords

Bayesian OptimizationGaussian ProcessExpected ImprovementKnowledge GradientEntropy SearchMulti‑fidelityParallel EvaluationNoisy EvaluationDerivative‑freeAcquisition Function

Table of Contents

  • 1 Introduction
  • 2 Overview of BayesOpt
  • 3 Gaussian Process (GP) Regression
  • 3.1 Choosing a Mean Function and Kernel
  • 3.2 Choosing Hyperparameters
  • 4 Acquisition Functions
  • 4.1 Expected Improvement
  • 4.2 Knowledge Gradient
  • 4.3 Entropy Search and Predictive Entropy Search
  • 4.4 Multi-Step Optimal Acquisition Functions
  • 5 Exotic Bayesian Optimization
  • 6 Software
  • 7 Conclusion and Research Directions
  • References

Knowls

  1. Knowl 1 — Basic Sequential Bayesian Optimization Algorithm

    algorithm

    Bayesian optimization (BayesOpt) is an iterative framework for global optimization of expensive, derivative-free black-box functions f:AoRf: A o \mathbb{R} over a domain A⊆RdA \subseteq \mathbb{R}^d. It maintains a Gaussian process surrogate model over ff and uses an acquisition function to determine optimal sampling locations under a budget of NN evaluations.

    Input: Objective function ff, feasible domain AA, initial sample budget n0n_0, total evaluation budget NN, prior Gaussian process (μ0,Σ0)(\mu_0, \Sigma_0), acquisition function αn(x)\alpha_n(x)
    Output: Recommended solution x∗∈Ax^* \in A
    Place Gaussian process prior (μ0,Σ0)(\mu_0, \Sigma_0) on ff
    Evaluate ff at n0n_0 initial points chosen according to a space-filling design (e.g., uniform random sampling in AA)
    Set n=n0n = n_0
    while n≤Nn \le N do
        Update Gaussian process posterior mean μn(x)\mu_n(x) and variance σn2(x)\sigma_n^2(x) using all accumulated evaluations {(xi,yi)}i=1n\{(x_i, y_i)\}_{i=1}^n
        Find xn∈arg⁡max⁡x∈Aαn(x)x_n \in \arg\max_{x \in A} \alpha_n(x) by maximizing the acquisition function computed under the current posterior
        Evaluate yn=f(xn)y_n = f(x_n)
        n=n+1n = n + 1
    end while
    Return x∗=arg⁡max⁡x∈{x1,…,xN}f(x)x^* = \arg\max_{x \in \{x_1, \dots, x_N\}} f(x) or x∗=arg⁡max⁡x∈AμN(x)x^* = \arg\max_{x \in A} \mu_N(x)

    The algorithm updates its uncertainty after every evaluation, balancing the exploration of uncertain regions with the exploitation of high predicted performance.

  2. Knowl 2 — Gaussian Process Posterior Inference for Noise-Free Evaluations

    equation

    Let f:Rd→Rf: \mathbb{R}^d \to \mathbb{R} be modeled as a Gaussian process with prior mean function μ0:Rd→R\mu_0: \mathbb{R}^d \to \mathbb{R} and positive semi-definite covariance kernel Σ0:Rd×Rd→R\Sigma_0: \mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}. Given a sequence of nn noise-free observations f(x1:n)=[f(x1),…,f(xn)]Tf(x_{1:n}) = [f(x_1), \dots, f(x_n)]^T at input locations x1:n=[x1,…,xn]x_{1:n} = [x_1, \dots, x_n], the posterior distribution of the function value f(x)f(x) at an unobserved point x∈Rdx \in \mathbb{R}^d is normally distributed:

    f(x)∣f(x1:n)∼N(μn(x),σn2(x))f(x) \mid f(x_{1:n}) \sim \mathcal{N}\left(\mu_n(x), \sigma_n^2(x)\right)

    where the posterior mean μn(x)\mu_n(x) and posterior variance σn2(x)\sigma_n^2(x) are given analytically by:

    μn(x)=Σ0(x,x1:n)Σ0(x1:n,x1:n)−1(f(x1:n)−μ0(x1:n))+μ0(x)\mu_n(x) = \Sigma_0(x, x_{1:n}) \Sigma_0(x_{1:n}, x_{1:n})^{-1} (f(x_{1:n}) - \mu_0(x_{1:n})) + \mu_0(x)

    σn2(x)=Σ0(x,x)−Σ0(x,x1:n)Σ0(x1:n,x1:n)−1Σ0(x1:n,x)\sigma_n^2(x) = \Sigma_0(x, x) - \Sigma_0(x, x_{1:n}) \Sigma_0(x_{1:n}, x_{1:n})^{-1} \Sigma_0(x_{1:n}, x)

    Here, Σ0(x,x1:n)\Sigma_0(x, x_{1:n}) is the 1×n1 \times n vector with entries Σ0(x,xi)\Sigma_0(x, x_i), Σ0(x1:n,x1:n)\Sigma_0(x_{1:n}, x_{1:n}) is the n×nn \times n covariance matrix with entries [Σ0(x1:n,x1:n)]i,j=Σ0(xi,xj)[\Sigma_0(x_{1:n}, x_{1:n})]_{i,j} = \Sigma_0(x_i, x_j), and Σ0(x1:n,x)=Σ0(x,x1:n)T\Sigma_0(x_{1:n}, x) = \Sigma_0(x, x_{1:n})^T.

  3. Knowl 3 — Expected Improvement Acquisition Function and Analytic Formula

    equation

    For noise-free evaluations of an objective function ff, let fn∗=max⁡m≤nf(xm)f_n^* = \max_{m \le n} f(x_m) denote the maximum function value observed across the first nn sample points. The Expected Improvement (EI) acquisition function measures the expected increase over the incumbent maximum achieved by sampling at a new location xx:

    EIn(x):=En[[f(x)−fn∗]+]\text{EI}_n(x) := \mathbb{E}_n\left[[f(x) - f_n^*]^+\right]

    where [a]+=max⁡(a,0)[a]^+ = \max(a, 0) and En[⋅]=E[⋅∣x1:n,f(x1:n)]\mathbb{E}_n[\cdot] = \mathbb{E}[\cdot \mid x_{1:n}, f(x_{1:n})]. Under the Gaussian process posterior distribution f(x)∣x1:n,f(x1:n)∼N(μn(x),σn2(x))f(x) \mid x_{1:n}, f(x_{1:n}) \sim \mathcal{N}(\mu_n(x), \sigma_n^2(x)), this expectation integrates analytically in closed form to:

    ight) - |\Delta_n(x)| \Phi\left(\frac{\Delta_n(x)}{\sigma_n(x)} ight)$$ where $\Delta_n(x) := \mu_n(x) - f_n^*$ represents the expected difference in quality between $x$ and the previous best observation, $\phi(\cdot)$ is the standard normal probability density function, and $\Phi(\cdot)$ is the standard normal cumulative distribution function.

  4. Knowl 4 — Knowledge Gradient Acquisition Function

    model/method

    The Knowledge Gradient (KG) acquisition function evaluates prospective candidate sample points under the premise that the decision-maker is risk-neutral and can report any point in the domain AA as the final recommendation, choosing the point that maximizes the posterior mean. Let μn∗:=max⁡x′∈Aμn(x′)\mu_n^* := \max_{x' \in A} \mu_n(x') be the maximum of the posterior mean at iteration nn.

    If an additional evaluation is conducted at xn+1=xx_{n+1} = x resulting in observation yn+1y_{n+1}, the posterior mean updates to μn+1(⋅)\mu_{n+1}(\cdot), and the maximum expected reward becomes μn+1∗:=max⁡x′∈Aμn+1(x′)\mu_{n+1}^* := \max_{x' \in A} \mu_{n+1}(x'). The Knowledge Gradient is the expected improvement in the maximum posterior mean:

    KGn(x):=En[μn+1∗−μn∗∣xn+1=x]\text{KG}_n(x) := \mathbb{E}_n\left[\mu_{n+1}^* - \mu_n^* \mid x_{n+1} = x\right]

    where En[⋅]\mathbb{E}_n[\cdot] is evaluated with respect to the predictive distribution of yn+1∼N(μn(x),σn2(x))y_{n+1} \sim \mathcal{N}(\mu_n(x), \sigma_n^2(x)) (or including measurement noise variance). Unlike Expected Improvement, KG accounts for global changes across the entire posterior mean function induced by sampling at xx, assigning value to evaluations that reveal information even if the observed point itself is not an incumbent optimum.

  5. Knowl 5 — Knowledge Gradient Optimization via Infinitesimal Perturbation Analysis

    algorithm

    Because the Knowledge Gradient acquisition function KGn(x)\text{KG}_n(x) cannot generally be written in closed form, its gradient ∇KGn(x)\nabla \text{KG}_n(x) is computed via Infinitesimal Perturbation Analysis (IPA) and the envelope theorem. Differentiating through the expectation yields ∇KGn(x)=En[∇μn+1∗∣xn+1=x]\nabla \text{KG}_n(x) = \mathbb{E}_n[\nabla \mu_{n+1}^* \mid x_{n+1} = x]. By expressing the future observation as yn+1=μn(x)+σn(x)Zy_{n+1} = \mu_n(x) + \sigma_n(x)Z where Z∼N(0,1)Z \sim \mathcal{N}(0, 1), and finding the maximizing point x^∗∈arg⁡max⁡x′μn+1(x′;x,μn(x)+σn(x)Z)\hat{x}^* \in \arg\max_{x'} \mu_{n+1}(x'; x, \mu_n(x) + \sigma_n(x)Z), the gradient is obtained by differentiating μn+1(x^∗;x,μn(x)+σn(x)Z)\mu_{n+1}(\hat{x}^*; x, \mu_n(x) + \sigma_n(x)Z) with respect to xx while holding x^∗\hat{x}^* fixed. Unbiased gradients are then optimized using multi-start stochastic gradient ascent.

    Input: Number of restarts RR, iterations per restart TT, stepsize scale parameter aa, sample size JJ, feasible domain AA
    Output: Point x∗x^* approximately maximizing KGn(x)\text{KG}_n(x)
    for r=1r = 1 to RR do
        Initialize x0(r)x_0^{(r)} uniformly at random from AA
        for t=1t = 1 to TT do
            for j=1j = 1 to JJ do
                Sample Zj∼N(0,1)Z_j \sim \mathcal{N}(0, 1)
                Set yn+1(j)=μn(xt−1(r))+σn(xt−1(r))Zjy_{n+1}^{(j)} = \mu_n(x_{t-1}^{(r)}) + \sigma_n(x_{t-1}^{(r)}) Z_j
                Compute posterior mean function μn+1(⋅;xt−1(r),yn+1(j))\mu_{n+1}(\cdot; x_{t-1}^{(r)}, y_{n+1}^{(j)})
                Find x^j∗∈arg⁡max⁡x′∈Aμn+1(x′;xt−1(r),yn+1(j))\hat{x}_j^* \in \arg\max_{x' \in A} \mu_{n+1}(x'; x_{t-1}^{(r)}, y_{n+1}^{(j)}) using L-BFGS
                Compute gradient G(j)=∇xμn+1(x^j∗;x,μn(x)+σn(x)Zj)∣x=xt−1(r)G^{(j)} = \nabla_x \mu_{n+1}(\hat{x}_j^*; x, \mu_n(x) + \sigma_n(x)Z_j) \mid_{x = x_{t-1}^{(r)}}
            end for
            Set stochastic gradient G=1J∑j=1JG(j)G = \frac{1}{J} \sum_{j=1}^J G^{(j)}
            Set step size αt=aa+t\alpha_t = \frac{a}{a + t}
            Update iterate xt(r)=xt−1(r)+αtGx_t^{(r)} = x_{t-1}^{(r)} + \alpha_t G
        end for
        Estimate KGn(xT(r))\text{KG}_n(x_T^{(r)}) by Monte Carlo simulation over JJ replications
    end for
    Return x∗=arg⁡max⁡r=1,…,RKGn(xT(r))x^* = \arg\max_{r=1,\dots,R} \text{KG}_n(x_T^{(r)})

  6. Knowl 6 — Entropy Search and Predictive Entropy Search Acquisition Functions

    model/method

    Entropy Search (ES) and Predictive Entropy Search (PES) are information-theoretic acquisition functions that select points to maximize the reduction in differential entropy regarding the location of the global maximizer x∗=arg⁡max⁡x∈Af(x)x^* = \arg\max_{x \in A} f(x). For a continuous probability density p(x)p(x), differential entropy is defined as H(p)=−∫p(x)log⁡p(x) dxH(p) = -\int p(x) \log p(x)\,dx.

    Entropy Search directly defines the information gain at point xx as the expected reduction in the differential entropy of the posterior distribution Pn(x∗)P_n(x^*):

    ESn(x)=H(Pn(x∗))−Ef(x)[H(Pn(x∗∣f(x)))]\text{ES}_n(x) = H(P_n(x^*)) - \mathbb{E}_{f(x)}\left[H(P_n(x^* \mid f(x)))\right]

    Predictive Entropy Search reformulates this quantity via the mutual information identity I(x∗;f(x))=H(Pn(x∗))−H(Pn(x∗∣f(x)))=H(Pn(f(x)))−H(Pn(f(x)∣x∗))I(x^*; f(x)) = H(P_n(x^*)) - H(P_n(x^* \mid f(x))) = H(P_n(f(x))) - H(P_n(f(x) \mid x^*)), yielding the equivalent formulation:

    PESn(x)=H(Pn(f(x)))−Ex∗[H(Pn(f(x)∣x∗))]\text{PES}_n(x) = H(P_n(f(x))) - \mathbb{E}_{x^*}\left[H(P_n(f(x) \mid x^*))\right]

    In PES, the first term H(Pn(f(x)))H(P_n(f(x))) represents the entropy of the one-dimensional Gaussian posterior predictive distribution N(μn(x),σn2(x))\mathcal{N}(\mu_n(x), \sigma_n^2(x)) and is computed in closed form as 12ln⁡(2πeσn2(x))\frac{1}{2}\ln(2\pi e \sigma_n^2(x)). The second term is approximated by drawing samples of x∗x^* from the GP posterior and applying expectation propagation.

  7. Knowl 7 — Decision-Theoretic Generalization of Expected Improvement to Noisy Evaluations

    model/method

    When function evaluations are contaminated with stochastic noise yn=f(xn)+ϵny_n = f(x_n) + \epsilon_n where ϵn∼N(0,σϵ2)\epsilon_n \sim \mathcal{N}(0, \sigma_\epsilon^2), the exact function values f(x)f(x) are unobserved, rendering the conventional Expected Improvement definition ill-posed.

    Under the restriction inherent to Expected Improvement—that the reported final recommendation must be one of the evaluated points {x1,…,xn}\{x_1, \dots, x_n\}—a risk-neutral decision-maker stops after nn measurements by returning arg⁡max⁡i=1,…,nμn(xi)\arg\max_{i=1,\dots,n} \mu_n(x_i), achieving expected value μn∗∗:=max⁡i=1,…,nμn(xi)\mu_n^{**} := \max_{i=1,\dots,n} \mu_n(x_i).

    If one additional noisy sample is gathered at xn+1=xx_{n+1} = x, the expected value of the best evaluated point under the updated posterior distribution becomes μn+1∗∗:=max⁡i=1,…,n+1μn+1(xi)\mu_{n+1}^{**} := \max_{i=1,\dots,n+1} \mu_{n+1}(x_i). The decision-theoretically grounded generalization of Expected Improvement for noisy evaluations is the expected increment:

    EInnoisy(x):=En[μn+1∗∗−μn∗∗∣xn+1=x]\text{EI}_n^{\text{noisy}}(x) := \mathbb{E}_n\left[\mu_{n+1}^{**} - \mu_n^{**} \mid x_{n+1} = x\right]

    Because an evaluation at xn+1x_{n+1} shifts the posterior means μn+1(xi)\mu_{n+1}(x_i) at previously evaluated locations xix_i (i≤ni \le n) via correlation, this formulation explicitly accounts for the revision in estimated values of previously measured points.

  8. Knowl 8 — Parallel Expected Improvement Acquisition Function

    equation

    To select a batch of qq candidate points x(1:q)=(x(1),…,x(q))x^{(1:q)} = (x^{(1)}, \dots, x^{(q)}) for simultaneous parallel evaluation, the Parallel Expected Improvement (multipoints EI) acquisition function computes the expected improvement of the maximum value within the proposed batch over the current incumbent best fn∗=max⁡m≤nf(xm)f_n^* = \max_{m \le n} f(x_m):

    EIn(x(1:q)):=En[[max⁡i=1,…,qf(x(i))−fn∗]+]\text{EI}_n(x^{(1:q)}) := \mathbb{E}_n\left[\left[\max_{i=1,\dots,q} f(x^{(i)}) - f_n^*\right]^+\right]

    where En[⋅]\mathbb{E}_n[\cdot] is taken over the joint Gaussian posterior distribution of (f(x(1)),…,f(x(q)))(f(x^{(1)}), \dots, f(x^{(q)})). The batch of points is chosen by solving arg⁡max⁡x(1:q)∈AqEIn(x(1:q))\arg\max_{x^{(1:q)} \in A^q} \text{EI}_n(x^{(1:q)}). Optimization over this q⋅dq \cdot d-dimensional space is performed using heuristic sequential approximations (such as the Constant Liar heuristic) or multistart stochastic gradient ascent using Infinitesimal Perturbation Analysis to sample unbiased stochastic gradients ∇EIn(x(1:q))\nabla \text{EI}_n(x^{(1:q)}).

  9. Knowl 9 — Multi-Fidelity and Multi-Information Source Bayesian Optimization

    model/method

    In multi-fidelity and multi-information source optimization, the objective function is evaluated through auxiliary information sources f(x,s)f(x, s) indexed by s∈Ss \in S, where x∈Ax \in A is the design parameter, s=0s=0 corresponds to the true target objective f(x,0)f(x, 0), and each source ss incurs a query cost c(s)c(s) with varying bias and noise.

    A single Gaussian process is placed over the joint domain A×SA \times S to model f(x,s)f(x, s). The objective is to maximize f(x,0)f(x, 0) under a total computational cost constraint:

    max⁡x∈Af(x,0)subject to∑n=1Nc(sn)≤B\max_{x \in A} f(x, 0) \quad \text{subject to} \quad \sum_{n=1}^N c(s_n) \le B

    Because standard Expected Improvement evaluates strictly against observed values of f(x,0)f(x, 0) and assigns EI=0\text{EI} = 0 for any low-fidelity source s≠0s \ne 0, acquisition functions that evaluate global posterior change—such as Knowledge Gradient (KG) and Predictive Entropy Search (PES)—are used to select joint pairs (xn,sn)(x_n, s_n) that deliver the highest information gain per unit cost.

  10. Knowl 10 — Bayesian Optimization with Random Environmental Conditions and Integrands

    model/method

    Optimization under random environmental conditions, multi-task Bayesian optimization, and the optimization of continuous integrated responses target problems of the form:

    max⁡x∈A∫f(x,w)p(w) dwormax⁡x∈A∑wf(x,w)p(w)\max_{x \in A} \int f(x, w) p(w)\,dw \quad \text{or} \quad \max_{x \in A} \sum_w f(x, w) p(w)

    where f(x,w)f(x, w) is an expensive black-box function depending on controllable design variables x∈Ax \in A and environmental or task parameters ww, and p(w)p(w) is a known probability distribution or weight function over ww.

    Rather than evaluating the complete integral or full summation at high cost during every iteration, f(x,w)f(x, w) is modeled with a Gaussian process surrogate over the joint domain (x,w)(x, w). Acquisition functions such as the Knowledge Gradient (KG) quantify the expected improvement in the integrated objective max⁡x′∫μn+1(x′,w)p(w) dw\max_{x'} \int \mu_{n+1}(x', w) p(w)\,dw resulting from evaluating f(x,w)f(x, w) at a single chosen pair (xn,wn)(x_n, w_n), significantly reducing the overall evaluation burden.

Coverage note — Omitted the high-level survey of software packages (Section 6) and historical literature review as they represent contextual references rather than standalone methodological contributions.

References

  1. 1.Ahmed, M. O., Shahriari, B., and Schmidt, M. (2016). Do we need ‐harmless‐ Bayesian optimization and ‐first-order‐ Bayesian optimization. In Neural Information Processing Systems 2016 Workshop on Bayesian Optimization.
  2. 2.Berger, J. O. (2013). Statistical Decision Theory and Bayesian Analysis. Springer Science & Business Media.
  3. 3.Blum, J. R. (1954). Multidimensional stochastic approximation methods. The Annals of Mathematical Statistics, pages 737–744.
  4. 4.Booker, A., Dennis, J., Frank, P., Serafini, D., Torczon, V., and Trosset, M. (1999). A rigorous framework for optimization of expensive functions by surrogates. Structural and Multidisciplinary Optimization, 17(1):1–13.
  5. 5.Bottou, L. (2012). Stochastic gradient descent tricks. In Montavon, G., Orr, G. B., and Müller, K. R., editors, Neural Networks: Tricks of the Trade, pages 421–436. Springer.
  6. 6.Brochu, E., Cora, M., and de Freitas, N. (2009). A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. Technical Report TR-2009-023, Department of Computer Science, University of British Columbia. arXiv:1012.2599.
  7. 7.Bull, A. D. (2011). Convergence rates of efficient global optimization algorithms. Journal of Machine Learning Research, 12(Oct):2879–2904.
  8. 8.Calvin, J. (1997). Average performance of a class of adaptive algorithms for global optimization. The Annals of Applied Probability, 7(3):711–730.
  9. 9.Calvin, J. and Žilinskas, A. (2005). One-dimensional global optimization for observations with noise. Computers & Mathematics with Applications, 50(1-2):157–169.
  10. 10.Calvin, J. and Žilinskas, A. (1999). On the convergence of the P-algorithm for one-dimensional global optimization of smooth functions. Journal of Optimization Theory and Applications, 102(3):479–495.
  11. 11.Calvin, J. and Žilinskas, A. (2000). One-dimensional P-algorithm with convergence rate O(n‐3+δ) for smooth functions. Journal of Optimization Theory and Applications, 106(2):297–307.
  12. 12.Cashore, J. M., Kumarga, L., and Frazier, P. I. (2016). Multi-step Bayesian optimization for one-dimensional feasibility determination. arXiv preprint arXiv:1607.03195.
  13. 13.Chang, P. B., Williams, B. J., Bhalla, K. S. B., Belknap, T. W., Santner, T. J., Notz, W. I., and Bartel, D. L. (2001). Design and analysis of robust total joint replacements: finite element model experiments with environmental variables. Journal of Biomechanical Engineering, 123(3):239–246.
  14. 14.Chick, S. E. and Inoue, K. (2001). New two-stage and sequential procedures for selecting the best simulated system. Operations Research, 49(5):732–743.
  15. 15.Clark, C. E. (1961). The greatest of a finite set of random variables. Operations Research, 9(2):145–162.
  16. 16.Cover, T. M. and Thomas, J. A. (2012). Elements of Information Theory. John Wiley & Sons.
  17. 17.Dynkin, E. and Yushkevich, A. (1979). Controlled Markov Processes. Springer, New York.
  18. 18.Forrester, A., Sóbester, A., and Keane, A. (2008). Engineering Design via Surrogate Modelling: A Practical Guide. Wiley, West Sussex, UK.
  19. 19.Forrester, A. I., Sóbester, A., and Keane, A. J. (2007). Multi-fidelity optimization via surrogate modelling. In Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, volume 463, pages 3251–3269. The Royal Society.
  20. 20.Frazier, P., Powell, W., and Dayanik, S. (2009). The knowledge-gradient policy for correlated normal beliefs. INFORMS Journal on Computing, 21(4):599–613.
  21. 21.Frazier, P. I. (2012). Tutorial: Optimization via simulation with bayesian statistics and dynamic programming. In Laroque, C., Himmelspach, J., Pasupathy, R., Rose, O., and Uhrmacher, A. M., editors, Proceedings of the 2012 Winter Simulation Conference Proceedings, pages 79–94, Piscataway, New Jersey. Institute of Electrical and Electronics Engineers, Inc.
  22. 22.Frazier, P. I., Powell, W. B., and Dayanik, S. (2008). A knowledge-gradient policy for sequential information collection. SIAM Journal on Control and Optimization, 47(5):2410–2439.
  23. 23.Frazier, P. I. and Wang, J. (2016). Bayesian optimization for materials design. In Lookman, T., Alexander, F. J., and Rajan, K., editors, Information Science for Materials Discovery and Design, pages 45–75. Springer.
  24. 24.Gardner, J. R., Kusner, M. J., Xu, Z. E., Weinberger, K. Q., and Cunningham, J. P. (2014). Bayesian optimization with inequality constraints. In ICML, pages 937–945.
  25. 25.Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., and Rubin, D. B. (2014). Bayesian Data Analysis, volume 2. CRC Press Boca Raton, FL.
  26. 26.Ginsbourger, D., Le Riche, R., and Carraro, L. (2007). A multi-points criterion for deterministic parallel global optimization based on kriging. In International Conference on Nonconvex Programming, NCP07, Rouen, France.
  27. 27.Ginsbourger, D., Le Riche, R., and Carraro, L. (2010). Kriging is well-suited to parallelize optimization. In Tenne, Y. and Goh, C. K., editors, Computational Intelligence in Expensive Optimization Problems, volume 2, pages 131–162. Springer.
  28. 28.Ginsbourger, D. and Riche, R. (2010). Towards Gaussian process-based optimization with finite time horizon. In Giovagnoli, A., Atkinson, A., Torsney, B., and May, C., editors, mODa 9–Advances in Model-Oriented Design and Analysis, pages 89–96. Springer.
  29. 29.González, J., Osborne, M., and Lawrence, N. (2016). GLASSES: Relieving the myopia of bayesian optimisation. In Artificial Intelligence and Statistics, pages 790–799.
  30. 30.Groot, P., Birlutiu, A., and Heskes, T. (2010). Bayesian monte carlo for the global optimization of expensive functions. In ECAI, pages 249–254.
  31. 31.Hennig, P. and Schuler, C. J. (2012). Entropy search for information-efficient global optimization. Journal of Machine Learning Research, 13:1809–1837.
  32. 32.Hernández-Lobato, J. M., Gelbart, M. A., Hoffman, M. W., Adams, R. P., and Ghahramani, Z. (2015). Predictive entropy search for bayesian optimization with unknown constraints. In Proceedings of the 32nd International Conference on International Conference on Machine Learning-Volume 37, pages 1699–1707. JMLR. org.
  33. 33.Hernández-Lobato, J. M., Hoffman, M. W., and Ghahramani, Z. (2014). Predictive entropy search for efficient global optimization of black-box functions. In Advances in neural information processing systems, pages 918–926.
  34. 34.Ho, Y.-C., Cao, X., and Cassandras, C. (1983). Infinitesimal and finite perturbation analysis for queueing networks. Automatica, 19(4):439–445.
  35. 35.Huang, D., Allen, T., Notz, W., and Miller, R. (2006). Sequential kriging optimization using multiple-fidelity evaluations. Structural and Multidisciplinary Optimization, 32(5):369–382.
  36. 36.Jedynak, B., Frazier, P. I., and Sznitman, R. (2012). Twenty questions with noise: Bayes optimal policies for entropy loss. Journal of Applied Probability, 49(1):114–136.
  37. 37.Jones, D. R., Schonlau, M., and Welch, W. J. (1998). Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13(4):455–492.
  38. 38.Ju, S., Shiga, T., Feng, L., Hou, Z., Tsuda, K., and Shiomi, J. (2017). Designing nanostructures for phonon transport via Bayesian optimization. Physical Review X, 7.
  39. 39.Kaelbling, L. P., Littman, M. L., and Moore, A. W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237–285.
  40. 40.Kandasamy, K., Dasarathy, G., Oliva, J. B., Schneider, J., and Póczos, B. (2016). Gaussian process bandit optimisation with multi-fidelity evaluations. In Advances in Neural Information Processing Systems, pages 992–1000.
  41. 41.Kandasamy, K., Schneider, J., and Póczos, B. (2015). High dimensional bayesian optimisation and bandits via additive models. In International Conference on Machine Learning, pages 295–304.
  42. 42.Keane, A. (2006). Statistical improvement criteria for use in multiobjective design optimization. AIAA Journal, 44(4):879–891.
  43. 43.Kersting, K., Plagemann, C., Pfaff, P., and Burgard, W. (2007). Most likely heteroscedastic gaussian process regression. In Proceedings of the 24th International Conference on Machine learning, pages 393–400. ACM.
  44. 44.Kleijnen, J. P. et al. (2008). Design and Analysis of Simulation Experiments, volume 20. Springer.
  45. 45.Klein, A., Falkner, S., Bartels, S., Hennig, P., and Hutter, F. (2016). Fast Bayesian optimization of machine learning hyperparameters on large datasets. arXiv preprint arXiv:1605.07079.
  46. 46.Knowles, J. (2006). ParEGO: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Transactions on Evolutionary Computation, 10(1):50–66.
  47. 47.Kushner, H. J. (1964). A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. Journal of Basic Engineering, 86(1):97–106.
  48. 48.Lam, R., Allaire, D. L., and Willcox, K. E. (2015). Multifidelity optimization using statistical surrogate modeling for non-hierarchical information sources. In 56th AIAA/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, page 0143.
  49. 49.Lam, R., Willcox, K., and Wolpert, D. H. (2016). Bayesian optimization with a finite budget: An approximate dynamic programming approach. In Advances in Neural Information Processing Systems, pages 883–891.
  50. 50.Liu, D. C. and Nocedal, J. (1989). On the limited memory BFGS method for large scale optimization. Mathematical Programming, 45(1-3):503–528.
  51. 51.Lizotte, D. (2008). Practical Bayesian Optimization. PhD thesis, University of Alberta.
  52. 52.Lizotte, D., Wang, T., Bowling, M., and Schuurmans, D. (2007). Automatic gait optimization with Gaussian process regression. In Proceedings of IJCAI, pages 944–949.
  53. 53.Mahajan, A. and Teneketzis, D. (2008). Multi-armed bandit problems. In Hero, A., Castañón, D., Cochran, D., and Kastella, K., editors, Foundations and Applications of Sensor Management, pages 121–151. Springer.
  54. 54.Martí, R., Lozano, J. A., Mendiburu, A., and Hernando, L. (2016). Multi-start methods. Handbook of Heuristics, pages 1–21.
  55. 55.McLeod, M., Osborne, M. A., and Roberts, S. J. (2017). Practical bayesian optimization for variable cost objectives. arXiv preprint arXiv:1703.04335.
  56. 56.Mehdad, E. and Kleijnen, J. P. (2018). Efficient global optimisation for black-box simulation via sequential intrinsic kriging. Journal of the Operational Research Society, 69:1–13.
  57. 57.Milgrom, P. and Segal, I. (2002). Envelope theorems for arbitrary choice sets. Econometrica, 70(2):583–601.
  58. 58.Minka, T. P. (2001). A family of algorithms for approximate Bayesian inference. PhD thesis, Massachusetts Institute of Technology.
  59. 59.Močkus, J. (1975). On Bayesian methods for seeking the extremum. In Optimization Techniques IFIP Technical Conference, pages 400–404. Springer.
  60. 60.Močkus, J. (1989). Bayesian Approach to Global Optimization: Theory and Applications. Kluwer Academic Publishers.
  61. 61.Močkus, J. and Močkus, L. (1991). Bayesian approach to global optimization and application to multi-objective and constrained problems. Journal of Optimization Theory and Applications, 70(1):157–172.
  62. 62.Močkus, J., Tiesis, V., and Žilinskas, A. (1978). The application of Bayesian methods for seeking the extremum. In Dixon, L. and Szego, G., editors, Towards Global Optimisation, volume 2, pages 117–129. Elsevier Science Ltd., North Holland, Amsterdam.
  63. 63.Neal, R. M. (2003). Slice sampling. Annals of Statistics, 31(3):705–741.
  64. 64.Negoescu, D. M., Frazier, P. I., and Powell, W. B. (2011). The knowledge gradient algorithm for sequencing experiments in drug discovery. INFORMS Journal on Computing, 23(1):46–363.
  65. 65.Osborne, M. A., Garnett, R., and Roberts, S. J. (2009). Gaussian processes for global optimization. In 3rd International Conference on Learning and Intelligent Optimization (LION3), pages 1–15. Citeseer.
  66. 66.Packwood, D. (2017). Bayesian Optimization for Materials Science, volume 3. Springer.
  67. 67.Perez, S. (2015). Twitter acquires machine learning startup whetlab. TechCrunch. Accessed July 3, 2018.
  68. 68.Poloczek, M., Wang, J., and Frazier, P. (2017). Multi-information source optimization. In Advances in Neural Information Processing Systems, pages 4291–4301.
  69. 69.Powell, W. B. (2007). Approximate Dynamic Programming: Solving the Curses of Dimensionality. John Wiley & Sons, New York.
  70. 70.Rasmussen, C. and Williams, C. (2006). Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA.
  71. 71.Regis, R. and Shoemaker, C. (2005). Constrained global optimization of expensive black box functions using radial basis functions. Journal of Global Optimization, 31(1):153–171.
  72. 72.Regis, R. and Shoemaker, C. (2007a). Improved strategies for radial basis function methods for global optimization. Journal of Global Optimization, 37(1):113–135.
  73. 73.Regis, R. and Shoemaker, C. (2007b). Parallel radial basis function methods for the global optimization of expensive functions. European Journal of Operational Research, 182(2):514–535.
  74. 74.Robbins, H. and Monro, S. (1951). A stochastic approximation method. The Annals of Mathematical Statistics, 22(3):400–407.
  75. 75.Roustant, O., Ginsbourger, D., and Deville, Y. (2012). Dicekriging, diceoptim: Two r packages for the analysis of computer experiments by kriging-based metamodeling and optimization. Journal of Statistical Software, Articles, 51(1):1–55.
  76. 76.Salemi, P., Nelson, B. L., and Staum, J. (2014). Discrete optimization via simulation using Gaussian Markov random fields. In Proceedings of the 2014 Winter Simulation Conference, pages 3809–3820. IEEE Press.
  77. 77.Sasena, M. (2002). Flexibility and Efficiency Enhancements for Constrained Global Design Optimization with Kriging Approximations. PhD thesis, University of Michigan.
  78. 78.Schonlau, M., Welch, W. J., and Jones, D. R. (1998). Global versus local search in constrained optimization of computer models. Lecture Notes — Monograph Series, 34:11–25.
  79. 79.Scott, W., Frazier, P. I., and Powell, W. B. (2011). The correlated knowledge gradient for simulation optimization of continuous parameters using Gaussian process regression. SIAM Journal on Optimization, 21(3):996–1026.
  80. 80.Seko, A., Togo, A., Hayashi, H., Tsuda, K., Chaput, L., and Tanaka, I. (2015). Prediction of low-thermal-conductivity compounds with first-principles anharmonic lattice-dynamics calculations and Bayesian optimization. Physical Review Letters, 115.
  81. 81.Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., and de Freitas, N. (2016). Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE, 104(1):148–175.
  82. 82.Shan, S. and Wang, G. G. (2010). Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions. Structural and Multidisciplinary Optimization, 41(2):219–241.
  83. 83.Shoemaker, C., Regis, R., and Fleming, R. (2007). Watershed calibration using multistart local optimization and evolutionary optimization with radial basis function approximation/calage au niveau du bassin versant à l’aide d’une optimisation locale à démarrage multiple et d’une optimisation évolutive avec approximation à fonctions de base radiale. Hydrological Sciences Journal/Journal des Sciences Hydrologiques, 52(3):450–465.
  84. 84.Snoek, J., Larochelle, H., and Adams, R. P. (2012). Practical Bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems, pages 2951–2959.
  85. 85.Snoek, J., Swersky, K., Zemel, R., and Adams, R. (2014). Input warping for Bayesian optimization of non-stationary functions. In International Conference on Machine Learning, pages 1674–1682.
  86. 86.Sóbester, A., Leary, S., and Keane, A. (2004). A parallel updating scheme for approximating and optimizing high fidelity computer simulations. Structural and Multidisciplinary Optimization, 27(5):371–383.
  87. 87.Sutton, R. S. and Barto, A. G. (1998). Reinforcement Learning: An Introduction. MIT press Cambridge.
  88. 88.Swersky, K., Snoek, J., and Adams, R. P. (2013). Multi-task Bayesian optimization. In Advances in Neural Information Processing Systems, pages 2004–2012.
  89. 89.Swersky, K., Snoek, J., and Adams, R. P. (2014). Freeze-thaw bayesian optimization. arXiv preprint arXiv:1406.3896.
  90. 90.Toscano-Palmerin, S. and Frazier, P. I. (2018). Bayesian optimization with expensive integrands. arXiv preprint arXiv:1803.08661.
  91. 91.Ueno, T., Rhone, T. D., Hou, Z., Mizoguchi, T., and Tsuda, K. (2016). COMBO: An efficient Bayesian optimization library for materials science. Materials Discovery, 4:18–21.
  92. 92.Žilinskas, A. (1975). Single-step Bayesian search method for an extremum of functions of a single variable. Cybernetics and Systems Analysis, 11(1):160–166.
  93. 93.Waeber, R., Frazier, P. I., and Henderson, S. G. (2013). Bisection search with noisy responses. SIAM Journal on Control and Optimization, 51(3):2261–2279.
  94. 94.Wang, J., Clark, S. C., Liu, E., and Frazier, P. I. (2016a). Parallel Bayesian global optimization of expensive functions. arXiv preprint arXiv:1602.05149.
  95. 95.Wang, Z., Hutter, F., Zoghi, M., Matheson, D., and de Feitas, N. (2016b). Bayesian optimization in a billion dimensions via random embeddings. Journal of Artificial Intelligence Research, 55:361–387.
  96. 96.Wang, Z., Zoghi, M., Hutter, F., Matheson, D., De Freitas, N., et al. (2013). Bayesian optimization in high dimensions via random embeddings. In IJCAI, pages 1778–1784.
  97. 97.Williams, B. J., Santner, T. J., and Notz, W. I. (2000). Sequential design of computer experiments to minimize integrated response functions. Statistica Sinica, 10(4):1133–1152.
  98. 98.Wu, J. and Frazier, P. (2016). The parallel knowledge gradient method for batch Bayesian optimization. In Advances in Neural Information Processing Systems, pages 3126–3134.
  99. 99.Wu, J., Poloczek, M., Wilson, A. G., and Frazier, P. (2017). Bayesian optimization with gradients. In Advances in Neural Information Processing Systems, pages 5273–5284.
  100. 100.Xie, J. and Frazier, P. I. (2013). Sequential Bayes-optimal policies for multiple comparisons with a known standard. Operations Research, 61(5):1174–1189.
  101. 101.Xie, J., Frazier, P. I., Sankaran, S., Marsden, A., and Elmohamed, S. (2012). Optimization of computationally expensive simulations with Gaussian processes and parameter uncertainty: Application to cardiovascular surgery. In 50th Annual Allerton Conference on Communication, Control, and Computing, pages 406–413. IEEE.

Citation

MLA
Frazier, P. I. “A Tutorial on Bayesian Optimization”. arXiv, 2018, https://doi.org/10.48550/arxiv.1807.02811.
APA
Frazier, P. I. (2018). A Tutorial on Bayesian Optimization. arXiv. https://doi.org/10.48550/arxiv.1807.02811
Chicago
Frazier, P. I. 2018. “A Tutorial on Bayesian Optimization”. Preprint, ArXiv. https://doi.org/10.48550/arxiv.1807.02811.
Harvard
Frazier, P.I. (2018) “A Tutorial on Bayesian Optimization”. arXiv. Available at: https://doi.org/10.48550/arxiv.1807.02811.
Vancouver
1. Frazier PI (2018) A Tutorial on Bayesian Optimization. https://doi.org/10.48550/arxiv.1807.02811

BibTeX

@misc{https://doi.org/10.48550/arxiv.1807.02811,
  doi = {10.48550/ARXIV.1807.02811},
  url = {https://arxiv.org/abs/1807.02811},
  author = {Frazier, Peter I.},
  keywords = {Machine Learning (stat.ML), Machine Learning (cs.LG), Optimization and Control (math.OC), FOS: Computer and information sciences, FOS: Computer and information sciences, FOS: Mathematics, FOS: Mathematics},
  title = {A Tutorial on Bayesian Optimization},
  publisher = {arXiv},
  year = {2018},
  copyright = {arXiv.org perpetual, non-exclusive license}
}
Metadata:DOI registry

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors