Sample Efficient Estimation and Optimization of Expensive to Evaluate Black-Box Functions

In many complex engineering problems, efficient estimation and optimization of black-box functions is a major concern. It requires an extensive number of evaluations which is often not available due to the time, budget, or technical constraints. For efficient estimation, we propose an active learning method based on the integration of the Laplacian regularization and gradient directed kernel for identification of the most informative points for learning expensive noisy black-box functions with a minimum number of points. Using a case study for analysis of the kinematics of pitching in baseball as well as simulation experiments, we demonstrate the performance of the proposed algorithm against existing methods in the literature. For global optimization, we propose a multi-armed bandit inspired regularized expected improvement (BREI) to adaptively adjust and improve the balance between the exploration and exploitation for efficient optimization of expensive experiments. Using a case study in the optimization of collision avoidance algorithm in mobile robot motion planning as well as extensive simulations, we validate the proposed algorithm against existing methods in the literature under different levels of noise.