Approximate Dynamic Programming
The dynamic programming paradigm provides sufficient conditions for optimality of control laws for dynamical systems. The optimal cost-to-go function, often referred to as the value function, provides the necessary information to derive the control law in each state. The computation of the value function is straightforward for discrete valued systems with a countable number of states. For systems with an infinite number of states, the value function is approximated by parameterized function approximation architecture. For illustration, figure on the right shows the value function for a linear quadratic problem.
Hybrid Automata
The dynamic programming principle is extended to hybrid automata. A sample run for this class of systems is illustrated in the figure below. The guard sets, triggering a discrete transition might by enforcing or enabling. A hybrid version of the value function is calculated on the hybrid state space, and an optimal control law can be derived from the value function. The solution of the hybrid optimal control problem, leads to the continuous control input of the subsystems, times and targets of the discrete transitions.
- Sample run of a hybrid automaton
Application
The introduced approach is demonstrated by an engine transmission system, representing the hybrid system. It consists of four gears determining the dynamic behavior and such the effect of the throttle position on the acceleration. Solving the hybrid optimal control problem means to find the optimal timing and switching sequence between the gears, as well as the optimal throttle position. The cost function is posed such that an optimal acceleration results. The value function and resulting hybrid trajectory is illustrated in the right plot.
Researcher
Matthias Rungger