Course Content
Review of probability theory; Stochastic approximation algorithms: stability and convergence, asynchronous implementations, two time scale schemes, examples from electrical engineering; Markov chain Monte Carlo: variance reduction, simulated annealing; Markov decision processes: stochastic dynamic programming, computational schemes, state and parameter estimation, control under partial observations, adaptive control, learning algorithms
Text / References
- 1 J. Spall, Introduction to Stochastic Search and Optimization, Wiley-Interscience, New York, 2003.
- 2 Sheldon Ross, Introduction to Stochastic Dynamic Programming, Academic Press, New York, 1995.3) S. Asmussen and P. W. Glynn, Stochastic Simulation, Springer Verlag, New York, 2007.