Continous State and Action Spaces -- Relational and First-Order Knowledge Representation -- Hierarchical Approaches -- Predictive Approaches -- Multi-Agent Reinforcement Learning -- Partially Observable Markov Decision Processes (POMDPs) -- Decentralized POMDPs (DEC-POMDPs) -- Features and Function Approximation -- RL as Supervised Learning (or batch learning) -- Bounds and complexity -- RL for Games -- RL in Robotics -- Policy Gradient Techniques -- Least Squares Value Iteration -- Models and Model Induction -- Model-based RL -- Transfer Learning in RL -- Using of and extracting Knowledge in RL -- Biological or Psychological Background -- Evolutionary Approaches -- Closing chapter, prospects, future issues.

Reinforcement learning encompasses both a science of adaptive behavior of rational beings in uncertain environments and a computational methodology for finding optimal behaviors for challenging problems in control, optimization and adaptive behavior of intelligent agents. As a field, reinforcement learning has progressed tremendously in the past decade. The main goal of this book is to present an up-to-date series of survey articles on the main contemporary sub-fields of reinforcement learning. This includes surveys on partially observable environments, hierarchical task decompositions, relational knowledge representation and predictive state representations. Furthermore, topics such as transfer, evolutionary methods and continuous spaces in reinforcement learning are surveyed. In addition, several chapters review reinforcement learning methods in robotics, in games, and in computational neuroscience. In total seventeen different subfields are presented by mostly young experts in those areas, and together they truly represent a state-of-the-art of current reinforcement learning research. Marco Wiering works at the artificial intelligence department of the University of Groningen in the Netherlands. He has published extensively on various reinforcement learning topics. Martijn van Otterlo works in the cognitive artificial intelligence group at the Radboud University Nijmegen in The Netherlands. He has mainly focused on expressive knowledge representation in reinforcement learning settings.