The 5th MSDM workshop is held in conjunction with AAMAS-2010 (the 9th International Joint Conference on Autonomous Agents and Multiagent Systems), in Toronto, Canada. It will take place on May 11, 2010, preceding the AAMAS conference.
In this talk I will introduce and discuss the complex dynamics that arise when multiple agents make decisions in complex dynamic situations, which require them to be adaptive or learn. For this purpose we will investigate Reinforcement Learning and Evolutionary Game Theory, and summarize some of the recent results obtained with these techniques at the RAI research group of Maastricht University. In the first part of the talk I will formally connect Reinforcement Learning to Evolutionary Game Theory using a variety of Replicator Dynamics models. The Replicator Dynamics are a central concept from Evolutionary Game Theory that describe how a population of strategies evolves through time using biological operators such as selection and mutation. We will both examine this relationship for stateless and multi-state games, and investigate the intricacies of the dynamics that arise from multiple concurrently learning agents. Using this relation we investigate what Evolutionary Game Theory tells us about multi-agent learning, and illustrate how e.g. parameters can be tuned and new learning algorithms can be designed. In the second part of the talk I will illustrate this work in classical game theoretic settings, continuous double auctions and No Limit Texas Hold'em Poker.
Feng Wu, Shlomo Zilberstein and Xiaoping Chen. Point-Based Policy Generation for Decentralized POMDPs
Camille Besse and Brahim Chaib-draa. Quasi-Deterministic POMDPs and DecPOMDPs
Hala Mostafa and Victor Lesser. Exploiting Structure To Efficiently Solve Loosely Coupled Stochastic Games
Maike Kaufman and Stephen Roberts. Coordination vs. Information in Multi-agent Decision Processes
João Messias, Matthijs Spaan and Pedro Lima. Multi-robot planning under uncertainty with communication: a case study
Prashant Doshi and Ekhlas Sonu. GaTAC: A Scalable and Realistic Testbed for Multiagent Decision Making
Robert McInerney, Stephen Roberts and Iead Rezek. Sequential Bayesian Decision Making for Multi-Armed Bandit
Ayman Ghoneim. Coordination-VCG Mechanism for Controlling Multiagent Planning
Arnaud Canu and Abdel-Illah Mouaddib. A New Approach for Solving Large Instances of DEC-POMDPs: Vector-Valued DEC-POMDPs
09:00 - 09:30 Camille Besse and Brahim Chaib-draa. Quasi-Deterministic POMDPs and DecPOMDPs
09:30 - 10:00 Stefan Witwicki and Edmund Durfee. Influence-based Policy Abstraction for Weakly-Coupled Dec-POMDPs
10:00 - 10:30 *** Coffee break ***
10:30 - 11:00 Feng Wu, Shlomo Zilberstein and Xiaoping Chen. Point-Based Policy Generation for Decentralized POMDPs
11:00 - 11:30 Prashant Doshi and Ekhlas Sonu. GaTAC: A Scalable and Realistic Testbed for Multiagent Decision Making
11:30 - 12:00 Hala Mostafa and Victor Lesser. Exploiting Structure To Efficiently Solve Loosely Coupled Stochastic Games
12:00 - 13:30 *** Lunch break ***
13:30 - 14:00 João Messias, Matthijs Spaan and Pedro Lima. Multi-robot planning under uncertainty with communication: a case study
14:00 - 14:30 Maike Kaufman and Stephen Roberts. Coordination vs. Information in Multi-agent Decision Processes
14:30 - 15:00 Robert McInerney, Stephen Roberts and Iead Rezek. Sequential Bayesian Decision Making for Multi-Armed Bandit
15:00 - 15:30 *** Coffee break ***
15:30 - 16:00 Arnaud Canu and Abdel-Illah Mouaddib. A New Approach for Solving Large Instances of DEC-POMDPs: Vector-Valued DEC-POMDPs
16:00 - 17:00 INVITED SPEAKER: Karl Tuyls. Complex Dynamics of Multi-Agent Sequential Decision Making
17:00 - 17:30 *** DISCUSSION ***
Sequential decision making under uncertainty is the problem an agent faces when it seeks to maximize its performance in an environment while making action choices based upon its observations of the world. Decision-theoretic approaches have been used very successfully in single-agent systems, so it is only natural to apply them to systems with many agents. The high computational complexity of finding optimal solutions in these multi-agent models has been a significant barrier to applying them to complex real world problems. Much of the work in this area relates to addressing this complexity through exploiting problem structure like locality of interaction, decomposition of reward and independence between the agents, and through approximate algorithms that converge to a local optimum instead of a global optimum.
The purpose of this workshop is to bring together researchers in the field of sequential decision-making in stochastic multi-agent systems to present and discuss promising new work, to discuss the relationships between the various models in use, and to establish important directions and goals for further research and collaboration. This workshop will strive to develop consensus within the community on benchmarks and evaluation methodology in order to contrast the alternative approaches and models, and also to study the associated trade-offs. Furthermore, we will discuss the creation of online problem sets for testing the various algorithms to facilitate comparison.
Possible topics include:
Authors are encouraged to submit papers up to 8 pages in length in the AAMAS2010 format. Submissions should be uploaded in PDF form at http://www.easychair.org/conferences/?conf=msdm2010. Each submission will be reviewed by at least two Program Committee members. The review process will be "single-blind"; thus authors do not have to remove their names when submitting papers.
Submissions should follow the same 8 page limit, but use the camera ready format found here and be uploaded on EasyChair by March 14th. Note that categories, keywords and general terms should be included.