Reinforcement Learning for Automated Negotiation: SCML as an Example
AJCAI 2023
Logistics
Date and Time: Nov 28th, 14:00 - 16:00
Location: Room: Sir Llew Edwards Building (14)-116
Duration: 1h 50min (+ 10min break)
Handouts: You can download the handouts of the whole tutorial here. See the bottom of this page for other related materials.
Why Attend?
Automated negotiation between intelligent agents is attracting more attention from the research community especially with the wider market penetration of intelligent agents and the need to coordinate their behavior.
The International Automated Negotiating Agents Competition (ANAC) provided stimulation for this research since its introduction in 2010. Since 2019, a new league was added to ANAC focusing on application of automated negotiation in a realistic business-like Supply Chain Management scenario (SCML). This tutorial will introduce the audience to SCML and walk them through the development of an agent for the competition highlighting the research challenges.
The main goal of the SCM league is to bring automated negotiation research more toward the real world by putting negotiation into a larger context from which endogenous utility functions are created instead of being predefined. This game provides amble opportunities for novel ideas in concurrent negotiation, utility function design, and applying machine learning techniques to automated negotiation.
Who should Attend?
The target audience are postgraduate students and researchers in the fields of multi-agent systems, game theory, simulation, and practical applications of MAS.
The tutorial will introduce the concepts it needs and is a beginner-level tutorial so the prerequisites are minimal. Knowledge of basic game-theoretic concepts are advantageous but not required.
Overview
Automated negotiation between intelligent agents is attracting more attention from the research community especially with the wider market penetration of intelligent agents and the need to coordinate their behavior.
The International Automated Negotiating Agents Competition (ANAC) provided stimulation for this research since its
introduction in 2010. Since 2019, a new league was added to ANAC focusing on application of automated negotiation in a realistic business-like Supply Chain Management scenario (SCML). In SCML 2024, a new RL based version of the problem is to be tackled.
This tutorial will introduce the audience to SCML, and its RL formulation and walk them through the development of an agent for the competition highlighting the research challenges.
Outline
- Theoretical Session (30min)
- This part of the tutorial introduces Automated Negotiation, the ANAC competition and the SCML describing the canonical structure of agent decomposition.
- The negotiation problem (10min) Different definitions of the negotiation problem, negotiation protocol, main differences between negotiations and auctions.
- ANAC (5min) A short history of the ANAC competition.
- SCM World (15min) Introduces the game design for SCML.
- Development Environment (30min)
- A hands-on installation and configuration tutorial
- Installing SCML (5min) Goes through the process of installation and configuration for the SCML package with the underlying NegMAS platform and the repository of SCML agents.
- Running a Simulation (10min) Goes through the process of running a single simulation and understanding log files.
- Running tournaments (5min) Introduces tournaments and their parameters as well as methods ensure that comparisons are fair when running tournaments.
- Breathing time (10min) Extra time for handling installation issues
break (10 min)
- RL for SCML (55min)
- This final part of the tutorial aims at giving the participants confidence that they could grasp the general structure of an SCML agent and that they can develop an agent for the competition and/or participate in related research in the future.
- The RL formulation of SCML (15min) Describes how can we model SCML as an RL and a MARL problem.
- Example RL Agent for SCML (15min) Describes how to train and test and RL agent for SCML.
- Hands-on hackathon (25min) Participants will be given 25min to develop their own agent focusing on modifying a single component of the newly developed agent. The main goal of this step is not to come up with a strong agent but to make sure that the participant have understood the structure of the agent and build confidence in her/his ability to develop a real agent for future competitions.
- Conclusions (5min)
- The tutorial will be wrapped-up by a summary of the information introduced and will provide interesting directions of research inviting the audience to actively participating in pushing forward this exciting domain
Material (GitHub)
You can find all the materials related to this tutorial at the corresponding
SCML
The library provides tools for the SCM case study and participation in ANAC's SCML.
pip install scml
Sample Agents
You can download all agents submitted to SCML2019, SCML2020 and SCML2021 here.
pip install scml-agents