Effectiveness of Q-learning as a tool for calibrating agent-based supply network models

This paper examines effectiveness of Q-learning as a tool for specifying agent attributes and behaviours in agent-based supply network models. Agent-based modelling (ABM) has been increasingly employed to study supply chain and supply network problems. A challenging task in building agent-based supply network models is to properly specify agent attributes and behaviours. Machine learning techniques, such as Q-learning, can be a useful tool for this purpose. Q-learning is a reinforcement learning technique that has been shown to be an effective adaptation and searching mechanism in distributed settings. In this study, Q-learning is employed by supply network agents to search for 'optimal' values for a parameter in their operating policies simultaneously and independently. Methods are designed to identify the 'optimal' parameter values against which effectiveness of the learning is evaluated. Robustness of the learning's effectiveness is also examined through consideration of different model settings and scenarios. Results show that Q-learning is very effective in finding the 'optimal' parameter values in all model settings and scenarios considered.