Thumbnail
Access Restriction
Subscribed

Author Holmesparker, C. ♦ Taylor, M.E. ♦ Agogino, A.K. ♦ Tumer, K.
Sponsorship IEEE Comput. Soc.
Source IEEE Xplore Digital Library
Content type Text
Publisher Institute of Electrical and Electronics Engineers, Inc. (IEEE)
File Format PDF
Copyright Year ©2014
Language English
Subject Domain (in DDC) Computer science, information & general works ♦ Special computer methods
Subject Keyword Noise ♦ shaped rewards ♦ UAV coordination ♦ multiagent learning ♦ multiagent coordination ♦ Equations ♦ multiagent UAV ♦ MAV communication network ♦ System performance ♦ Learning (artificial intelligence) ♦ UAV communication network ♦ reward shaping ♦ Mathematical model ♦ Joints ♦ Multi-agent systems
Abstract Coordinating the joint-actions of agents in cooperative multiagent systems is a difficult problem in many real world domains. Learning in such multiagent systems can be slow because an agent may not only need to learn how to behave in a complex environment, but also to account for the actions of other learning agents. The inability of an agent to distinguish between the true environmental dynamics and those caused by the stochastic exploratory actions of other agents creates noise in each agent's reward signal. This learning noise can have unforeseen and often undesirable effects on the resultant system performance. We define such noise as exploratory action noise, demonstrate the critical impact it can have on the learning process in multiagent settings, and introduce a reward structure to effectively remove such noise from each agent's reward signal. In particular, we introduce two types of Coordinated Learning without Exploratory Action Noise (CLEAN) rewards that allow an agent to estimate the counterfactual reward it would have received had it taken an alternative action. We empirically show that CLEAN rewards outperform agents using both traditional global rewards and shaped difference rewards in two domains.
Description Author affiliation: Washington State Univ., Pullman, WA, USA (Taylor, M.E.) || Parflux LLC, Salem, OR, USA (Holmesparker, C.) || Oregon State Univ., Corvallis, OR, USA (Tumer, K.) || Univ. of California at Santa Cruz, Mountain View, CA, USA (Agogino, A.K.)
ISBN 9781479941438
Educational Role Student ♦ Teacher
Age Range above 22 year
Educational Use Research ♦ Reading
Education Level UG and PG
Learning Resource Type Article
Publisher Date 2014-08-11
Publisher Place Poland
Rights Holder Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Size (in Bytes) 354.66 kB
Page Count 8
Starting Page 127
Ending Page 134


Source: IEEE Xplore Digital Library