Thumbnail
Access Restriction
Open

Author Gusmão, António ♦ Raiko, Tapani
Source CiteSeerX
Content type Text
File Format PDF
Subject Domain (in DDC) Computer science, information & general works ♦ Data processing & computer science
Subject Keyword Real-time Strategy Game ♦ Reinforcement Learning ♦ Action Space ♦ Experimental Result ♦ Testing Environment ♦ Continuous State ♦ Model-based Algorithm ♦ Control Policy ♦ Temporally-extended Action ♦ Large Continuous State ♦ Rts Game ♦ Traditional Reinforcement ♦ Modern Real-time Strategy ♦ Discrete State Space ♦ Model-based Monte-carlo Method ♦ Novel Online Search Procedure ♦ Decision-making Process
Abstract We consider the problem of effective and automated decisionmaking in modern real-time strategy (RTS) games through the use of reinforcement learning techniques. RTS games constitute environments with large, high-dimensional and continuous state and action spaces with temporally-extended actions. To operate under such environments we propose Exlos, a stable, model-based Monte-Carlo method. Contrary to existing model-based algorithms, Exlos assumes models are imperfect, reducing their influence in the decision-making process. Its effectiveness is further improved by including a novel online search procedure in the control policy. Experimental results in a testing environment show the superiority of Exlos in discrete state spaces when compared to traditional reinforcement learning methods such as Q-learning and Sarsa. Furthermore, Exlos is shown to be effective and efficient on an environment with a large continuous state and action space. This work is a summary of [Gusmao 2011].
Educational Role Student ♦ Teacher
Age Range above 22 year
Educational Use Research
Education Level UG and PG ♦ Career/Technical Study