RSPSO Algorithm for Finding the Best Point for Robot to Score a Goal

Authors

  • Asseel Jabbar Almahdi University of Thi-Qar, Computer Science and Mathematics Collage, Computer Science Department

Keywords:

Reinforcement learning, Particle swarm optimization (PSO), robot soccer, Q-learning

Abstract

The issue of robotic football game is one of the most complex multi-agent systems. In order to make the
game more enjoyable and exciting by scoring goals, therefore it's important for a robot football agent to
have a technique on how to score a goal. This paper proposes RSPSO algorithm, where the particle swarm
optimization algorithm (PSO) based on Q-learning is used to find the best point from the goal gate to
shoot the ball toward it. As we show in simulation, our method enables the robot to learn soccer skills to
score a goal.

References

Kitano, H., Asada, M., Kuniyoshi, Y., Noda, I., Osawa, E., Robocup, H.M.: "A challenge problem

for ai. AI mag.", 18(1), 73, 1997.

Visser U & Burkhard HD, "RoboCup: 10 years of achievements and future challenges", AI

Magazine, 28(2), p. 115, 2007.

A. Likas, "Multivalued Parallel Recombinative Reinforcement Learning: A Multivalued Genetic

Algorithm", In Proc, HERCMA Conference, 1998.

Watkins CJCH," Learning from Delayed Rewards", Doctoral Dissertation, University of Cambridge

(MA), 1989.

Budden DM," Applications of machine learning techniques to humanoid robot platforms", Doc-toral

Dissertation, University of Newcastle, Australia, 2012.

Ogino M, Katoh Y, Aono M, Asada M & Hosoda K, "Reinforcement learning of humanoid rhythmic

walking parameters based on visual information", Advanced Robotics, 18(7), pp. 677-697, 2004.

Kobayashi H, Osaki T, Williams E, Ishino A & Shinohara," A, RoboCup 2006: Robot Soccer World

Cup X", Springer, Berlin, Germany, Available from http://dx.doi.org/10.1007/978-3-540-74024-7_8,

Riedmiller M & Gabel T, "On experiences in a complex and competitive gaming domain:

Reinforcement learning meets RoboCup", Proceedings of the Computational Intelligence and Games,

CIG 2007, IEEE Symposium, pp. 17-23, 2007.

Duan Y, Liu Q & Xu X, "Application of reinforcement learning in robot soccer", Engineering

Applications of Arti_cial Intelligence, 20(7), pp. 936-950, 2007.

Riedmiller M, Gabel T, Hafner R & Lange S," Reinforcement learning for robot soccer",

Autonomous Robots, 27(1), pp. 55-73, 2009.

Hester T, Quinlan M & Stone P, "Generalized model learning for reinforcement learning on a

humanoid robot", Proceedings of the IEEE International Conference on Robotics and Automation

(ICRA), pp. 2369-2374, 2010.

Stone P, Sutton RS & Kuhlmann G, "Reinforcement learning for RoboCup soccer keepaway,

Adaptive Behavior", 13(3), pp. 165-188, 2005.

Neri JRF, Zatelli MR, Farias dos Santos C & Fabro JA, "A proposal of Q-learning to control the

attack of a 2D robot soccer simulation team", Proceedings of the Robotics Symposium and Latin

American Robotics Symposium (SBR-LARS), pp. 174-178, 2012.

E. Uchibe. "Cooperative Behavior Acquisition by Learning and Evolution in a Multi-Agent

Environment for Mobile Robots". PhD thesis, Osaka University, January 1999.

P. Stone and M. Veloso. "Team-partitioned, opaque-transition reinforcement learning". In M.

Asada and H. Kitano, editors, RoboCup-98: Robot Soccer World Cup

II. Springer Verlag, Berlin, 1999. Also, in Proceedings of the Third International Conference on

Autonomous Agents,1999.

T. Andou. "Refinement of soccer agents' positions using reinforcement learning". In H. Kitano,

editor, RoboCup-97: Robot Soccer World Cup I, pages 373-388. Springer Verlag, Berlin, 1998.

M Yoon, J Bekker, S Kroon, "New reinforcement learning algorithm for robot soccer", ORiON

journals, Volume 33 (1), pp. 1-20, 2017.

Jo˜ao Cunha · Rui Serra · Nuno Lau., " Batch Reinforcement Learning for Robotic Soccer Using

the Q-Batch Update-Rule", Springer Science+Business Media Dordrecht, J Intell Robot Syst, DOI

1007/s10846-014-0171-1, 2015.

HAOBIN SHI, ZHIQIANG LIN, KAO-SHING HWANG," An Adaptive Strategy Selection

Method with Reinforcement Learning for Robotic Soccer Games", IEEEAccess, VOLUME 6, 2018.

Kennedy, J.; Eberhart, R. "Particle Swarm Optimization". Proceedings of IEEE International

Conference on Neural Networks IV. pp. 1942– 1948, (1995).

Downloads

Published

2020-12-01

Issue

Vol. 10 No. 1 (2020)

Section

Articles

License

The Authors submitting a manuscript do so on the understanding that if accepted for publication, copyright of the article shall be assigned to Journal of education for Pure Science (Jeds), University of Thi-Qar as publisher of the journal.

Copyright encompasses exclusive rights to reproduce and deliver the article in all form and media, including reprints, photographs, microfilms and any other similar reproductions, as well as translations. The reproduction of any part of this journal, its storage in databases and its transmission by any form or media, such as electronic, electrostatic and mechanical copies, photocopies, recordings, magnetic media, etc. , will be allowed only with a written permission from Journal of education for Pure Science (Jeds), University of Thi-Qar.

Journal of education for Pure Science (Jeds), University of Thi-Qar, the Editors and the Advisory International Editorial Board make every effort to ensure that no wrong or misleading data, opinions or statements be published in the journal. In any way, the contents of the articles and advertisements published in the Journal of education for Pure Science (Jeds), University of Thi-Qar are sole and exclusive responsibility of their respective authors and advertisers.