RSPSO Algorithm for Finding the Best Point for Robot to Score a Goal

Authors

  • Asseel Jabbar Almahdi University of Thi-Qar, Computer Science and Mathematics Collage, Computer Science Department

DOI:

https://doi.org/10.32792/jeps.v10i1.29

Keywords:

Reinforcement learning, Particle swarm optimization (PSO), robot soccer, Q-learning

Abstract

The issue of robotic football game is one of the most complex multi-agent systems. In order to make the
game more enjoyable and exciting by scoring goals, therefore it's important for a robot football agent to
have a technique on how to score a goal. This paper proposes RSPSO algorithm, where the particle swarm
optimization algorithm (PSO) based on Q-learning is used to find the best point from the goal gate to
shoot the ball toward it. As we show in simulation, our method enables the robot to learn soccer skills to
score a goal.

References

Kitano, H., Asada, M., Kuniyoshi, Y., Noda, I., Osawa, E., Robocup, H.M.: "A challenge problem

for ai. AI mag.", 18(1), 73, 1997.

Visser U & Burkhard HD, "RoboCup: 10 years of achievements and future challenges", AI

Magazine, 28(2), p. 115, 2007.

A. Likas, "Multivalued Parallel Recombinative Reinforcement Learning: A Multivalued Genetic

Algorithm", In Proc, HERCMA Conference, 1998.

Watkins CJCH," Learning from Delayed Rewards", Doctoral Dissertation, University of Cambridge

(MA), 1989.

Budden DM," Applications of machine learning techniques to humanoid robot platforms", Doc-toral

Dissertation, University of Newcastle, Australia, 2012.

Ogino M, Katoh Y, Aono M, Asada M & Hosoda K, "Reinforcement learning of humanoid rhythmic

walking parameters based on visual information", Advanced Robotics, 18(7), pp. 677-697, 2004.

Kobayashi H, Osaki T, Williams E, Ishino A & Shinohara," A, RoboCup 2006: Robot Soccer World

Cup X", Springer, Berlin, Germany, Available from http://dx.doi.org/10.1007/978-3-540-74024-7_8,

Riedmiller M & Gabel T, "On experiences in a complex and competitive gaming domain:

Reinforcement learning meets RoboCup", Proceedings of the Computational Intelligence and Games,

CIG 2007, IEEE Symposium, pp. 17-23, 2007.

Duan Y, Liu Q & Xu X, "Application of reinforcement learning in robot soccer", Engineering

Applications of Arti_cial Intelligence, 20(7), pp. 936-950, 2007.

Riedmiller M, Gabel T, Hafner R & Lange S," Reinforcement learning for robot soccer",

Autonomous Robots, 27(1), pp. 55-73, 2009.

Hester T, Quinlan M & Stone P, "Generalized model learning for reinforcement learning on a

humanoid robot", Proceedings of the IEEE International Conference on Robotics and Automation

(ICRA), pp. 2369-2374, 2010.

Stone P, Sutton RS & Kuhlmann G, "Reinforcement learning for RoboCup soccer keepaway,

Adaptive Behavior", 13(3), pp. 165-188, 2005.

Neri JRF, Zatelli MR, Farias dos Santos C & Fabro JA, "A proposal of Q-learning to control the

attack of a 2D robot soccer simulation team", Proceedings of the Robotics Symposium and Latin

American Robotics Symposium (SBR-LARS), pp. 174-178, 2012.

E. Uchibe. "Cooperative Behavior Acquisition by Learning and Evolution in a Multi-Agent

Environment for Mobile Robots". PhD thesis, Osaka University, January 1999.

P. Stone and M. Veloso. "Team-partitioned, opaque-transition reinforcement learning". In M.

Asada and H. Kitano, editors, RoboCup-98: Robot Soccer World Cup

II. Springer Verlag, Berlin, 1999. Also, in Proceedings of the Third International Conference on

Autonomous Agents,1999.

T. Andou. "Refinement of soccer agents' positions using reinforcement learning". In H. Kitano,

editor, RoboCup-97: Robot Soccer World Cup I, pages 373-388. Springer Verlag, Berlin, 1998.

M Yoon, J Bekker, S Kroon, "New reinforcement learning algorithm for robot soccer", ORiON

journals, Volume 33 (1), pp. 1-20, 2017.

Jo˜ao Cunha · Rui Serra · Nuno Lau., " Batch Reinforcement Learning for Robotic Soccer Using

the Q-Batch Update-Rule", Springer Science+Business Media Dordrecht, J Intell Robot Syst, DOI

1007/s10846-014-0171-1, 2015.

HAOBIN SHI, ZHIQIANG LIN, KAO-SHING HWANG," An Adaptive Strategy Selection

Method with Reinforcement Learning for Robotic Soccer Games", IEEEAccess, VOLUME 6, 2018.

Kennedy, J.; Eberhart, R. "Particle Swarm Optimization". Proceedings of IEEE International

Conference on Neural Networks IV. pp. 1942– 1948, (1995).

Downloads

Published

2020-12-01

Issue

Vol. 10 No. 1 (2020)

Section

Articles

License

Copyright Policy

Authors retain copyright of their articles published in the Journal of Education for Pure Science (JEPS).

By submitting their work, authors grant the journal a non-exclusive license to publish, distribute, and archive the article in all formats and media.

License

All articles published in JEPS are licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).

This license permits unrestricted use, distribution, and reproduction in any medium, provided that the original author(s) and the source are properly credited.

Author Rights

Authors have the right to:

  • Share their articles on personal websites, institutional repositories, and academic platforms

  • Reuse their work in future research and publications

  • Distribute the published version without restriction

Journal Rights

The journal retains the right to:

  • Publish and archive the articles

  • Include them in indexing and archiving systems such as LOCKSS and CLOCKSS

  • Promote and disseminate the published work

Responsibility

The contents of all articles are the sole responsibility of the authors. The journal, editors, and editorial board are not responsible for any errors, opinions, or statements expressed in the published articles.

Open Access Statement

JEPS provides immediate open access to its content, supporting the principle that making research freely available to the public enhances global knowledge exchange.

This work is licensed under a Creative Commons Attribution 4.0 International License.
https://creativecommons.org/licenses/by/4.0/