Loading repository data…
Loading repository data…
Oleffa / repository
LeagueAI software framework for League of Legends that provides information about the state of the game based on Image Recognition using OpenCV and Pytorch.
A transparent discovery signal based on current public GitHub metadata.
This score does not audit code, security, maintainers, documentation quality, or suitability. Verify the repository and its current documentation before adoption.
Implementation of an A.I. Player for the videogame League of Legends based on Image Recognition using PyTorch
Demo Video: https://youtu.be/iB4PoNJuXzc
Attention: The old version using Tensorflow can be found in the branch "LeagueAI_2017".
Attention: This project is still under development, the datasets I generate and the YOLOv3 object detection weights I trained will be made available soon!
This is a newer version of the old implementation and shall provide a framework that lets you build your own applications based on detecting objects in the game. As an example I will implement the old 2017 LeagueAI bot on this framework. Demo video of Tensorflow implementation from 2017: https://www.youtube.com/watch?v=KRWFCaXfOTk
The task is to create an agent that is able to play 3rd person massive multiplayer online battle arena games (MOBA) like League of Legends, Dota 2 and Heroes of the Storm with the same input as a human player, namely vision. Since League of Legends does not provide an interface to the game, object detection is used. In this project a python implementation of Yolo v3 object detector and a way of randomly generating an infinite amout of training data is introduced.
For more information on the object detector refer to my technical report at: https://arxiv.org/abs/1905.13546 or directly to the YOLOv3 website [2]
TODO
Creating large datasets from scratch can be very work intensive. For the first implementation of the LeageAI about 700 hand labeled pictures were used. Labeling 700 pictures took about 4 days of work and only included 4 game objects (1 champion model, allied and enemy minions and enemy towers). Therefore, the new dataset was created by automatically generating training data based on 3D models extracted from the game.
pyFrameExporter.py script to extract individual pictures from the clips.
For the minions I used Adobe After Effects to add a green background to the videos of the minions and towers (all objects where I could not find the 3D models).
For each of the objects exported frames I used the pyExportTransparentPNG.py script.
The script removes the green/purple background from the individual screenshots and leaves you with the masked png of an objects.
Furthermore, the scrip crops the images to the content and removes excess seethrough space.This leaves me with about 1000 masked images of each object that can be later used to generate labeled fake screenshots of the game.
To generate a large variety of screenshots the script can be adjusted to:
Using this method a dataset of many thousands of different labeled fake screenshots can be generated in a matter of hours.
A full description of the dataset generation process and evaluation compared to hand labeled data can be found in the following publication: https://arxiv.org/pdf/1905.13546.pdf
To cite please use:
@article{leagueaidatasetgeneration,
title={LeagueAI: Improving object detector performance and flexibility through automatically generated training data and domain randomization},
author={Struckmeier, Oliver},
journal = {arXiv},
year={2019}
}
TODO
[1] Implementing Yolov3 object detection from scratch: https://github.com/ayooshkathuria/YOLO_v3_tutorial_from_scratch
[2] For training: https://pjreddie.com/darknet/yolo/, Yolov3: An Incremental Improvement, J. Redmond and A. Farhadi, 2018