Data collection with screen capture

Jason Traish, James Tulip, Wayne Moore

Research output: Book chapter/Published conference paperConference paper

Abstract

Game traces are an important aspect of analysing how players interact with computer games and developing case based reasoning agents for such games. We present a computer vision based approach using screen capture for extracting such game traces. The system uses image templates of to identify and log changes in game state. The advantage of the system is that it only captures events which actually occur in a game and is robust in the face of multiple redundant commands and command cancellation. This paper demonstrates the use of such a vision based system to gather build orders from Starcraft 2 and compares the results generated with those produced by a system based on analysing log files of user actions. Our results show that the vision based system is capable of not only automatically retrieving data via screen capture, but does so more accurately and reliably than a system relying completely on recorded user interactions. Screen capture also allows access to data not otherwise available from an application. We show how screen capture can be used to retrieve data from the DotA 2 picking phase in real time. This data can be used to support meta-game activity, and guide in-game player behaviours.
Original languageEnglish
Title of host publicationAISB Convention 2015
Place of PublicationUnited Kingdom
PublisherThe Society for the Study of Artificial Intelligence and the Simulation of Behaviour
Pages1-4
Number of pages4
Publication statusPublished - 2015
EventArtifical Intelligence and Simulation of Behaviour Convention - University of Kent, Canterbury; United Kingdom, United Kingdom
Duration: 20 Apr 201522 Apr 2015

Conference

ConferenceArtifical Intelligence and Simulation of Behaviour Convention
CountryUnited Kingdom
Period20/04/1522/04/15

Fingerprint

Computer games
Case based reasoning
Computer vision

Cite this

Traish, J., Tulip, J., & Moore, W. (2015). Data collection with screen capture. In AISB Convention 2015 (pp. 1-4). United Kingdom: The Society for the Study of Artificial Intelligence and the Simulation of Behaviour.
Traish, Jason ; Tulip, James ; Moore, Wayne. / Data collection with screen capture. AISB Convention 2015. United Kingdom : The Society for the Study of Artificial Intelligence and the Simulation of Behaviour, 2015. pp. 1-4
@inproceedings{329941fcfb5e4635b997c539fbc4deb1,
title = "Data collection with screen capture",
abstract = "Game traces are an important aspect of analysing how players interact with computer games and developing case based reasoning agents for such games. We present a computer vision based approach using screen capture for extracting such game traces. The system uses image templates of to identify and log changes in game state. The advantage of the system is that it only captures events which actually occur in a game and is robust in the face of multiple redundant commands and command cancellation. This paper demonstrates the use of such a vision based system to gather build orders from Starcraft 2 and compares the results generated with those produced by a system based on analysing log files of user actions. Our results show that the vision based system is capable of not only automatically retrieving data via screen capture, but does so more accurately and reliably than a system relying completely on recorded user interactions. Screen capture also allows access to data not otherwise available from an application. We show how screen capture can be used to retrieve data from the DotA 2 picking phase in real time. This data can be used to support meta-game activity, and guide in-game player behaviours.",
keywords = "Computer vision, Data collection, Engineering controlled terms: Case based reasoning, Engineering main heading: Computer games, Game players, Screen capture, System use, User action, User interaction, Vision based system, Vision-based approaches",
author = "Jason Traish and James Tulip and Wayne Moore",
note = "Imported on 03 May 2017 - DigiTool details were: publisher = 2015. Event dates (773o) = 20-22 April 2015; Parent title (773t) = Artifical Intelligence and Simulation of Behaviour Convention.",
year = "2015",
language = "English",
pages = "1--4",
booktitle = "AISB Convention 2015",
publisher = "The Society for the Study of Artificial Intelligence and the Simulation of Behaviour",

}

Traish, J, Tulip, J & Moore, W 2015, Data collection with screen capture. in AISB Convention 2015. The Society for the Study of Artificial Intelligence and the Simulation of Behaviour, United Kingdom, pp. 1-4, Artifical Intelligence and Simulation of Behaviour Convention, United Kingdom, 20/04/15.

Data collection with screen capture. / Traish, Jason; Tulip, James; Moore, Wayne.

AISB Convention 2015. United Kingdom : The Society for the Study of Artificial Intelligence and the Simulation of Behaviour, 2015. p. 1-4.

Research output: Book chapter/Published conference paperConference paper

TY - GEN

T1 - Data collection with screen capture

AU - Traish, Jason

AU - Tulip, James

AU - Moore, Wayne

N1 - Imported on 03 May 2017 - DigiTool details were: publisher = 2015. Event dates (773o) = 20-22 April 2015; Parent title (773t) = Artifical Intelligence and Simulation of Behaviour Convention.

PY - 2015

Y1 - 2015

N2 - Game traces are an important aspect of analysing how players interact with computer games and developing case based reasoning agents for such games. We present a computer vision based approach using screen capture for extracting such game traces. The system uses image templates of to identify and log changes in game state. The advantage of the system is that it only captures events which actually occur in a game and is robust in the face of multiple redundant commands and command cancellation. This paper demonstrates the use of such a vision based system to gather build orders from Starcraft 2 and compares the results generated with those produced by a system based on analysing log files of user actions. Our results show that the vision based system is capable of not only automatically retrieving data via screen capture, but does so more accurately and reliably than a system relying completely on recorded user interactions. Screen capture also allows access to data not otherwise available from an application. We show how screen capture can be used to retrieve data from the DotA 2 picking phase in real time. This data can be used to support meta-game activity, and guide in-game player behaviours.

AB - Game traces are an important aspect of analysing how players interact with computer games and developing case based reasoning agents for such games. We present a computer vision based approach using screen capture for extracting such game traces. The system uses image templates of to identify and log changes in game state. The advantage of the system is that it only captures events which actually occur in a game and is robust in the face of multiple redundant commands and command cancellation. This paper demonstrates the use of such a vision based system to gather build orders from Starcraft 2 and compares the results generated with those produced by a system based on analysing log files of user actions. Our results show that the vision based system is capable of not only automatically retrieving data via screen capture, but does so more accurately and reliably than a system relying completely on recorded user interactions. Screen capture also allows access to data not otherwise available from an application. We show how screen capture can be used to retrieve data from the DotA 2 picking phase in real time. This data can be used to support meta-game activity, and guide in-game player behaviours.

KW - Computer vision

KW - Data collection

KW - Engineering controlled terms: Case based reasoning

KW - Engineering main heading: Computer games

KW - Game players

KW - Screen capture

KW - System use

KW - User action

KW - User interaction

KW - Vision based system

KW - Vision-based approaches

M3 - Conference paper

SP - 1

EP - 4

BT - AISB Convention 2015

PB - The Society for the Study of Artificial Intelligence and the Simulation of Behaviour

CY - United Kingdom

ER -

Traish J, Tulip J, Moore W. Data collection with screen capture. In AISB Convention 2015. United Kingdom: The Society for the Study of Artificial Intelligence and the Simulation of Behaviour. 2015. p. 1-4