in partial fulfillment of the requirements for the degree of

Plan recognition is the task of inferring a plan, and consequently a goal, that takes into account the observed behavior of the agent. We will see later that this set of candidates is necessary for many current plan recognition approaches. Then chapter 3 presents the recognition of the web design we developed for the RTS StarCraft game, the different experiments they have.

The second contribution of the thesis is presented in Chapter 4, where we introduce the problem of inferring an agent's goals. We have developed a method to infer, in real time, the goal the opponent is trying to achieve in an RTS game and the most likely plan for the opponent to achieve it. For a more general purpose, we also developed a target recognition method; this method has the particularity of building a set of possible goals for the agent, where the other approaches start with a given set of possible goals and will select the most likely ones.

With our method we do not predict which goal is most likely, but we build the set of possible goals of the agent. Knowing that most current approaches to plan recognition require a set of possible goals to work, we could combine such plan recognition.

INTRODUCTION 10

Plan and Goal Recognition in AI

INTRODUCTION 11

INTRODUCTION 12

INTRODUCTION 13

INTRODUCTION 14

Game AI: a Challenging Domain of Application

Plan and Goal Recognition in Games

INTRODUCTION 15

INTRODUCTION 16

INTRODUCTION 17

Real-Time Strategy Games

INTRODUCTION 18

INTRODUCTION 19

INTRODUCTION 20

Organization of the Thesis

INTRODUCTION 21

Plan Recognition as Planning

In their paper [14], Ramirez and Geffner assume that the agent they observe is perfectly rational and always follows optimal plans. The observations are sequences of actions, and although some actions could be missing due to noise, they assume that the order is the same as it was when it was executed by the agent. They have a set of possible goals, design an optimal plan to achieve each goal from the agent's starting position, and then, according to the observations, remove the plans and goals that do not include them.

Using planning will generate three plans, one to reach each possible destination via the shortest path, from "A". Later we observe the transition of the agent from "A" to "B", all the plans are preserved, since they all involve this transition. We then observe the transition from “F” to “G”, which leaves the goal of achieving “C” and its associated plan from the optimal plan to achieve it.

BACKGROUND 23

BACKGROUND 24

Definitions

Planning Problem
Plan Recognition Problem
Satisfiability of an Action Sequence

BACKGROUND 25

Concept Learning

In this chapter of the dissertation we describe the plan recognition method we developed for StarCraft. Using the definitions introduced earlier, we explain in section 3.1 which specific planning problem we have to solve and how we solve it.

Heuristic Planning in RTS Games

ONLINE PLAN RECOGNITION FOR RTS GAMES 27

ONLINE PLAN RECOGNITION FOR RTS GAMES 30

Plan Recognition Method

ONLINE PLAN RECOGNITION FOR RTS GAMES 35

Evaluation

Methodology

ONLINE PLAN RECOGNITION FOR RTS GAMES 44

Results

ONLINE PLAN RECOGNITION FOR RTS GAMES 54

Possible Improvements and Discussion

Multi-Horizons Planning

ONLINE PLAN RECOGNITION FOR RTS GAMES 56

New Planner and Heuristic

Discussion

ONLINE PLAN RECOGNITION FOR RTS GAMES 57

In this part we are going to study a method developed to perform target recognition. Current approaches in goal recognition have not yet attempted to apply concept learning to a propositional logic formalism. We developed a method to infer an agent's possible goal by observing this agent in series of successful attempts to achieve its goal and using concept learning on these observations.

We propose an algorithm, LFST (Learning From Successful Traces) to produce concise hypotheses about the agent's goal. We show that, if such a goal exists, our algorithm always provides a possible goal for the agent, and we evaluate the performance of our algorithm in different settings. We compare it with another concept learning algorithm that uses a formalism close to ours, and we get better results in producing the hypotheses with our algorithm.

We present a way to use assumptions about agent behavior and environment dynamics, thereby improving agent goal inference by optimizing the search space of possible goals. As we saw in Chapter 1, Section 1.1, much previous work has attempted to solve the plan recognition problem by using a set of predefined goals for the agent and guessing which one is most likely. Often, they assume that this set of predetermined goals is a given, but it usually isn't.

This assumption, made in previous work and which we do not take for granted, motivated our work.

GOAL RECOGNITION 59

Inferring an Agent’s Goals Using Concept Learning

Problem Formalization

GOAL RECOGNITION 60

GOAL RECOGNITION 61

GOAL RECOGNITION 62

Learning From Successful Traces

GOAL RECOGNITION 63

Experiments

GOAL RECOGNITION 64

Experimental Protocol

GOAL RECOGNITION 65

GOAL RECOGNITION 66

GOAL RECOGNITION 67

GOAL RECOGNITION 68

GOAL RECOGNITION 69

Measures

Results

GOAL RECOGNITION 70

GOAL RECOGNITION 71

GOAL RECOGNITION 72

GOAL RECOGNITION 73

GOAL RECOGNITION 74

GOAL RECOGNITION 75

Extensions and Discussion

Inferring the Agent’s Model and Environment Rules From Data

GOAL RECOGNITION 76

GOAL RECOGNITION 77

GOAL RECOGNITION 78

GOAL RECOGNITION 79

Discussion

GOAL RECOGNITION 80

GOAL RECOGNITION 81

GOAL RECOGNITION 82

GOAL RECOGNITION 83

GOAL RECOGNITION 85

GOAL RECOGNITION 86

Contribution of the Thesis

But, unfortunately, there are still some serious problems that slow down the use of goal recognition in large-scale real-world situations (computation time, action space, etc.). As we indicated in the introduction, RTS games are very complex games, and the solution of plan recognition in such games and in real time brings us closer to real-world applications. Even when it limits the scope of application in games, it is actually a very important economic sector with an increasing demand for adaptive AI in order to make games more fun for people.

Compared to most current plan recognition approaches, we also deal with unordered actions and intertwined plans. Our method predicts the goal of the observed player and infers the plan to achieve this goal. The performance is comparable to machine learning methods used in previous work for the same purpose (target recognition).

However, our method has the advantage that it can work without training data, but only using expert knowledge. We proposed to extend this method by including multi-horizon prediction, which means predicting the player's plan at the five-minute horizon, and when we reach this horizon, predict the new plan at the ten-minute horizon based on the previous prediction and continue for future horizons.

CONCLUSION 88

Combining Plan and Goal Recognition

CONCLUSION 89

CONCLUSION 90

Further Perspectives

CONCLUSION 91

CONCLUSION 92

CONCLUSION 93

Miller, ‘A new model of plan Recognition’, in Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, 1999. Wellman, ‘Probabilistic state-dependent grammaticas for plan Recognition’, in Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence, 1999. , 2000. Mooney, ‘Abductieve markov-logica voor planherkenning.’, in AAAI Proceedings of the Twenty5th National Conference on Artificial Intelligence 2011, pp.

Kantharaju, "Leer kombinerende kategorie-grammatika vir planherkenning," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. Geffner, “Plan recognition as planning,” in Proceedings of the Twenty- First International Joint Conference on Artificial Intelligence (IJCAI-09), 2009. Smith, “A Fast goal recognition technique based on interaction skattings,” in Proceedings of the 24th International Conference on Kunsmatige Intelligensie, pp.

Kaminka, ‘Heuristische online doelherkenning in continue domeinen’, in Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pp. Meneguzzi, ‘Doelherkenning in latente ruimte’, in 2018 International Joint Conference on Neural Networks (IJCNN), pp. Karpas, ‘Doelherkenningsontwerp’, in Proceedings of the International Conference on Automated Planning and Scheduling, vol.

Lester, "Goal recognition with markov logic networks for player-adaptive games." in Proceedings of the Seventh AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment 2011, 2011. Bessiere, "A Bayesian model for plan recognition in rts games used to starcraft , ” in Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, vol. Lester, “Player Goal Recognition in Open World Digital Games Using Long Short-Term Memory Networks,” in Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI'16, p.

Lester, “Multimodal Target Recognition in Open-World Digital Games,” in Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, vol. Building useful virtual agents using pattern recognition and planning,” in Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, vol. Buro, “Build order optimization in starcraft,” in Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, vol.