THEORETICAL BASIS 3.1 Introduction
3.3 Agent-based Modelling
3.3.2 Agent-based Computational Economics
3.3.2.1 Overview
According to Tesfatsion (2003, pg. 1) ACE is “the computational study of economics modelled as evolving systems of autonomous interacting agents”. Levy (2009, pg. 1)
66
adds “Agent Based Computational Economics is a framework allowing economics to expand beyond the realm of the “rational representative agent”. ACE can be conceptualised as a mutation of CAS specifically for the field of economics (Chen and Kao, 2016; Tesfatsion, 2002). Economists can utilise ACE to explore economic systems with heterogeneous agents that have bounded rationality (Dilaver et al., 2016;
Levy, 2009).
The three fundamental parts of an ACE model are: the agents, interaction rules and the environment (Corredor, 2007; Van Dinther, 2008). However, Li (2013, pg. 5) states that there are four elements: “The scope of the economic system and its environment”, “The interrelation between the economic system and its environment”,
“Elements of the economic system, i.e. economic agents considered in the economic system” and “The structure of the economic system, which is the interrelation among elements of the economic system”.
The ACE modelling process begins with researchers specifying the structural aspects of agents within an economy and thereafter the characteristics of the agents are included (Tesfatsion, 2003; Gilbert and Terna, 2000). The characteristics of agents can include mental models, behaviour traits and memory. It is critical to set the initial state of the economy before the simulation is initiated. After the initial characteristics of the agents within the economy are specified, behavioural nuances developed and environmental factors set, the model is initiated and there is no further intervention from the researcher until the simulation has run its course. This method sees the agents act as they would in real life, where their actions affect the system, which in turn affects their actions. The feedback loops, whether positive or negative, are critical for the dynamic nature of these models as they will also inform a learning function. Axelrod (2005) contends that simulation, which is the foundation of ACE, is a third way of undertaking science, in addition to induction and deduction.
LeBaron and Tesfatsion (2008) identify three key aspects that need to ideally be present in an ACE model: an empirically based structure and set of agents, the model must be appropriately scaled and the model must be empirically validated.
In addition, Li (2013) makes a good distinction between two types of agents, active economic agents are those that have the ability to make decisions and act on their own
67
accord. Passive economic agents, on the other hand, do not act on their own accord, such as gold.
It is important to note that through this research only a framework has been developed, not a simulation model. The reasons for this are: emanating from the primary data collection process most, if not all, participants seemed to not be interested in laborious exercises, respondents seem to not have the required skills to perform simulation exercises and it was perceived that the stakeholders of the eThekwini green economy will better internalise information through interacting with a framework that is easy to read.
3.3.2.2 Applications for Agent-based Computational Economics
Tesfatsion (2002) provides eight broad areas that ACE research can be categorised into and utilised for: learning of systems, behavioural evolution, organisation modelling, bottom-up system modelling, development of economic structures, development of agents for automated systems, development of ACE laboratories, and real and computation agent concurrent experimentation.
It is important to note that Tesfatsion (2006b) further synthesis ACE research into four areas, which are differentiated by the goals of the research, these are: empirical understanding of why actual events and occurrences mutate and continue to be present without strategic control, normative understanding that aims to test proposed policies and regulations to determine their effectiveness over a specific duration, qualitative insight and theory generation, which seeks to understand a system through its complex and dynamic behaviour and methodological advancement, which aims to provide the relevant theoretical and computational tools that will allow researchers to essentially confirm a hypothesis, derived from a modelling exercise against empirical data.
In terms of the revised classification, this research will fit perfectly in the third objective, that of gaining a qualitative insight to understand a system.
3.3.2.3 Learning algorithms/systems
According to Tesfatsion (2002), the following are the main types of learning algorithms utilised in ACE: Genetic Algorithm learning, Q-Learning and the Classifier system. This is supported by Blecic et al. (2016).
68
Q-Learning can be described by Watkins and Dayan (1992, pg. 55), “… is a form of model-free reinforcement learning. It can also be viewed as a method of asynchronous dynamic programming. It provides agents with the capability of actions, without requiring them to build maps of the domains”. A Q-Learning system tries various actions in a specific state, assesses the immediate outcome as positive or negative for that particular state and stores this in a memory bank (Watkins and Dayan, 1992; Ye and Xu, 2014; Solferino et al., 2015). Ultimately the system is designed to optimise decisions.
According to Liepins and Hilliard (1989) and Sahu et al. (2013), Genetic Algorithms were developed along the lines of natural selection as an optimisation method. Liepins and Hilliard (1989), supported by Kermani et al. (2016) and D’addona and Teti (2013) go to explain that Genetic Algorithm systems are essentially made up of a population of approximately fifty to two hundred agents and works in a step-by-step fashion. There are three fundamental processes that occur in a Genetic Algorithm, these are: the health of each agent is assessed, there is an establishment of strong
‘genes’, primarily from stronger agents, or characteristics of agents and combination of genes evolves to form the agents for the next ‘generation’.
Classifier systems have been utilised in previous ABM research (Bonarini et al., 2007; Holsapple et al., 1998) and in ACE modelling, or similar research (Beltrametti et al., 1997; LeBaron, 2002; Kirman and Vriend, 2001). According to Booker et al.
(1989) and supported by Holmes et al. (2002), a classifier system is a type of reinforcement learning system that is operated in a parallel fashion and functions on fundamental notions of ‘messages’ and rules that are given credit for success and debited for their failures, while searching of new rules. Classifier systems, as highlighted by Booker et al. (1989), is applicable under certain scenarios that exhibit some or all of the following tenets: unique occurrences that are accompanied by a legion of immaterial information, perpetual immediate actions are needed, esoteric objectives and confirmation of positive or negative reinforcement is only available after a lengthy progression of steps.
69 Figure 3.1: Working of a classifier system Source: Adapted from Booker et al. (1989).
According to Booker et al. (1989) and corroborated by Beltrametti et al. (1997), classifier systems essentially works in the following series of steps, as depicted in Figure 3.1: information from the system are received by the ‘Input Interface’ and communicated to the ‘Message List’, the messages are then juxtaposed to the pre- specified conditions of the ‘Classifiers’, to find matches for each matched message, a specific instruction/action is then posted as a further message, all messages on the
‘Messenger List’ are then replaced by newer messengers, the message that results as an ‘Action’ from the ‘Classifier’ is converted to the requirements of the current sequences output and the process is repeated from the beginning.
3.3.3 Advantages and disadvantages of Agent-based Computational Economics