Studying Adaptation in an Artificial World
SMARTS is an artificial life program. It simulates a two-dimensional world, creatures that move around in that world ("critters"), and the food and rocks that they encounter. The program has two compiled modules; in both of these the critters become smarter as they adapt to their environments. In the Learning module, the critters have the potential to learn how to negotiate their world since they receive rewards and punishments for their actions. The critters' "brains" use reinforcement learning to allow them to behave more appropriately for the environment they have been placed in. In the Evolution module, a population of critters evolves using a genetic algorithm. The critters' behavior is specified in their genes, and those critters whose genes cause them to act appropriately tend to survive and reproduce, leading to a more fit population.
You are also free to work with the program in ways that go beyond these two modules. The source code is available with the downloaded program and is freely modifiable under the terms of the GNU General Public License. However, instructions are provided only for the two compiled modules.
At a given time, each critter sits on a hexagonal cell with a heading in one of the six possible directions. It has a set of touch sensors for gathering input from one or more of the six cells around it and a set of actions that it can choose to take on the basis of what it senses. Time is divided into discrete time steps. On each time step, a critter can move one cell in the direction it is heading, it can turn so that it is heading in one of the other five directions, or it can choose to do nothing. If it moves onto a cell with a plant, it automatically eats the plant. In the Evolution version of the program, if a critter moves onto a cell with another critter, under certain circumstances, it automatically mates with the other critter, producing two offspring.
Each critter has a strength which increases when it eats and decreases when it mates or bumps into a rock. Moving and turning may also cause a slight decrease in strength. Once a critter's strength reaches zero, it dies and can no longer sense or take actions.
SMARTS is designed to acquaint you with reinforcement learning and genetic algorithms and to show how learning and evolution can solve similar problems. The program allows you to change the rewards and punishments that the critters receive for different actions, the values of parameters governing the algorithms, and the number of things of different types that appear in the world. In this way you can experiment with the effects of different environments on the learning and evolution.
Separate programs are provided for the versions of SMARTS that implement reinforcement learning and genetic algorithms. Each of these is in the form of a "JAR archive", an archived version of a Java program. In order to run the programs, you will need a computer running Windows, Mac OS, or Linux that has the Java Virtual Machine, version 5.0 or later, installed.
Download one or both of the SMARTS JAR archives: SMARTS Learning, SMARTS Evolution. Now you should be able to run either version of the program by double-clicking on its icon or selecting it and then selecting "Open".
The Java source files for the program are also included in the JAR archives. To extract all of the files in an archive execute the following command in the directory where the archive is on your computer:
jar xf <JAR archive>
where <JAR archive> should be replaced by the filename.
When you start up either version of the program, you will see three windows: the Control window (with "SMARTS Learning" or "SMARTS Evolution" as its title); the Messages window; and the World window.
The Control window has three menus (these menus may also be available to the whole program, depending on the operating system): Behavior, Options, and Windows. The "Behavior" menu controls whether the critters learn (in the Learning version of the program) or evolve (in the Evolution version). There are two additional menu items in the Evolution version of the program; they are discussed on the Evolution page. The "Options" menu controls whether certain additional windows appear and also permits the user to print more or fewer messages as the program runs. Selecting "Increase verbosity" causes more messages to be printed; selecting "Decrease verbosity" causes fewer messages to be printed. The "Windows" menu allows the user to bring an open window to the front.
The Control window also has five buttons. The "Play" and "Step" buttons are indicated by the icons and . The "Step" button causes the world to move forward one "time step"; each critter senses what's around it, decides what to do, and then executes one action. In addition, if any plants have been eaten, enough new plants appear to make the number constant. The "Play" button causes the world to move forward 200 time steps. During a run, the "Play" button's icon changes to a pause icon (); clicking on the button then causes the world to "pause" until the button is clicked again. The "Quit" button causes the program to quit. The "Reinitialize" button creates a new state of the world; in particular all of the critters start with their default initial strength. The "Change params" button brings up a window that allows the user to change the value of particular parameters controlling the world or the critters. To change a parameter, first select the value you want to change by double-clicking it. Then type the new value, and either hit "Return" or select another parameter. When you are finished changing parameters, click on "Done". You can then continue running the program with the new parameter values by clicking on the "Play" or "Step" button or start over with the new values by clicking on "Reinitialize".
The World window gives a graphical representation of the world and the critters (red or magenta circles), plants (green circles), and rocks (gray circles) within it. Each of these colors, as well as black, representing emptiness, corresponds to a different "texture" that is distinguishable by the critters' touch sensors. The strength of critters is indicated by the saturation of the color; white critters have died. In the Evolution version of the program, critters may be either red or magenta. Magenta critters are "matable" — that is, they can mate with another matable critter — red critters cannot mate (see SMARTS: Evolution for details). Each critter has a current heading, one of the six "cardinal" directions and the only direction that it can move in (without turning). A critter's heading is indicated by its "mouth" in the display.
The Messages window displays information about what is going on in the world. You can make the program more "verbose", so that more information is displayed, by selecting "Increase verbosity" from the "Options" menu. The current verbosity is shown in parentheses after this menu item and the "Decrease verbosity" menu item. When the verbosity is greater than 0, you should not click on the "Play" button because this will result in a very large number of messages.