Encoding strategies

Format
- Binary encoding
  - Possibly advantageous because it yields more schemas and implicit parallelism: each string samples an even larger number of schemas
  - Encoding real-valued parameters such as weights in a neural network
    - Gray code — successive values differ in only one bit:
      000, 001, 011, 010, 110, 111, 101, 100
- Many-character and real-valued encodings
  - Mutation: in evolution strategies, Gaussian or log normal distribution around the current value, with a given standard deviation
Adapting the encoding
- Problems with fixed encodings
  - How to know the proper encoding beforehand
  - Order of the bits in the encoding matters: functionally related loci (members of building blocks) should stay close together (the "linkage problem")
  - Fixed-length representations limit candidate solutions
- "Messy coding": encoding that separates functional position and "real" position: each gene has a functional location and a value (allele), in addition to the position that crossover works with
  - ```
  [(1,0) (2,0) (3,0) (4,1) (5,0) (6,1) (7,0) (8,1)]
```
  is functionally the same string as
```
  [(1,0) (2,0) (6,1) (5,0) (4,1) (3,0) (7,0) (8,1)]
```
- Inversion (Holland)
  - Inversion chooses 2 points within string and reverses order of bits between them.
  - Inversion does not change fitness, but it may lead beneficial schemas to survive longer by decreasing their defining length.
  - Following inversion, crossover will not necessarily produce offspring with a full set of loci without extra provisions; two possibilities:
    - Permit crossover only between genomes with the same permutation of the loci.
    - Use a master/slave approach: choose one parent to be the master, temporarily reorder the other parent to match the master, and use this ordering to produce offspring.

Selection

Fitness-proportionate selection
- Expected number of times an individual is selected is the invidual's fitness divided by the average fitness
  - Roulette wheel sampling
    - Wheel is spun N times, with individual under the marker chosen to be a parent
    - With small populations, the actual number of offspring allocated can be far from its expected value
  - Stochastic universal sampling
    - Wheel spun once, placing N equally spaced pointers which determine which individuals become parents
- With fitness-proportionate selection the rate of evolution depends on the fitness variance in the population.
- Early on, when variance is high, fitness-proportionate selection may put too much emphasis on "exploitation", too little on "exploration", leading to "premature convergence".
- Later, when variance is low, there may be too little selection pressure favoring more fit individuals.
Sigma scaling
- Fitness is scaled by the standard deviation of the fitnesses, keeping selection pressure independent of fitness variance. Scaled fitness of a individual is the deviation of the raw fitness from the mean divided by a multiple of the standard deviation.
  `hat{f}(i,t)={(1 + (f(i,t)-bar{f}(t))/(2 sigma(t)),if sigma(t)!=0),(1,if sigma(t)=0):}`
Boltzmann selection
- Selection pressure starts out low, gradually increases ("exploration" is favored early on, "exploitation" later).
- A temperature parameter controls the rate of selection according to a preset schedule, as in simulated annealing; temperature decreases over time.
  `hat{f}(i,t) = e^(f(i,t)/T_t)/(sum_x e^(f(x,t)/T_t))`
- As temperature decreases, difference in expected value between high and low fitnesses increases.
Rank selection
- Expected value of individual depends on its fitness rank, rather than its absolute fitness.
- Reduces selection pressure when fitness variation is high and keeps up selection pressure when fitness variance is low.
- Given expected values max and min for the highest and lowest ranked individuals, one way to do this:
  `hat{f}(i, t) = min + (max - min) (rank(i, t) - 1) / (N - 1)`
Tournament selection
- Fitness-proportionate methods require two passes through the population; rank selection requires ranking after fitness calculation.
- Tournament selection: one pass, amenable to parallelism
- Choose two individuals at random, then a random number r. If r < k (k a parameter greater than .5), select the fitter of the two; otherwise select the less fit.
Steady-state selection, elitism
- Some relatively large proportion of population maintained from one generation to next
- Useful for domains where incremental "learning" is being modeled or where the whole population is solving the problem together

Crossover

Single-point crossover
- Positional bias: schemas that can be created or destroyed depend strongly on the location of bits in the gene.
- Cannot combine all possible schemas. For example, 11*****1 and ****11** cannot be combined to form 11**11*1.
- Likely to destroy schemas with long defining lengths.
- Hard to know in advance what ordering will group functionally related bits together.
- Particular bits might be crucial in more than one schema, so it may be impossible to put all functionally related bits together in a string.
- Keeping short schemas intact can lead to the preservation of hitchhikers, bits that are not part of a schema but come along with the beneficial elements.
- Segments exchanged always contain the endpoints of the string.
Multiple-point crossover
- Multiple positions are chosen at random, and segments between them exchanged.
- Less likely to disrupt schemas with long defining lengths.
- Can combine more schemas than single-point crossover.
- Exchanged segments do not necessarily contain endpoints.
- Parameterized uniform crossover: exchange happens at each bit position with probability p: no positional bias but can be very disruptive of schemas.
Some encodings require specially defined crossover operators.
Evolving crossover "hot spots" (Schaffer and Morishima)
- Evolve the points at which crossover can occur (inspired by real evolution)
- Each genome has an associated crossover template specifying where multiple-point crossover can take place
- Crossover points get inherited along with the bits to which they are attached
```
Parents      1  0  0  1! 1  1  1! 1
             0  0  0  0  0  0! 0  0

Offspring    1  0  0  1! 0  0! 1! 0
             0  0  0  0  1  1  0  1
```
- Mutation operates on both genomes and crossover templates, but only genome itself is used to determine fitness
- As search continues, number of crossover points increases rapidly, though number of productive crossover operations does not.
Linkage Evolving Genetic Operator (Smith and Fogarty)
- Each genome specify vectors of left and right linking flags for each gene, as well as the genome itself.
- The linking flags define linked blocks of adjacent genes, the units of recombination.
- Creation of a new individual
  - Starting from the left end of the genome, a parent is selected from the whole population using the selection operator.
  - The first linked block of genes is copied to the new individual from the parent.
  - Until the end of the genome is reached, a member of the sub-population with a block starting at the current locus is selected using the selection operator, and the next block is copied from it to the new individual.
- Mutation affects linking flags as well as alleles in genome.

Other operators and parameters

Crowding: newly formed offspring replaces the individual most similar to itself.
Fitness sharing: individuals similar to many others punished, individuals that are different are rewarded; can induce "speciation".
Mating constraints
- Prohibit mating between similar individuals.
- Only permit mating between similar individuals.
- Genomes include mating tags which identify prospective mates; evolve along with rest of genome, implementing appropriate restrictions on mating.
- Spatially restricted mating: individuals more likely to mate with those in their neighborhoods; tends to preserve diversity.
Parameters
- Optimize the parameters using a separate meta-level GA; the fitness of a parameter individual is a function of the performance of the ordinary GA using the parameters encoded by that individual.
- In the evolution strategies framework
  - Different strategies may be appropriate in different parts of the search space
  - No crossover
  - Mutation standard deviations (σ) for specific features (alleles) are part of the genome
  - Variance "rotation parameters" (α) for pairs of specific features are part of the genome
  - Both the σs and the αs are mutated along with the rest of the genome