Intelligent action

A repertoire of behaviors
Redundancy
- Solving the inverse kinematics problem
- Multiple ways to achieve the same result (degrees of freedom in the effector > degrees of freedom in the world) makes the system robust and flexible
- But this makes the mapping not a function, so hard to learn
Coupling with sensory systems and environment

Intelligent senses

Lots of information on the input end
- Crucial when the environment is unpredictable, when it is impossible to "know" beforehand what will matter
Less information on the output end
- What perception is for
- But how does perception "figure out" how to do this?
Attention, focus
- Another way to solve the problem of information overload
- Requires coupling between motor and sensory systems
Redundancy
- When the signal in one system is degraded or unreliable, the other can take over.
- When two or more systems agree, the pattern is likely to be something other than noise. We can measure quantities like correlation or mutual information to extract useful information.
Kinds of information about environment
- What things are out there (in terms of useful categories)
- Where the things are
- How the things are related to one another
Tradeoffs with different sensory systems
- Good at WHERE
- Good at WHAT: a rich range of feature detectors
- Good at focusing
- Good at a wide range of distances
- Not obscured by barriers
- Associated closely with effectors

Associates percepts with actions
A lookup table
- Finite set of sensory/perceptual inputs, finite set of action outputs
- Hard-wired or learned
- Does not permit generalization to novel inputs
- Fails to handle infinite or very large input or output spaces
A neural network
- Distributed representation of sensory/perceptual inputs
- Hard-wired or learned
- Permits generalization: interpolation between familiar inputs
- Can handle potentially infinite input or output spaces

Adaptation: changing behavior in response to feedback
Evolution
- Genetic "memory": property of a whole population of agents
- Genetic memory changes slowly as individual agents with different properties succeed or fail
Learning
- Long-term memory of each agent has an initial inherited (evolved) state
- The memory changes through the lifetime of the agent in response to regularities in the environment and the consequences of its actions
- Types of learning
  - Supervised learning
    - There is an external teacher that provides information about the appropriate response.
  - Reinforcement learning
    - The environment provides feedback sometimes about the goodness of the response (but not information about what the response should have been or when the mistake may have occurred).
  - Unsupervised learning
    - The environment provides no feedback at all because there is no real output.
    - The learner extracts regularity from input patterns by clustering them or compressing them.
  - Intelligence comes from a combination of learning types, in particular, reinforcement + unsupervised learning
- Cultural memory
  - Each generation benefits from the experience-based learning of previous generations
  - Requires a sophisticated ability to teach behaviors (supervised learning)

Evolution can take its time; learning has to take place within the lifetime of a single agent (though cultural memory provides a way around this problem).
Environments may change too rapidly for evolution to keep up; learning gives agents the flexibility to adapt to rapid change.
Learning takes time, especially in complex environments; the agent must live long enough to get intelligent.
If a lot must be learned, the agent starts out relatively helpless and requires nurturing "parents".
Genetic memory is small and can only store limited information; when complex behaviors are desirable, learning may be the only way.
Evolution and learning
- Evolving the architecture of the initial neural network and the interfaces to sensory and motor systems
- Evolving a learning rule

A neural network associating distributed representations of sensory input with locally represented actions
The weights are either hard-wired, learned, or evolved.
Decision about an action can be deterministic (the most activated output unit) or stochastic, using the Luce choice rule:
```
P(i) = f(x_i) / ∑_j f(x_j)
```