More on Connectionist Structure

Inference Using RAAM

• Representations from RAAM network as inputs to a separate auxiliary feedforward network which transforms and analyzes them (Chalmers, Blank et al.)
• Combining RAAM and auxiliary network (Chrisman)
  • Two RAAM networks share a single hidden layer
  • Subtree training on separate networks
  • Training on "inference" task with entire trees
  • Applied to translation: hidden layer as "interlingua"

Convolution Associative Memory (Role Binding)

• Encoding (binding): 2 item vectors (e.g., role, filler) -> memory trace
• Decoding (unbinding): memory trace and single item (cue) -> item associated with cue
• Composition: combining associations in a single trace
• Capacity: number of associations that can be represented
• Tensor product approach (Smolensky)
  • Encoding (binding): tensor product (generalized outer product)
  • Decoding (unbinding): inner product of cue and trace
  • Composition: tensor addition
  • Size of trace increases exponentially with the depth of the structure being represented
• Holographic reduced representations (Plate)
  • Encoding (binding): circular convolution
  • Decoding (unbinding): circular correlation

    

  • Composition: vector addition
  • Size of trace is constant as depth of structure increases
  • For correlation to decode convolution, elements of each vector must be independently distributed with mean zero and variance 1/n
  • Convolution trace stores pairs only with enough information to discriminate item from others; a cleanup memory (e.g., a Hopfield net) is required if an accurate output is needed
  • (Some problems: types and tokens, microfeature representation)

Take me back to the C661 Home Page.

Last updated: 30 October 1995
URL: http://www.cs.indiana.edu/classes/c661/cx_struc3.html
Comments: gasser@salsa.indiana.edu
Copyright 1995, The Trustees of Indiana University