after(eight, nine)
chase(dogs, cats)
knows(Anne, thinks(Bill, likes(Charlie, Dave)))
[number:eight next:nine]
[subject:dogs action:chase object:cats]
[subject:Anne action:knows object:[subject:Bill action:thinks object:[subject:Charlie action:likes object:Dave]]]
Problems
Implementing Symbol Systems in Neurons
There is an alternate approach
Something that's similar to the symbolic approach, but much more tied to biology
Based on vectors and functions on those vectors
Example
BLUE
$\circledast$ SQUARE + RED
$\circledast$ CIRCLE
Lots of nice properties
[number:eight next:nine]
NUMBER
$\circledast$ EIGHT + NEXT
$\circledast$ NINE
[subject:Anne action:knows object:[subject:Bill action:thinks object:[subject:Charlie action:likes object:Dave]]]
SUBJ
$\circledast$ ANNE + ACT
$\circledast$ KNOWS + OBJ
$\circledast$ (SUBJ
$\circledast$ BILL + ACT
$\circledast$ THINKS + OBJ
$\circledast$ (SUBJ
$\circledast$ CHARLIE + ACT
$\circledast$ LIKES + OBJ
$\circledast$ DAVE))
RED
is similar to PINK
then RED
$\circledast$ CIRCLE
is similar to PINK
$\circledast$ CIRCLE
But rather complicated
import nengo
import nengo.spa as spa
D=64
model = spa.SPA(label='Binding')
with model:
model.a = spa.Buffer(D)
model.b = spa.Buffer(D)
model.c = spa.Buffer(D)
model.q = spa.Buffer(D)
model.r = spa.Buffer(D)
model.cortical = spa.Cortical(spa.Actions(
'c = a*b',
'c = c',
'r = c*~q'), synapse=0.1)
nengo.Probe(model.a.state.output)
nengo.Probe(model.b.state.output)
nengo.Probe(model.c.state.output)
nengo.Probe(model.q.state.output)
nengo.Probe(model.r.state.output)
This is not an actual question on the test
How can we model people doing this task?
A fair number of different attempts
Does this vector approach offer an alternative?
First we need to represent the different patterns as a vector
How do we represent a picture?
SHAPE
$\circledast$ ARROW + NUMBER
$\circledast$ ONE + DIRECTION
$\circledast$ UP
We have shown that it's possible to build these sorts of representations up directly from visual stimuli
The memory of the list is built up by using a basal ganglia action selection system to control feeding values into an integrator
The same system can be used to do a version of the Raven's Matrices task
S1 = ONE
$\circledast$ P1
S2 = ONE
$\circledast$ P1 + ONE
$\circledast$ P2
S3 = ONE
$\circledast$ P1 + ONE
$\circledast$ P2 + ONE
$\circledast$ P3
S4 = FOUR
$\circledast$ P1
S5 = FOUR
$\circledast$ P1 + FOUR
$\circledast$ P2
S6 = FOUR
$\circledast$ P1 + FOUR
$\circledast$ P2 + FOUR
$\circledast$ P3
S7 = FIVE
$\circledast$ P1
S8 = FIVE
$\circledast$ P1 + FIVE
$\circledast$ P2
what is S9
?
Let's figure out what the transformation is
T1 = S2
$\circledast$ S1'
T2 = S3
$\circledast$ S2'
T3 = S5
$\circledast$ S4'
T4 = S6
$\circledast$ S5'
T5 = S8
$\circledast$ S7'
T = (T1 + T2 + T3 + T4 + T5)/5
S9 = S8
$\circledast$ T
S9 = FIVE
$\circledast$ P1 + FIVE
$\circledast$ P2 + FIVE
$\circledast$ P3
This becomes a novel way of manipulating structured information
compute functions of cortical state to determine utility of actions
basal ganglia select the action with the highest utility
thalamus has routing connections between cortical areas
good timing data
import nengo
import nengo.spa as spa
model = spa.SPA(label="SPA1")
with model:
model.state = spa.Buffer(16)
model.motor = spa.Buffer(16)
actions = spa.Actions(
'dot(state, DOG) --> motor=BARK',
'dot(state, CAT) --> motor=MEOW',
'dot(state, RAT) --> motor=SQUEAK',
'dot(state, COW) --> motor=MOO',
)
model.bg = spa.BasalGanglia(actions)
model.thalamus = spa.Thalamus(model.bg)
nengo.Probe(model.state.state.output)
nengo.Probe(model.motor.state.output)
nengo.Probe(model.bg.input)
nengo.Probe(model.thalamus.actions.output)
import nengo
import nengo.spa as spa
model = spa.SPA(label="SPA2")
with model:
model.state = spa.Buffer(16)
actions = spa.Actions(
'dot(state, A) --> state=B',
'dot(state, B) --> state=C',
'dot(state, C) --> state=D',
'dot(state, D) --> state=E',
'dot(state, E) --> state=A',
)
model.bg = spa.BasalGanglia(actions)
model.thalamus = spa.Thalamus(model.bg)
nengo.Probe(model.state.state.output)
nengo.Probe(model.bg.input)
nengo.Probe(model.thalamus.actions.output)
import nengo
import nengo.spa as spa
model = spa.SPA(label="SPA1")
with model:
model.state = spa.Buffer(16)
actions = spa.Actions(
'dot(state, A) --> state=B',
'dot(state, B) --> state=C',
'dot(state, C) --> state=D',
'dot(state, D) --> state=E',
'dot(state, E) --> state=A',
)
model.bg = spa.BasalGanglia(actions)
model.thalamus = spa.Thalamus(model.bg)
def state_in(t):
if t<0.1:
return 'C'
else:
return '0'
model.input = spa.Input(state=state_in)
nengo.Probe(model.state.state.output)
nengo.Probe(model.bg.input)
nengo.Probe(model.thalamus.actions.output)
import nengo
import nengo.spa as spa
model = spa.SPA(label="SPA1")
with model:
model.vision = spa.Buffer(16)
model.state = spa.Buffer(16)
actions = spa.Actions(
'dot(vision, A+B+C+D+E) --> state=vision',
'dot(state, A) --> state=B',
'dot(state, B) --> state=C',
'dot(state, C) --> state=D',
'dot(state, D) --> state=E',
'dot(state, E) --> state=A',
)
model.bg = spa.BasalGanglia(actions)
model.thalamus = spa.Thalamus(model.bg)
def vision_in(t):
if t<0.1:
return 'C'
else:
return '0'
model.input = spa.Input(vision=vision_in)
nengo.Probe(model.state.state.output)
nengo.Probe(model.vision.state.output)
nengo.Probe(model.bg.input)
nengo.Probe(model.thalamus.actions.output)