Connect EnsembleArray neuron_output to Node

General Discussion
1

I’ve started working on implementing something close to a Hopfield NN in Nengo and was struggling with a few points:

  • I was thinking of representing my inputs as n-dimensional vectors whose elements are 0 or 1.
    Is this a good idea?
    Is the best way to do this by using something like EnsembleArray(n_neurons=10,n_ensembles=dimensionality)?

  • As I’m planning on using a custom Hebbian learning rule that takes the pre and post neural activities, I’d like to have direct access to these in my learning Node() object.
    I’ve previously achieved this with Ensemble() by:

    pre_nrn = 10
    post_nrn=10    

    pre = nengo.Ensemble(
            n_neurons=pre_nrn,
            dimensions=1,
            encoders=generate_encoders( pre_nrn ),
            # intercepts=[ 0.1 ] * pre_nrn,
            label="Pre"
            )
    learn = nengo.Node(
            output=memr_arr,
            size_in=pre_nrn + post_nrn,
            size_out=post_nrn,
            label="Learn"
            )

    
    nengo.Connection( pre.neurons, learn[ :pre_nrn ], synapse=0.005 )
    nengo.Connection( learn, post.neurons, synapse=None )

I’m currently trying the following to achieve an equivalent effect with an EnsembleArray() for an input vector of 3 elements, but I’m getting the error
nengo.exceptions.ValidationError: init: Shape of initial value () does not match expected shape (10, 30):

    pre_nrn = 10
    post_nrn=10  
    dimensionality = 3

    pre = EnsembleArray(
            n_neurons=pre_nrn,
            n_ensembles=dimensionality,
            label="Pre"
            )
    pre.add_neuron_output()
    learn = nengo.Node(
            output=memr_arr,
            size_in=(pre_nrn + post_nrn) * dimensionality,
            size_out=post_nrn * dimensionality,
            label="Learn"
            )
    nengo.Connection( pre.neuron_output, learn[ :pre_nrn ], synapse=0.005 )

I would have expected pre.neuron_output to have shape (10,3), what am I doing wrong?

2

You might find this example helpful, showing one way to implement a Hopfield network in Nengo https://github.com/s72sue/std_neural_nets/blob/master/hopfield_network.ipynb. You don’t have to do it the same way of course, but that’d be a good starting point!

The neuron outputs are all concatenated together (the idea is that EnsembleArray acts kind of like an Ensemble, so you can just think of it as a big group of neurons). So the output will have shape (30,), not (10, 3).

3

Thanks!
But my learn node should already have correct input_size as size_in=(pre_nrn + post_nrn) * dimensionality = (10+10)*3 = 60 gives space for inputs from both pre and post. Why is the ValidationError being thrown?

4

You’re connecting to learn[ :pre_nrn ], where pre_nrn=10, so you’re only connecting to the first 10 elements of learn.

5

Correct, thanks!

The example you posted is certainly interesting but I have two questions:

  • why is it using Sigmoid as a neural model and not LIF? Should it work anyways in the latter case?
  • the model seems to actually be broken because it seems to always converge to the pattern [1,1,1,1] instead of the claimed [0,1,0,1]
    image
    image1348×942 75.2 KB

    or to [0,0,0,0] in Nengo GUI
    image
    image1348×1196 46.7 KB

The problem seems to lie in the response curves, as they should be

image
image726×414 28.8 KB
but are
image
image726×414 32 KB
in Nengo 3.0 .

6

Ah good catch. That example is quite old, and hasn’t been updated to more recent versions of Nengo (we’ve got plans in the works to do a big rework of all the examples). If you change it to

ens = nengo.Ensemble(...
                     intercepts=Uniform(0,0),
                     ...)

then you should get the same response curve in Nengo 3.0.

1 Like
7

Would that be equivalent to Choice([0]*N)?

8

It’d be equivalent to just Choice([0]) (that would be a better way to do it, I think that example was written before Choice existed :slight_smile: )