Custom Hebbian Learning Rule Confusion

[n299]

Hi!
I’m trying to translate a Python (non-Nengo) script I have into Nengo, and the learning rule in my model is a modified STDP rule, which is basically one line of Python code:

W1 = W1 + dt * (np.matmul(eta_learn * Ret["H"], np.transpose(LGN["H"]))) 

where W1 is a 1600 x 512 matrix of weight values (first layer is 1600 neurons, second layer is 512 neurons, but obviously this can be generalized for two layers of any number of neurons), eta_learn is .1, and Ret["H"] and LGN["H"] are 1600x1 and 512x1 vectors indicating the spiked neurons (1 if spiked, 0 if not). I’m looking at Nengo learning rules to figure out where the update is happening, and it seems to be in the “step_sim” function, but I want to know what “post_filtered” and “pre_filtered” mean. Is there a way to access the “spiked” values of two layers being connected via a learning rule so that I can use them to update the weights? Also, is what is being returned in the below rule what is being added to the weight matrix? Also, for the below rule, where is alpha being determined? Is it when you call the rule when you build a model?

The Nengo learning rule I am referencing:

def step_simoja():
    # perform forgetting
    post_squared = alpha * post_filtered * post_filtered
    delta[...] = -beta * weights * post_squared[:, None]

    # perform update
    delta[...] += np.outer(alpha * post_filtered, pre_filtered)

return step_simoja

Thank you! Any information helps, even if its just where the weights are being changed in the documentation.

[xchoo]

Hi @n299! You packed quite a few questions in your post, so below I’ll try to address them individually (and slightly out of order, but hopefully it’ll make sense to you).

I’m looking at Nengo learning rules to figure out where the update is happening …

It’s awesome that you are looking at the Nengo code base to learn how it is doing things. Here’s a quick rundown of what’s happening when you specify a learning rule and build a nengo.Simulator object:

1. Nengo iterates through all of the objects in your model and runs each one through the Nengo builder (nengo.builder.builder.Model.build(...))
2. If you have specified a learning rule on a connection in your model, it calls the build method on the learning rule.
3. For each learning rule type defined in Nengo, there are associated registered builder functions. It is this function that gets called by the build method from the previous step. There is also a registered builder function generic to all learning rules.
4. The “job” of each builder function is to construct an operator graph and add it to existing operator graph for the Nengo model. For learning rules, you’ll notice that most of them add a Sim<Name> operator. These operators are in turned defined in builder.learning_rules.
5. When an operator is added to the Nengo model, it’s make_step function is called to determine what computation should be added.
6. Once the make_step function has been processed, the builder continues on to the next Nengo object in the model.

Right, with that “quick” overview done, let’s get to answering your questions!

==========

I’m looking at Nengo learning rules to figure out where the update is happening, and it seems to be in the “step_sim” function, but I want to know what “post_filtered” and “pre_filtered” mean.

Also, is what is being returned in the below rule what is being added to the weight matrix

Well, sort of. For learning rules, the update is sort of happening in multiple places. The code to calculate the exact change in values (I’m being expressly vague here and not saying “change in weights” for a reason) caused by the learning rule is done in the step_<name> function. You are correct in that respect. Using the step_simoja function as an example, the value changes are applied to the delta nengo.Signal object. This signal itself is created in the generic learning rule builder function, and passed to the SimOja class when it is created.

So, the step_simoja call updates the value of the delta signal, but this has not yet updated any weights in the model. The weight updates are done by an operator added by the generic learning rule builder, where target is defined according to what the learning rule affects.

To summarize, the updates for a learning rule are performed in two steps:

1. The step_sim<name> function (from the Sim<Name> operator) computes the change in values and updates the delta signal.
2. The Copy operator copies the value of the delta signal to the target signal (which happens to be the connection weights for the Oja rule.

==========

but I want to know what “post_filtered” and “pre_filtered” mean

Documentation for the SimOja learning rule operator can be found here. To answer your specific question about pre/post_filtered, these variables are the nengo.Signals that represent, respectively, the spike trains from the pre and post ensembles of the learning rule, filtered by any synapses defined for these spike trains.

==========

Is there a way to access the “spiked” values of two layers being connected via a learning rule so that I can use them to update the weights?

Yes! If you define the Oja learning rule with synapse values of None, this will leave the pre and post spike trains unfiltered, so you get the raw spiking data. You can do this with:

nengo.Connection(pre, post, 
                 learning_rule_type=Oja(..., pre_synapse=None, post_synapse=None))

Note that the spike data will still be accessible to the learning rule step function through pre/post_filtered. There is no special variable used to denote non-filtered data specifically. Additionally, the spike data will be in a format where spikes are represented with a value that is 1/dt. This is done to keep the area of a single spike to 1 (1/dt * dt = 1).

==========

Also, for the below rule, where is alpha being determined? Is it when you call the rule when you build a model?

For the Oja rule in particular, alpha is computed when the make_step function is called (see Step 5 in the quick rundown above). And you are correct, this is done when nengo.Simulator(model) is called to build your Nengo model.

[n299]

Thank you! I now understand where in the learning rule the change is being made to the weights, and what pre and post_filtered are.

Additionally, the spike data will be in a format where spikes are represented with a value that is 1/dt . This is done to keep the area of a single spike to 1 ( 1/dt * dt = 1 ).

Could you clarify this a bit more? If dt = .001, which I think is the default in Nengo, then each spike would be 1000. In this rule, alpha is learning_rate * dt, so wouldn’t this mean that
delta[...] += np.outer(alpha * post_filtered, pre_filtered) would result in a single time-step’s change in delta that is essentially
.001 * learning_rate * 1000 * 1000 = learning_rate * 1000 for a weight connecting two spiking neurons?
This seems to increase the weights way too much (like, by 1 or 2 factors of ten, depending on learning_rate).

It doesn’t seem right that I would have to change the learning rate depending on what value I choose for dt.

[xchoo]

Could you clarify this a bit more? If dt = .001 , which I think is the default in Nengo, then each spike would be 1000.

That is correct. However, as for the impact it has on the outcome of the learning rule, the current code seems to do what we expect it to do. In fact, we have a test (which is run every time a change is made to the code base) to ensure that we haven’t accidentally introduced a dependency on dt into the code.

There is the caveat that this test is only testing on learning rules with filtered signals (i.e., not using direct spikes), so it may be the case that with pure spikes, a different formulation may need to be used. In discussions with the other Nengo developers, we think that we have not yet run into an issue with using spikes because the additional dt factor will only occur if you get a pre and post spike occurring at the same time. And the spike trains are sparse enough that it doesn’t seem to happen often, so this effect may be masked.

I’ll need to perform some experimentation to investigate this, and I’ll keep you updated.

[ntquyen2]

@xchoo

l. The weight updates are done by an operator added by the generic learning rule builder, where target is defined according to what the learning rule affects.

I see, but I am not clear about how the sum of the delta is calculated if the same connection has multiple learning rule types? According to my knowledge, add_op function only add an operator to the existing operator graph in a model. And copy function is used to in add_op function aims this as you said before

@xchoo

The Copy operator copies the value of the delta signal to the target signal (which happens to be the con!

THis is add_op function in def build_learning_rule(model, rule):

[xchoo]

@ntquyen2

I see, but I am not clear about how the sum of the delta is calculated if the same connection has multiple learning rule types?

I may be wrong, but I believe a connection can only have one learning rule type associated with it. Assigning a different learning rule type to a connection after one has already been assigned overwrites the first assignment.