The learning rules included in the default Nengo installation use the difference in neural activity to calculate the weight update. The neural activity is filtered by the pre_synapse
parameter before being fed to the learning rule. I suppose that if you set pre_synapse=None
when creating the learning rule, it will only update the weights when a spike occurs. You can try doing this for your code and see if that modifies the behaviour in any way?
I tried it with the same example posted above, I see a little improvement based on the heatmap compared to the previous one, but I still can’t see the effect of the inhibition despite setting the inhibition value to something higher like -10, i don’t know maybe I am doing something wrong or the BCM rule doesn’t work that way with inhibition?
Just to clarify, do you see the effect of the lateral inhibition when you use something like the PES learning rule?
Yes, it does work with the PES rule (clear when using -10 as inhibition as shown below).
and for the STDP (using -2 for the inhibition as shown below).
Also by using the pre_synapse=None
now I can see different variations of the two numbers (7 and 9) being learned and without it all the neurons learn the samething.
without pre_synapse=None
with pre_synapse=None
So i think you are right about the pre_synapse filter and weights update
Your results look promising!
I’m still in the process of cleaning up the STDP code that was sent to me (I’ve been working on another problem), and I hope to have it posted here mid next week.
As for your question about the BCM rule and the lateral inhibition, I will have to default to @tbekolay’s opinion on this. I’m not super familiar with that rule, and he’s the original author of Nengo’s implementation of it.
I’ve finally cleaned up the code that was sent to me, and I’ve uploaded it here.
In this folder, there are are two python files: stdp.py
and example.py
.
stdp.py
is the python module containing the custom STDP learning rules. Two learning rules are included, an STDP learning rule, and a STDP triplet learning rule.
example.py
is an example Nengo model demonstrating how to use the STDP learning rule in a simple 2 neuron network. See extending-nengo.zip
for an example on how to use the triple STDP learning rule.
I’ve also included the original code that was sent to me (extending-nengo.zip
). In this zip file, you will find more examples and analysis of the various STDP learning rules (including the triplet rule). Inside the zip file, Learning.ipynb
contains a very detailed description of the derivation of the STDP learning rules.
I was wondering how do we characterize an arbitrary learning rule and compare it with biological data(The usual plots of STDP curve that we see everywhere and in the learning.ipynb).
From what’s in the notebook, it appears that we just take 2 neurons (Doublet rule) and connect it via a connection. But there is no signal transmission happening from pre to post neuron(transform = [0]). Is this the standard practice or do we extract data from a network of neuron in which transmission occurs(transform != 0) with weight update?
@tbekolay might be able to elaborate on this further, but it is my understanding that the STDP learning rule merely increases or decreases the value of the connection weight between the two neurons depending on the spike timing between the two neurons. In the example notebook, the weights were initialized to 0 to make this effect more clear (an increase or decrease from 0 would mean the STDP rule is working), however, there is no restriction on what the initial weights have to be initialized to.
Yes, that’s correct. Additionally, starting with a 0 weight means that the presynaptic spike didn’t cause a postsynaptic spike, which makes the plot easier to understand (though if you continued the experiment for a long time, the presynaptic spike would eventually cause postsynaptic spikes).
Alright. I had this doubt because in the notebook posted here, the transform applied in connection is [0] and therefore I don’t think pre neuron would ever lead to post neuron spike.
Yes, we artificially stimulate the pre/post neurons using a Node
to ensure their exact spike timing. This is analogous to the classical patch clamping experiments that discovered STDP.
Hello! Where exactly are you using the PES rule. I am also trying to classify MNIST using unsupervised learning. As far as I understand, the learning is happening in weights connecting the input_layer and layer_1 through the BCM rule. I don’t see an error propagation during unsupervised learning. Can you help?
Hi @nikhilgarg ! I think the term error propagation is something related to supervised learning, since in unsupervised learning you basically have local learning rule updating the synapses weight based on the activity of the pre and post neuron. For the learning rule i use the STDP for that, the learning rules used in this topic were basically for the inhibition issue i had, so didn’t really used BCM or PES for learning.
A combination of STDP and lateral inhibition should give you a good result.
Okay but if there is no learning signals, how would we know that which output neuron corresponds to which class. Do we label output neuron based on their activity while we present training data?
Yes, that’s exactly what i used to do but in testing phase, you have basically two approaches : you set the labels at the end of testing phase based on either Max spikes or First spike and compute at the end the accuracy.
I had a doubt regarding implementing lateral inhibition.
inhib = nengo.Connection(
layer1.neurons,
layer1.neurons,
transform=inhib_wegihts)
What I instead wish to do, is to clamp the voltages of all other neurons to zero for 20ms. Any clue how can I do this?
You can emulate this by applying a synaptic filter on the lateral inhibition connection. Try a synapse of about 60ms to 80ms:
inhib = nengo.Connection(
layer1.neurons,
layer1.neurons,
transform=inhib_wegihts,
synapse=0.06)
Note that you may have to play with the inhibition weights to get it to respond quickly, or have multiple lateral inhibition connections with different synapses to span the 20ms window.
I would test what inhibition connection parameters would work for your requirements by creating a test network with just a feed-forward inhibition network, and then probing and plotting the output to see what effects changing the inhibition weights or synapse value does on the inhibited population.
Thanks
I did not get this. What do you mean by multiple lateral inhibition.
That would be of great help. Thanks
I shall try.
If you need to, you can actually create multiple lateral inhibition connections to get weird effects. For example, you could do:
with model:
inhib = nengo.Connection(
layer1.neurons,
layer1.neurons,
transform=inhib_weights)
inhib2 = nengo.Connection(
layer1.neurons,
layer1.neurons,
transform=inhib_weights,
synapse=0.01)
And you would get sort of 2 lateral inhibition events. You’ll have to experiment to see if this is the behaviour you’ll want in your code. Although, from my own experimentation, you’ll probably only need one lateral inhibition connection.
I’ve attached some test code here: test_inhib_synapse.py (2.0 KB)
It looks like you should be able to get the 20ms inhibition delay with an inhibitory synapse of about 0.0025. The code has two ensembles, ens1
that inhibits ens2
:
inhib_weight = 2
inhib_synapse = 0.0025
sim_runtime = 0.5
with nengo.Network() as model:
in_node = nengo.Node(1)
ens1 = nengo.Ensemble(
1,
1,
intercepts=nengo.dists.Choice([0]),
max_rates=nengo.dists.Choice([8]),
# We're using a low firing rate so that the spikes are far enough apart to
# demonstrate the 20ms inhibition of the ens2 spikes.
encoders=nengo.dists.Choice([[1]]),
)
ens2 = nengo.Ensemble(100, 1)
Note that this model is a pure feed-forward network just so that the only effects you see here are the direct inhibition effects and not a lateral inhibition. However, the same parameters should work with the lateral inhibition version (you’ll need to test it though).
Here’s an example output using the code:
And the code reports an average inhibition time of 20.86ms.