Nengo, WTA and Homeostasis

Timodz · October 28, 2020, 10:35am

Hello everyone,

I am trying to implement a one-layer fully connected SNN, in order to classify MNIST digits using STDP.
I was able to load the data using a Node and add the STDP rule to Nengo and I can see that the learning is happening as intended.

Now I want to add the Lateral inhibition (for the WTA) and neuron threshold adaptation (for the Homeostasis), in order to force the neurons to specify into different classes.

So, for example, using a simple two neurons ensemble and taking only two digits from the MNIST dataset.

For the Lateral inhibition i used the following approach to do it:

inhib_wegihts = np.full((n_neurons,n_neurons),-5)

inhib = nengo.Connection(
                      layer1.neurons, 
                      layer1.neurons, 
                      transform=inhib_wegihts)

but still I am getting an identical result of the both neurons learning just one class (even when i added the number of neurons)

Preview of the weights after training (for 7 and 9):

For the threshold adaptation, I used the Adaptive LIF since (based on my understanding) it already contains an adaptive threshold based on the activity.

The full example (replaced STDP with BCM but still the same issue):

import nengo
import numpy as np
from numpy import random
from nengo_extras.data import load_mnist

#############################
# load the data
#############################

img_rows, img_cols = 28, 28
input_nbr = 1000

(image_train, label_train), (image_test, label_test) = load_mnist()

image_train = 2 * image_train - 1 # normalize to -1 to 1
image_test = 2 * image_test - 1 # normalize to -1 to 1

# select the 0s and 1s as the two classes from MNIST data

image_train_filtered = []
label_train_filtered = []

for i in range(0,input_nbr):
   if label_train[i] == 7:
        image_train_filtered.append(image_train[i])
        label_train_filtered.append(label_train[i])
      

for i in range(0,input_nbr):
    if label_train[i] == 9:
       image_train_filtered.append(image_train[i])
       label_train_filtered.append(label_train[i])


image_train_filtered = np.array(image_train_filtered)
label_train_filtered = np.array(label_train_filtered)

#############################
# Model construction
#############################

model = nengo.Network("My network")

presentation_time = 0.20 #0.35

#input layer
n_in = 784
n_neurons = 2

with model:

    # input node 
    picture = nengo.Node(nengo.processes.PresentInput(image_train_filtered, presentation_time))

    # input layer
    input_layer = nengo.Ensemble(
        784,
        1,
        label="input",
        max_rates=nengo.dists.Uniform(22, 22)
        )

    # Connection between the node and input_layer
    input_conn = nengo.Connection(picture,input_layer.neurons)

    # layer
    layer1 = nengo.Ensemble(
         n_neurons,
         1,
         neuron_type=nengo.neurons.AdaptiveLIF(),
         label="layer1",
         )

    # weights randomly initiated 
    layer1_weights = random.random((n_neurons, 784))

    # Connection between the input and layer1
    conn1 = nengo.Connection(
        input_layer.neurons, 
        layer1.neurons, 
        transform=layer1_weights,
        learning_rule_type=nengo.BCM(learning_rate=1e-4) # replaced later with the STDP rule
        )

    # Inhibition
    
    inhib_wegihts = np.full((n_neurons,n_neurons),-10)

    inhib = nengo.Connection(layer1.neurons, layer1.neurons, transform=inhib_wegihts)

with nengo.Simulator(model) as sim:
    sim.run(presentation_time * label_train_filtered.shape[0])

So coming from another SNN event-driven simulator to Nengo, my questions are:

About my use-case:
1- Is there any issue with the way the inhibition is implemented in my example?
2- for the Homeostasis, is by using the AdaptiveLIF the right way to provide it?

About Nengo:
1- Nengo simulation uses dt, is it possible to implement event-driven simulations into Nengo? or is there an event-driven mode in Nengo maybe?
2- For the inhibition issue i had, i suspected maybe the default radius in Nengo is causing a problem since its by default between -1 and 1, while i used to work with a range from 0 to 1, it is possible to set it to that range? since i tried to modify it but there are always negative values involved in the interval.

Sorry for taking so long, i hope it is clear.

Have a nice day .

xchoo · October 31, 2020, 3:17am

Hi @Timodz!

I took a quick look at your code and I’ll try to address some of your questions below.

Lateral Inhibition

Syntactically, what you are doing is the way lateral inhibition would be implemented in Nengo. That is to say, to implement an inhibitory connection where the inputs come from a population of neurons, and inhibits the same population of neurons, you’ll want a recurrent connection like so:

nengo.Connection(layer1.neurons, layer1.neurons, transform=inhib_weights)

However, I believe that you have made a slight error when defining the inhibitory weights. You used np.full(...) to generate the weights, which creates a lateral inhibitory weights, as well as recurrent inhibitory weights. In effect, the neurons are inhibiting themselves, and I would be surprised if the neurons even fired at all.

For lateral inhibition, I believe the connection weights you want would be:

inhib_wegihts = (np.full((n_neurons, n_neurons), 1) - np.eye(n_neurons)) * -2

I used the -2 factor above as the minimum value (I’ll elaborate more on this below) you’ll need for an inhibitory connection, but you can increase or decrease this value to change the strength of the inhibitory connection.

Homeostatis

This question really depends on what you want to achieve with “homeostatis”. If you want to achieve a sort of normalization of neural activity across the whole ensemble, then using a homeostatic learning rule might be the way to go (I believe the Oja rule does homeostatis, but I’ll have to double check on that). But, if you just want neural activity that falls to a baseline value after some period of spiking, then the ALIF neuron is an appropriate use.

Event-based Simulations

I’m not 100% certain on this (I’ll message the devs and get back to you shortly), but I don’t believe there is an event-based mode in Nengo (nor does it support event-based processing).

Neuron Characteristics

Technically, a neural ensemble in Nengo can represent any input from -infinity to infinity. What the radius parameter determines is the range of input values for which the encoders and decoders for the ensemble are optimized to best represent. This is part of the NEF algorithm and you can read about it here!

However, because you are using direct-to-neuron connections, the radius parameter does not affect the behaviour of your model. For direct-to-neuron connections, the parameter that determines the “range” of the neuron representation is the neuron intercept value. The intercept value determines the represented value at which the neuron starts firing (see example). If you want the neuron to operate from a range of 0 to 1** (actually infinity), I would set the neuron intercepts to intercepts=nengo.dists.Choice([0]).

A few notes about your model, by default, Nengo generates the intercept values from a random uniform distribution of -1 to 1. Thus, both your input_layer and layer1 ensembles are being generated with randomly chosen intercepts, and that might explain some of the behaviour you are seeing as well.

The randomly chosen intercept values also explain why I chose the value of -2 in the inhibitory connection above. This is to account for an intercept that is generated at -1 (i.e., the neuron is active when the input is -1 through infinity), which an input signal of 1. Thus, to fully inhibit the ensemble in this scenario we’ll need to offset the input signal by -2 to get an effective input to the neuron that is -1 (i.e., 1 + (-2) = -1).

Timodz · November 3, 2020, 8:28am

Thank you @xchoo for your clear explanation!

Lateral Inhibition

Yes indeed, I fixed it now.

Homeostatis

Yes, its a sort of normalization to prevent a neuron from dominating every time and inhibit the others. I will check the Oja rule.

Event-based Simulations

Yes, that could change everything since I am trying to implement an event-based STDP which explains also some of the behaviour i am seeing, and if this is true I would have to go with another type of STDP suitable with Nengo. Waiting for your confirmation.

After taking everything into consideration, the behaviour was always the same and in fact I had also introduced a pause between each input using a modified version of

nengo.processes.PresentInput which I didn’t mention in the previous post, when I removed that pause everything went as expected. I can see the inhibition working now.

Thank you again for the explanation and I hope to hear from you soon about the event-driven possibility.

xchoo · November 4, 2020, 5:07am

I spoke to one of the Nengo devs with a lot of experience implementing learning rules, and it seems like it is possible to implement event-based STDP in Nengo. He has provided me some code, and I’ll need to clean it up a bit before posting it here.

Here’s a sneak preview at the Jupyter notebook they included with the STPD code implementing STDP in Nengo:

Timodz · November 4, 2020, 9:49am

That looks interesting !, btw i saw somewhere in the forum (can’t find the comment) that we can have a sort of Event-based when it comes to the learning rule (something like the weights update is only done when there is an activity maybe ?).

Maybe the code you have can make it clear.

xchoo · November 5, 2020, 3:59am

The learning rules included in the default Nengo installation use the difference in neural activity to calculate the weight update. The neural activity is filtered by the pre_synapse parameter before being fed to the learning rule. I suppose that if you set pre_synapse=None when creating the learning rule, it will only update the weights when a spike occurs. You can try doing this for your code and see if that modifies the behaviour in any way?

Timodz · November 6, 2020, 7:59am

I tried it with the same example posted above, I see a little improvement based on the heatmap compared to the previous one, but I still can’t see the effect of the inhibition despite setting the inhibition value to something higher like -10, i don’t know maybe I am doing something wrong or the BCM rule doesn’t work that way with inhibition?

xchoo · November 6, 2020, 2:53pm

Just to clarify, do you see the effect of the lateral inhibition when you use something like the PES learning rule?

Timodz · November 6, 2020, 6:17pm

Yes, it does work with the PES rule (clear when using -10 as inhibition as shown below).

and for the STDP (using -2 for the inhibition as shown below).

Also by using the pre_synapse=None now I can see different variations of the two numbers (7 and 9) being learned and without it all the neurons learn the samething.

without pre_synapse=None

with pre_synapse=None

So i think you are right about the pre_synapse filter and weights update

xchoo · November 6, 2020, 6:46pm

Your results look promising!
I’m still in the process of cleaning up the STDP code that was sent to me (I’ve been working on another problem), and I hope to have it posted here mid next week.

As for your question about the BCM rule and the lateral inhibition, I will have to default to @tbekolay’s opinion on this. I’m not super familiar with that rule, and he’s the original author of Nengo’s implementation of it.

xchoo · November 13, 2020, 2:33am

I’ve finally cleaned up the code that was sent to me, and I’ve uploaded it here.
In this folder, there are are two python files: stdp.py and example.py.

stdp.py is the python module containing the custom STDP learning rules. Two learning rules are included, an STDP learning rule, and a STDP triplet learning rule.

example.py is an example Nengo model demonstrating how to use the STDP learning rule in a simple 2 neuron network. See extending-nengo.zip for an example on how to use the triple STDP learning rule.

I’ve also included the original code that was sent to me (extending-nengo.zip). In this zip file, you will find more examples and analysis of the various STDP learning rules (including the triplet rule). Inside the zip file, Learning.ipynb contains a very detailed description of the derivation of the STDP learning rules.

Timodz · November 13, 2020, 9:55am

Thank you @xchoo.

nikhilgarg · November 13, 2020, 8:32pm

I was wondering how do we characterize an arbitrary learning rule and compare it with biological data(The usual plots of STDP curve that we see everywhere and in the learning.ipynb).

From what’s in the notebook, it appears that we just take 2 neurons (Doublet rule) and connect it via a connection. But there is no signal transmission happening from pre to post neuron(transform = [0]). Is this the standard practice or do we extract data from a network of neuron in which transmission occurs(transform != 0) with weight update?

xchoo · November 13, 2020, 9:29pm

@tbekolay might be able to elaborate on this further, but it is my understanding that the STDP learning rule merely increases or decreases the value of the connection weight between the two neurons depending on the spike timing between the two neurons. In the example notebook, the weights were initialized to 0 to make this effect more clear (an increase or decrease from 0 would mean the STDP rule is working), however, there is no restriction on what the initial weights have to be initialized to.

tbekolay · November 14, 2020, 4:21pm

Yes, that’s correct. Additionally, starting with a 0 weight means that the presynaptic spike didn’t cause a postsynaptic spike, which makes the plot easier to understand (though if you continued the experiment for a long time, the presynaptic spike would eventually cause postsynaptic spikes).

nikhilgarg · November 14, 2020, 4:56pm

Alright. I had this doubt because in the notebook posted here, the transform applied in connection is [0] and therefore I don’t think pre neuron would ever lead to post neuron spike.

tbekolay · November 14, 2020, 4:58pm

Yes, we artificially stimulate the pre/post neurons using a Node to ensure their exact spike timing. This is analogous to the classical patch clamping experiments that discovered STDP.

nikhilgarg · November 15, 2020, 6:35pm

Hello! Where exactly are you using the PES rule. I am also trying to classify MNIST using unsupervised learning. As far as I understand, the learning is happening in weights connecting the input_layer and layer_1 through the BCM rule. I don’t see an error propagation during unsupervised learning. Can you help?

Timodz · November 15, 2020, 10:47pm

Hi @nikhilgarg ! I think the term error propagation is something related to supervised learning, since in unsupervised learning you basically have local learning rule updating the synapses weight based on the activity of the pre and post neuron. For the learning rule i use the STDP for that, the learning rules used in this topic were basically for the inhibition issue i had, so didn’t really used BCM or PES for learning.

A combination of STDP and lateral inhibition should give you a good result.

nikhilgarg · November 16, 2020, 8:58am

Okay but if there is no learning signals, how would we know that which output neuron corresponds to which class. Do we label output neuron based on their activity while we present training data?