Gaussian noise to the convolutional layers in nengo

General Discussion
1

Hi,

Can we add gaussian noise to the convolutional layers in a spiking neural network and then use it for classification using nengo? As per my understanding, we can only access data from the convolutional layers when we use sim.predict but I want to run inference with convolution+noise. Also, adding the gaussian noise to a certain layer would require editing the data of that layer e.g. convolution_layer_data + gaussian_noise.

Regards,
Ayesha

2

Hi @ayesha.mkzhh,

I took a quick look at the NengoDL codebase (since you are asking about sim.predict, I assume you mean NengoDL?), and it seems like it may be possible to do what you want through Nengo ensembles. I’m not entirely sure how exactly you are defining your model, are you using the NengoDL converter, or have you defined your model in terms of Nengo objects?

In Nengo, nengo.Ensemble objects have a noise parameter that you can use to inject noise into the neuron currents as the simulation is running. The noise is defined as a nengo.Process, and there are a few built-in processes with different random distributions that you can use. You can also define your own process to generate the type of noise you want to inject into the neurons.

If you are using the NengoDL converter, you should be able to get access to the ensemble’s attribute with

nengo_converter.layers[<layer_obj>]._ensemble.noise

Assuming you only want to have the noise in the inference step (i.e., after training), with this method, you would train the model using the standard training method, then modify the converted network’s ensemble properties before running the inference (sim.run or sim.predict).

If you are defining your model with Nengo objects, you can access the ensemble’s noise attribute directly. Although, note that in order for the noise attribute to take effect, you’ll need to create the nengo_dl.simulator object after the change has been made. Thus, if you want to have no noise during training, and noise after training, you might need to save the network weights or set a network seed to ensure that both networks are created with identical neuron parameters.

If you want to add noise during training, and your network is defined as a TF network, you can insert an ensemble into the network like so:
This code

    # add the first convolutional layer
    x = nengo_dl.Layer(tf.keras.layers.Conv2D(filters=32, kernel_size=3))(
        inp, shape_in=(28, 28, 1)
    )
    x2 = nengo.Ensemble(x.size_out, 1, neuron_type=neuron_type)
    nengo.Connection(x, x2.neurons, synapse=None)
    x = x2.neurons

would be equivalent to this:

    # add the first convolutional layer
    x = nengo_dl.Layer(tf.keras.layers.Conv2D(filters=32, kernel_size=3))(
        inp, shape_in=(28, 28, 1)
    )
    x = nengo_dl.Layer(neuron_type)(x)
1 Like
3

Many thanks for providing such response in detail. That was really helpful.