Why are spikes divided by dt in LIF implementation?

Deep Learning
1

Hello, guys

I’m optimizing an SNN using nengoDL and the TensorFlow approach and the Adam alg. In the training phase I understand that the algorithm is using SoftLifRate for finding the optimal weights. And in the predict phase it is using the LIF neuron itself.

I have read the algorithm of LIF implemented in Nengo, and I got two doubts.

1- In the following part of the algorithm the output of the neurons that spike is divided by dt and multiplied by the amplitude. I do not understand why

    # determine which neurons spiked (set them to 1/dt, else 0)
    spiked_mask = voltage > 1
    output[:] = spiked_mask * (self.amplitude / dt)

2 - In my code I have 3 layers (inp, layer A, layer B, and output), kinda simular to the example in: Optimizing a spiking neural network — NengoDL 3.6.1.dev0 docs, and I have already trained, so I’m in predict part.

I have read that if I add a probe in the output layer with a synapse (filter), the spiking train of the output will be smoothed and then multiplied by the optimal weights finding during train. So I’m assuming this is the output of network over time.

My doubt is: Are the output of the hidden layers also smoothed by the synapse filter ?
If they are not smoothed, then the output from a hidden layer to a neuron_i that spiked is: (amplitude/dt) * weights?

Thank you so much

2

Dividing by dt gives the spikes an integral of one; this has the nice feature that if you pass a spike train through a lowpass filter, the result will be an estimate of the spike rate.

The amplitude is an easy way to scale the tuning curve of a neuron. This is often useful when training ML networks, since you can scale the LIF curve to be similar to e.g. a sigmoid curve, so that you can keep parameter initializations the same. If you’re worried about bio-plausibility, the amplitude can always be folded into the weights of a subsequent layer (since all neurons in a layer have the same amplitude).

Synapses apply to whatever connection/probe you put them on. By default, connections have a Lowpass(0.005) synapse, and probes have none. If you set a synapse to None, then any spikes passing through that connection will be unfiltered (as you described).

1 Like
3

Thank you for your quick answer. I understood now