Deriving spiking voltage and spiking frequency of a neuron from a train nengo_dl model

Hello nengo_dl community,

I hope you all are doing well.
I would like to extract parameters from a trained nengo_dl model. I can extract the weights of the model but I am more interested in the parameters like the spiking threshold of a neuron and the frequency at which it is spiking? What I mean that how can I know what is a spiking threshold of a neuron and at what frequency it is spiking? Is there any way to extract or estimate them?

Thank you in advance for your reply.

Hi @Choozi,

The firing threshold of a neuron is typically defined by the bias current provided to the neuron when the network is running. The bias current is in turn determined by the bias weights, which are probably one of the values extracted as part of all of the network weights. Note that the bias weights can interact with the other connection weights to change the neuron spiking threshold, so determining the exact firing threshold may not be straightforward.

As a side note, I should clarify that for any neuron of a single type, the spiking threshold is always the same: when the input current slightly exceeds the spiking threshold current. The “difference” in spiking thresholds for neurons in a network can be attributed to the bias currents fed to the neurons when the network is in a resting (no input) state.

In regards to the spiking frequency, that information can be obtained with the use of nengo.Probes. This NengoDL example has example code on how to do this. Note that if the inputs to your networks change over time, so will the spiking frequency of the neurons in the network.

@xchoo Thank you for your prompt and detailed reply.

Let me clarify myself with an example. Consider we have a single LIF neuron that is taking only single input. Then we have one weight connection and one bias weight. The firing threshold is determined with the bias weight right?
Let suppose the bias weight is not interacting with other connection weights. How can I then interpret or determine the spiking threshold from bias weight of let say 0.6?

Note that the bias weights can interact with the other connection weights to change the neuron spiking threshold, so determining the exact firing threshold may not be straightforward.

Alright. Can we determine at least the range of the spiking threshold i.e the maximum spiking threshold it can acheive?

As a side note, I should clarify that for any neuron of a single type, the spiking threshold is always the same

Sorry, this is confusing me. I am using LIF neuron, does this mean the spiking threshold is predefined or what?

That is correct (i.e., the bias weight does play a role in the firing threshold), but it’s not the full answer. I got into more depth further down this post.

If you are using Nengo, and the nengo.LIF neuron (i.e., the neuron has a range of -1 to 1), and the neuron has a gain of 1, that would be equivalent to an intercept of 0.4. You can test this out with the following code:

import matplotlib.pyplot as plt
import nengo
from nengo.utils.ensemble import response_curves

with nengo.Network() as model:
    ens = nengo.Ensemble(1, 1, gain=[1], bias=[0.6])

with nengo.Simulator(model) as sim:
    eval_points, activities = response_curves(ens, sim)

plt.figure()
plt.plot(eval_points, activities)
plt.show()

This results in the following response curve:

Note that changing the gain of the neuron will affect the x value at which the neuron starts firing (I assume this point is what you are referring to when you say “spiking threshold”)

To understand why this is the case, we need to look at the LIF equation. In Nengo, the LIF equation is implemented as follows (a(x) is the activity of the neuron for a given input x):

a(x) = \frac{1}{\tau_{ref}-\tau_{RC}\text{ln}(1-\frac{J_{th}}{\alpha x + J_{bias}})}

From the equation, the point at which the neuron starts firing (the x intercept, or x_{int} is when the term inside the natural log goes just a bit above 0 (for values lesser than or equal to 0, this term is undefined). I.e., the neuron starts firing when 1 - \frac{J_{th}}{\alpha x + J_{bias}} > 0. Rearranging the terms, we get:

x_{int} > \frac{J_{th} - J_{bias}}{\alpha}

In Nengo, we use J_{th} = 1 for simplicity, so this equation becomes:

x_{int} > \frac{1 - J_{bias}}{\alpha}

If we substitute a gain value of 1 (i.e., \alpha = 1), and a bias weight of 0.6 (as per your question), we see that the x intercept works out to be 0.4.

What I meant by my statement is that for an LIF neuron on it’s own (no input weight, no bias weight), the spiking threshold is always when the input current just exceeds the firing threshold current (J_{th}). This value is the same for all LIF neurons under the same conditions (no input weight, no bias weight). In Nengo, heterogenous neuron response curves are generated by randomizing the neuron gains and biases.