NengoDL simulator provides different results in training network

General Discussion
1

Hello everyone,

I have a question about optimizing neuron elements using NengoDL. According to the NengoDL documentation(NengoDL Simulator), by default, NengoDL will optimize the following elements in a model:

  1. Connection weights (neuron–neuron weight matrices or decoders)
  2. Ensemble encoders
  3. Neuron biases

In order to verify the optimized parameters during training, the documentation basically proposes two methods, using the sim.data[ensamble] and sim.get_nengo_params([ensamble]) functions. These functions return various data from the ensemble such as: Encoders, Intercepts, Max Rates, Scaled Enconders, Gain, Bias.

However, when optimize elements using sim.fit function in NengoDL, and seeking to verify/save the optimized elements before and after training using the above functions, I could verify that in addition to the parameters that should be trained by default (connection weights, encoders, bias) the intercepts and max rates elements are also optimized/modified during optimization, is this correct? Also, the gain element checked in the sim.data[ensamble] function is optimized and in the sim.get_nengo_params([ensamble]) function it is not, the opposite also happens with the encoders element, and I get confused which function returns the correct values. This way, I would like to know if this is correct and what is the best function to extract the optimized elements.

To understand a little better the problem I’m trying to solve, see the discussion Import sensor dataset

Any comments/suggestions are welcome!

Thanks!

2

Hi @jone,

Could you provide some source code demonstrating the issue you are experiencing? This will help with the debugging process and assist in identifying if it is an issue with the model itself, or maybe a misunderstanding of how NengoDL is working, or, maybe it’s a bug within NengoDL itself.

3

Thanks for the reply @xchoo

Yes!! I will provide a demonstrating source code and response of the network.

This is a very simple code, but it is already possible to visualize the problems I mentioned and the same happens if we add more neurons in the layers. In this case, the input node of the neural network is the derivative of a force waveform from an FSR sensor. Also, the metrics resulting from this simple code are very good, but I can’t understand how NengoDL works internally with ensemble parameters!

Code:

import warnings
import os
import matplotlib.pyplot as plt
import nengo
import numpy as np
import tensorflow as tf
import nengo_dl
from nengo.dists import Choice
from nengo.dists import Uniform
import collections
import datetime

# Parameters simulation
sim_run = 80                              # Time simulation
s_time = 0.005                            # Sample time
s_freq = 200                              # Sample frequency

# Import train data
data = np.genfromtxt(
    'C:\\localdata\\traindata.txt', 
    delimiter=' ')

f = data.T[0, 0:40000]                    # FSR
slip = data.T[1, 0:40000]                 # Ground Truth 
f_der = data.T[3, 0:40000]                # Derivated

# Import test data
dataTest = np.genfromtxt(
    'C:\\localdata\\testdata.txt', 
    delimiter=' ')
    
f_test = dataTest.T[0, 0:40000]           # FSR
slip_test = dataTest.T[1, 0:40000]        # Ground Truth 
f_der_test = dataTest.T[3, 0:40000]       # Derivated

# General parameters from ensambles
ens_params2 = dict(
    dimensions = 1,
    neuron_type=nengo.LIF(tau_rc=0.02, tau_ref=0.002, min_voltage=0, amplitude=1),
)

warnings.simplefilter("ignore")

with nengo.Network(seed=0) as model:

    # Input node 
    inpt = nengo.Node(output=nengo.processes.PresentInput(f_der, s_time))

    # First ensemble
    SNN1 = nengo.Ensemble(1, **ens_params2)

    # Connect everything together
    conn = nengo.Connection(inpt, SNN1)
 
    # Add a probe on the input and output, in this case snn1_p is the output
    inpt_p = nengo.Probe(inpt)
    snn1_p = nengo.Probe(SNN1)
    
    # set all the connection synapses to None
    for conn in model.all_connections:
        conn.synapse = None

with model:
    # Reshape inputs train data 
    inp_dr = np.reshape(f_der,(f_der.size,1,1))
    inp_slip = np.reshape(slip,(slip.size,1,1))
    
    print('--------Shape of inputs train---------')
    print('Shape inp_dr: ' , inp_dr.shape)
    print('Shape inp_slip: ', inp_slip.shape)
    
    # create data dictionaries
    train_inputs = {inpt: inp_dr}
    train_targets = {snn1_p: inp_slip}
    
    # reshape inputs test data 
    inp_dr_test = np.reshape(f_der_test,(f_der_test.size,1,1))
    inp_slip_test = np.reshape(slip_test,(slip_test.size,1,1))
    
    print('\n')
    print('--------Shape of inputs test--------')
    print('Shape inp_dr_test: ', inp_dr_test.shape)
    print('Shape inp_slip_test: ',  inp_slip_test.shape)

    # Create test data dictionary
    # note: using snn1_p instead of snn1_p_nofilt
    test_inputs = {inpt: inp_dr_test}
    test_targets = {snn1_p: inp_slip_test}
    
    def test_mse(y_true, y_pred):
        return tf.reduce_mean(tf.square(y_pred[:, :] - y_true[:, :]))
    
    # The number of simultaneous inputs that will be passed through the network.
    minibatch_size = 32
    
with nengo_dl.Simulator(model, minibatch_size=minibatch_size, seed=0, dt=s_time) as sim:
    
    # Get SNN1 parameters with sim.data[] function before training
    snn1_data = sim.data[SNN1]
    
    # Get SNN1 parameters with sim.get_nengo_params() function before training
    paramsBeforeSNN1 = sim.get_nengo_params([SNN1])
    
    # Configure the model for training/evaluation before training
    sim.compile(loss={snn1_p: test_mse}, 
    metrics= {snn1_p: [tf.keras.metrics.Accuracy(), 
                       tf.keras.metrics.AUC(),
                       tf.keras.metrics.FalseNegatives(), 
                       tf.keras.metrics.FalsePositives(),
                       tf.keras.metrics.TrueNegatives(),
                       tf.keras.metrics.TruePositives(),
                       tf.keras.metrics.Recall(),
                       tf.keras.metrics.Precision(),
                       ]})
    # Verbosity mode. 0 = silent, 1 = progress bar, 2 = one line per epoch
    evaluate_b = sim.evaluate(test_inputs, test_targets, verbose = 0)
    
    # Run training
    # If do_training = True -> training the neural network
    # If do_training = False -> load previously trained parameters
    do_training = True
    if do_training:
        
        #### Save and epoch parameters ####
        nameNet = "1-neuronNFP-F-5ep"
        nameNetSv = "./logs/1-neuronNFP-F-5ep"
        ep =5
        
        print('\n')
        print("#### Training #### ")
        sim.compile(optimizer=tf.optimizers.Adam(0.01), loss={snn1_p: tf.losses.mse}, 
        metrics= {snn1_p: [tf.keras.metrics.Accuracy(),
                       ]})
        
        # Parameters callback tensorboard
        log_dir = "logs/fit/" + nameNet + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
        tb_keras_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
        tb_nengo_callback = nengo_dl.callbacks.NengoSummaries(log_dir, sim, [SNN1, SNN1.neurons])
        
        # Train the model
        history = sim.fit(train_inputs, train_targets, epochs=ep, callbacks=[tb_keras_callback, tb_nengo_callback])
        
        # Save the parameters to file
        sim.save_params(nameNetSv)
    
    else:
        # Load parameters
        sim.load_params(nameNetSv)
    
    # Configure the model for training/evaluation after training
    sim.compile(loss={snn1_p: test_mse}, 
    metrics= {snn1_p: [tf.keras.metrics.Accuracy(),
                       tf.keras.metrics.AUC(),
                       tf.keras.metrics.FalseNegatives(), 
                       tf.keras.metrics.FalsePositives(),
                       tf.keras.metrics.TrueNegatives(),
                       tf.keras.metrics.TruePositives(),
                       tf.keras.metrics.Recall(),
                       tf.keras.metrics.Precision(),
                       ]})
    
    # Verbosity mode. 0 = silent, 1 = progress bar, 2 = one line per epoch
    evaluate_a = sim.evaluate(test_inputs, test_targets, verbose = 0)
    
    # Get SNN1 parameters with sim.get_nengo_params() function after training
    paramsAfterSNN1 = sim.get_nengo_params([SNN1])
    
    print('#### Neural Network: ', nameNet, ' ####')
    print('\n')
    print('#### Metrics Data Before Training ####')
    print('MSE: ', evaluate_b["loss"])                 
    print('Accuracy: ', evaluate_b["probe_1_accuracy"])
    print('AUC: ', evaluate_b["probe_1_auc"])
    print('False Negatives: ', evaluate_b["probe_1_false_negatives"])
    print('False Positives: ', evaluate_b["probe_1_false_positives"])
    print('True Negatives: ', evaluate_b["probe_1_true_negatives"])
    print('True Positives: ', evaluate_b["probe_1_true_positives"])
    print('Recall: ', evaluate_b["probe_1_recall"])
    print('Precision: ', evaluate_b["probe_1_precision"])

    print('\n')
    print('#### Metrics Data After Training ####')
    print('MSE: ', evaluate_a["loss"])                 
    print('Accuracy: ', evaluate_a["probe_1_accuracy"])
    print('AUC: ', evaluate_a["probe_1_auc_1"])
    print('False Negatives: ', evaluate_a["probe_1_false_negatives_1"])
    print('False Positives: ', evaluate_a["probe_1_false_positives_1"])
    print('True Negatives: ', evaluate_a["probe_1_true_negatives_1"])
    print('True Positives: ', evaluate_a["probe_1_true_positives_1"])
    print('Recall: ', evaluate_a["probe_1_recall_1"])
    print('Precision: ', evaluate_a["probe_1_precision_1"])
    
    print('\n')
    print('################### LAYER 1 #####################')
    print('#### Parameters Layer1 Before Training (sim.data) ####')
    print('Encoders: ', snn1_data.encoders)                 
    print('Intercepts', snn1_data.intercepts)
    print('Max Rates: ', snn1_data.max_rates)
    print('Scaled Enconders ', snn1_data.scaled_encoders)
    print('Gain: ', snn1_data.gain)
    print('Bias: ', snn1_data.bias)
    
    print('\n')
    print('#### Parameters Layer1 After Training (sim.data) ####')
    print('Encoders: ', sim.data[SNN1].encoders)                 
    print('Intercepts', sim.data[SNN1].intercepts)
    print('Max Rates: ', sim.data[SNN1].max_rates)
    print('Scaled Enconders ', sim.data[SNN1].scaled_encoders)
    print('Gain: ', sim.data[SNN1].gain)
    print('Bias: ', sim.data[SNN1].bias)
    
    print('\n')
    print('#### Parameters Layer1 Before Training (get_nengo_params) ####')
    print('Encoders: ', paramsBeforeSNN1[0]['encoders'])  
    print('Intercepts: ', paramsBeforeSNN1[0]['intercepts'])
    print('Max Rates: ', paramsBeforeSNN1[0]['max_rates'])
    print('Normalize encoders: ', paramsBeforeSNN1[0]['normalize_encoders'])
    print('Gain: ', paramsBeforeSNN1[0]['gain'])
    print('Bias: ', paramsBeforeSNN1[0]['bias'])

    print('\n')
    print('#### Parameters Layer1 After Training (get_nengo_params) ####')
    print('Encoders: ', paramsAfterSNN1[0]['encoders'])  
    print('Intercepts: ', paramsAfterSNN1[0]['intercepts'])
    print('Max Rates: ', paramsAfterSNN1[0]['max_rates'])
    print('Normalize encoders: ', paramsAfterSNN1[0]['normalize_encoders'])
    print('Gain: ', paramsAfterSNN1[0]['gain'])
    print('Bias: ', paramsAfterSNN1[0]['bias'])

Response of the training:

#### Neural Network:  1-neuronNFP-F-5ep  ####


#### Metrics Data Before Training ####
MSE:  0.0018431361531838775
Accuracy:  0.992133617401123
AUC:  0.9989702701568604
False Negatives:  0.0
False Positives:  19.0
True Negatives:  9207.0
True Positives:  54.0
Recall:  1.0
Precision:  0.7397260069847107


#### Metrics Data After Training ####
MSE:  0.001441666390746832
Accuracy:  0.993534505367279
AUC:  0.9996747970581055
False Negatives:  0.0
False Positives:  6.0
True Negatives:  9220.0
True Positives:  54.0
Recall:  1.0
Precision:  0.8999999761581421

################### LAYER 1 #####################

#### Parameters Layer1 Before Training (sim.data) ####
Encoders:  [[1.]]
Intercepts [0.4569144]
Max Rates:  [346.5095]
Scaled Enconders  [[40.654743]]
Gain:  [40.654743]
Bias:  [-17.575737]


#### Parameters Layer1 After Training (sim.data) ####
Encoders:  [[1.]]
Intercepts [0.5316207]
Max Rates:  [326.63724]
Scaled Enconders  [[39.16842]]
Gain:  [39.16842]
Bias:  [-19.822742]


#### Parameters Layer1 Before Training (get_nengo_params) ####
Encoders:  [[1.]]
Intercepts:  Uniform(low=-1.0, high=0.9)
Max Rates:  Uniform(low=200, high=400)
Normalize encoders:  False
Gain:  [40.654743]
Bias:  [-17.575737]


#### Parameters Layer1 After Training (get_nengo_params) ####
Encoders:  [[0.9634403]]
Intercepts:  Uniform(low=-1.0, high=0.9)
Max Rates:  Uniform(low=200, high=400)
Normalize encoders:  False
Gain:  [40.654743]
Bias:  [-19.822742]
4

Righto, I think I understand what is going on here. So… in Nengo, the input weights to a population of neurons can be split up into “encoders” and the neuron “gains”, or, they can be combined together to form the “scaled encoders”. When you are training the weights in NengoDL, it modifies the combined (scaled encoders) value, and these are the values you are getting when you use sim.data to probe the model.

The get_nengo_params function, however, doesn’t give you the values in the network. Rather, it returns the values that you would need to construct an equivalent Nengo network. Since the scaled encoders are the combination of the encoders and gains, the get_nengo_params function has to “reverse engineer” them to retrieve the encoder values back. The get_nengo_params function opts to keep the neuron biases as they were during the initial construction of the network (i.e., before the training) as this maintains maximum compatibility with things that you may add to this equivalent Nengo network after you have done this conversion.

You can see from your output (Parameters Layer1 After Training (get_nengo_params)) that 0.9634403 (encoders) x 40.654743 (gain) = 39.16842 (scaled encoders / gain with sim.data)

5

@xchoo thank you so much for the response! Now I can better understand the difference of this two functions.

Also, i still have doubts about this question below:

Could you explain to me why this happens?

6

Yup! The intercepts and max rates of a neuron are properties of the bias and gains of said neuron. You can solve for the intercepts from the neuron bias, and you can solve for the max rates from the neuron gain (and vice versa for both). There is an example of this in the Nengo codebase, although this example computes the gain and bias values from given intercept and max rates (but the reverse computation can be done too).

Here’s a quick little network that lets you play around with the neuron gain and bias values and see how it affects the intercepts and max rates (you’ll see the changes in the neuron response curve):

import matplotlib.pyplot as plt
import nengo
from nengo.utils.ensemble import response_curves

with nengo.Network() as model:
    ens = nengo.Ensemble(1, 1, gain=[1], bias=[0.6])

with nengo.Simulator(model) as sim:
    eval_points, activities = response_curves(ens, sim)

plt.figure()
plt.plot(eval_points, activities)
plt.show()
7

@xchoo you’re the best! Thank you for clarifying this question in such a simple-to-understand way.

Now, I have other questions that go along with the same theme! With the same code above, but using for example 10 neurons, and still “forcing” initial ensemble parameters like intercepts and max rates, like this:

# General parameters from ensambles (forced initial parameters: intercepts and max_rates)
ens_params = dict(
    dimensions = 1,
    neuron_type = nengo.LIF(tau_rc=0.02, tau_ref=0.002, min_voltage=0, amplitude=1),
    intercepts = Uniform(low=0.25, high=0.25),
    max_rates = nengo.dists.Choice([300]),
    encoders = Choice([[1]]))

# General parameters from ensambles (not forced parameters)
ens_params2 = dict(
    dimensions = 1,
    neuron_type = nengo.LIF(tau_rc=0.02, tau_ref=0.002, min_voltage=0, amplitude=1))

I have this response of the network:


###### Response of neural network with forced initial parameters ######

#### Parameters SNN1 Before Training ####
Encoders:  [[1.][1.][1.][1.][1.][1.][1.][1.][1.][1.]]
Intercepts [0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25 0.25]
Max Rates:  [300. 300. 300. 300. 300. 300. 300. 300. 300. 300.]
Scaled Enconders  [[19.34074][19.34074][19.34074][19.34074][19.34074][19.34074][19.34074][19.34074][19.34074][19.34074]]
Gain:  [19.34074 19.34074 19.34074 19.34074 19.34074 19.34074 19.34074 19.34074 19.34074 19.34074]
Bias:  [-3.835185 -3.835185 -3.835185 -3.835185 -3.835185 -3.835185 -3.835185 -3.835185 -3.835185 -3.835185]

#### Parameters SNN1 After Training ####
Encoders:  [[1.][1.][1.][1.][1.][1.][1.][1.][1.][1.]]
Intercepts [0.51157594 0.51157594 0.51157594 0.51157594 0.51157594 0.51157594 0.51157594 0.51157594 0.46794277 0.46794277]
Max Rates:  [272.1361  272.1361  272.1361  272.1361  272.1361  272.1361  272.1361 272.1361  280.71762 280.71762]
Scaled Enconders  [[23.442541][23.442541][23.442541][23.442541][23.442541][23.442541][23.442541][23.442541][23.13315 ][23.13315 ]]
Gain:  [23.442541 23.442541 23.442541 23.442541 23.442541 23.442541 23.442541 23.442541 23.13315  23.13315 ]
Bias:  [-10.99264 -10.99264 -10.99264 -10.99264 -10.99264 -10.99264 -10.99264 -10.99264  -9.82499  -9.82499]

###################################################################

###### Response of neural network with don't  forced initial parameters ######
#### Parameters Layer1 Before Training (sim.data) ####
Encoders:  [[ 1.][ 1.][-1.][-1.][ 1.][ 1.][-1.][-1.][ 1.][-1.]]
Intercepts [-0.5493578   0.6119181  -0.10976071  0.16738959 -0.45214522 -0.27786326  -0.921912   -0.5491982   0.5237297  -0.39544743]
Max Rates:  [355.18857 307.13345 399.72696 392.96005 300.53802 281.62195 378.42706 338.01846 385.69867 298.1063 ]
Scaled Enconders  [[ 15.510341][ 39.75938 ][-35.472458][-43.494305][ 10.035475][  9.705682][-15.937389][-13.14984 ][ 69.80595 ][-10.226936]]
Gain:  [15.510341 39.75938  35.472458 43.494305 10.035475  9.705682 15.937389 13.14984  69.80595  10.226936]
Bias:  [  9.520726  -23.329485    4.893482   -6.2804937   5.537492    3.6968524	   15.692871    8.2218685 -35.559444    5.0442157]


#### Parameters Layer1 After Training (sim.data) ####
Encoders:  [[ 1.][ 1.][-1.][-1.][-1.][-1.][-1.][-1.][ 1.][-1.]]
Intercepts [-8.0088673e+00  5.6406927e-01  3.0910268e-03  1.6738959e-01  -5.9472275e+00 -5.8257103e+00 -7.3883110e-01 -7.6665741e-01  5.7421625e-01 -7.6239222e-01]
Max Rates:  [316.74918 369.26248 383.75067 392.96005 246.12065 232.80823 437.21774  431.12595 393.2284  425.84   ]
Scaled Enconders  [[  1.8637006][ 63.651222 ][-32.614334 ][-43.494305 ][ -1.3246995][ -1.2046679][-39.76323  ][-35.149803 ][ 85.32784  ][-32.29885  ]]
Gain:  [ 1.8637006 63.651222  32.614334  43.494305   1.3246995  1.2046679  39.76323   35.149803  85.32784   32.29885  ]
Bias:  [ 15.926131  -34.903698    0.8991882  -6.2804937   8.878289    8.018046   30.378311   27.947857  -47.99663    25.624393 ]

As you can see, when I “force” the ensemble parameters, all neurons before and after training are optimized in the same way. On the other hand, when I don’t “force” the parameters, each neuron in the ensemble is optimized differently. And the same happens if I add more neurons to the ensemble. Why does this happen? Even with forced initial parameters, shouldn’t the network optimize each neuron separately?

I just want to better understand what happens with the parameters when training a network.

8

I’m going to preface this by saying that I’m not fully intimate with how the internals of TensorFlow works, or what algorithm it uses to modify the network weights during the sim.fit function call… So, what I discuss below is an educated guess at best. :slight_smile:

The behaviour you are observing has a lot to do with the network structure, as well as the training algorithm used. From what I understand about machine learning algorithms, they work by computing some error value (or some equivalent metric), and then apply some math to this error value to figure out how to modify the weights in the network (this can be the neuron parameters – gain and bias, as well as the connection weights).

Some algorithms figure out the contribution of individual neurons to the output value, and use this contribution value to proportion the weight change. As an example (a super simplistic example), if neuron A contributes 25% of the activity to the output, while neuron B contributes 75% of the activity to the output, then the error value might be proportioned as 0.25 x error to neuron A, and 0.75 x error to neuron B.

Given that in your network, there is only one output value, if all of the neurons are initialized to the same parameters, each neuron would contribute equally to the output value. Thus, it would not be surprising to observe that the changes to the network are distributed equally to all of the neurons, which in turn means that all of the neurons end up with the same values, even after training. Now, if you configure your network to have multiple outputs (i.e. a vector output), you’ll probably see this behaviour go away, since each neuron now contributes differently to each element of the vector.

The other way to make this behaviour go away is to, as you have observed, initialize the population of neurons with different initial parameters. In this case, the contribution of each neuron to the output value is different, and the error value will affect each differently.

The behaviour you are observing should be independent to the number of neurons you have in the ensemble (assuming your output dimensionality is 1). If all of the neurons are initialized with the same parameters, I would expect that after training, all of the neurons would still have parameters equal to each other. This would not be the case with randomly generated neurons though.

9

@xchoo thank you for the reply and the help! I understand what you have explain, and this make sense for me.

Gratefully!!

10

Hi @xchoo , how are you?
I have a new question of my problem and I think you can help me!

As you can see in the code example, I was able to train a very very simple “network” with only 1 neuron, it’s as if I were going to specialize a neuron in detecting my problem! And, in this way, the metrics after training are very good. Right? But I would like to train a network for this (don’t just 1 neuron), for example with 2 layers and a few more neurons.

If I add neurons (2, 10, 100) in the ensemble (1 layer like the example code above) the metrics are not so good. Also, if I add more layers, just like this simple example:

with nengo.Network(seed=0) as model:

    # Input node 
    inpt = nengo.Node(output=nengo.processes.PresentInput(f_der, s_time))
    
    # First ensemble
    SNN1 = nengo.Ensemble(1, **ens_params2)
    out = nengo.Ensemble(1, **ens_params2)

    # Connect everything together
    conn1 = nengo.Connection(inpt, SNN1)
    conn2 = nengo.Connection(SNN1, out)
    
    # Add a probe on the input and output, in this case out_p is the output
    inpt_p = nengo.Probe(inpt)
    out_p = nengo.Probe(out)

The training metrics are terrible (even training with many epochs, like 1000), and the training parameters (bias, gain, connection weights) do not optimize much (they change but very little). I don’t understand why this happens, since I only added 1 layer and 1 neuron.

My impression is that the network can’t converge the training parameters to detect the problem, but with only 1 layer and 1 neuron the ‘network’ works well and the metrics are very good. Why is this happening? Could you help me?

And the same happens if I add more neurons in the first layer, or add more layers in the network.

Thanks!!

11

Hi @jone,

Whether or not a network is able to train effectively on a dataset depends very much on the network architecture, the data set used, and even the error evaluation function, and can very much be a “black art” in figuring out how to adjust these things to make your network work. Since I’m unsure exactly what tests you are doing with your network, I can only make educated guesses here.

This strikes me as odd. If your network contains just one ensemble (with randomly generated neuron parameters), the more neurons you add, the better the metrics should be. At the very least, your metrics should not get worse. As a sanity check, I recommend changing the neuron type to nengo.RectifiedLinear() to see if that improves the performance at all. If it doesn’t, I would next examine how the metrics are being computed. What is it you are trying to optimize?

Generally, try to stick with rate neurons with a linear firing rate function (i.e., a rate ReLU neuron) with your network until you are certain the architecture is trainable with the dataset. This is the point where you can experiment on the network architecture / number of neurons to see how it impacts the model’s performance. Once you get this basic model working, then try out the spiking ReLU neuron in your network. Switching over to the spiking ReLU neuron may expose issues in the network like needing to modify the synapses on the connections for better performance. Once the spiking ReLU network works, only then should you try the LIF neuron (you can do the rate LIF neuron first too). The LIF neuron has a non-linear response curve so additional neurons may be needed to get the network to behave like you want.

This may make some sense. I’m speculating here, but remember that the only way the training algorithm can figure out how to adjust the network weights is by calculating an error based on the output of the network. With 2 layers, the neuron activity of both layers have to be active for the network to produce an output. This probably makes it very difficult for the training algorithm to figure out how to adjust the weights to get a decent output, since whatever it does, the output of the network remains at 0. I recommend probing and plotting the network output during the training process to see what is actually happening with the network output.

Unintuitively, if you add more neurons to both the first and second layer, it may enable the training algorithm to do a better job at figuring out how to adjust the network weights to achieve the desired function. Additionally, you can try doing a neuron-to-neuron connection between the two layers:

nengo.Connection(SNN1.neurons, out.neurons, transform=nengo_dl.dists.Glorot())

to give the training algorithm more free parameters to adjust. Note that in the code above, the tranform matrix is randomized (using the Glorot distribution) just to make sure that there is some activity in the second layer for any given input. If the weights were initialized to all 0’s, the second layer would produce no output regardless of the input, which gets us back to the original problem of the training algorithm not being able to figure out how to adjust the weights.

As I mentioned earlier, working with these training algorithms is a little bit of a learned art. However, the advantage of using Nengo is that you can easily probe the outputs of each layer and plot these outputs in order to get an insight to what is going on with your network.

12

Hi @xchoo, thank you for the response!

Of course, I will explain what I’m trying to optimize. Basically, I want to train a spiking neural network to detect slippage in prostheses, for this I have several experiments with FSR sensor fixed on prostheses holding objects. I know exactly in time when the objects are slip and this is my ground truth (train/test target), like a binary spiking signal when occur slippage. Also, the input of my network can be just the signal waveform of the FSR, the absolute derivate of the FSR sensor or the response of electronic mechanoreceptors (the input of the mechanoreceptor is also the FSR waveform). Now, the testing who I’m sharing in this post have the derivate signal with input (train input), but the main objective is to use the mechanoreceptor response. All data from the training experiments are concatenated in a single file, as well as the test data (in a separate file). Therefore, I want to perform a supervised training with spiking neural network using Nengo to detect slippage event.

When I put the neuron type nengo.RectifiedLinear() In the network I have a specific error in the sim.evaluate() function. It doesn’t matter if I have the network with 1 or 2 layers (1 or more neurons as well) and the same happens when I use neuron type nengo.SpikingRectifiedLinear. This error just doesn’t occur when I use ​the LIF neuron type or when in the sim.compile() function I don’t add the evaluation metrics (but I need them to evaluate the model). Can you help me to resolve this problem? I tried to solve it but without success.

Error:

File "SNN21.py", line 355, in <module>
    evaluate_b = sim.evaluate(test_inputs, test_targets, verbose = 0)
  File "C:\Users\Jone\anaconda3-64\lib\site-packages\nengo\utils\magic.py", line 181, in __call__
    return self.wrapper(self.__wrapped__, self.instance, args, kwargs)
  File "C:\Users\Jone\anaconda3-64\lib\site-packages\nengo_dl\simulator.py", line 67, in require_open
    return wrapped(*args, **kwargs)
  File "C:\Users\Jone\anaconda3-64\lib\site-packages\nengo_dl\simulator.py", line 904, in evaluate
    return self._call_keras(
  File "C:\Users\Jone\anaconda3-64\lib\site-packages\nengo\utils\magic.py", line 181, in __call__
    return self.wrapper(self.__wrapped__, self.instance, args, kwargs)
  File "C:\Users\Jone\anaconda3-64\lib\site-packages\nengo_dl\simulator.py", line 50, in with_self
    output = wrapped(*args, **kwargs)
  File "C:\Users\Jone\anaconda3-64\lib\site-packages\nengo_dl\simulator.py", line 1043, in _call_keras
    outputs = getattr(self.keras_model, func_type)(**func_args)
  File "C:\Users\Jone\anaconda3-64\lib\site-packages\tensorflow\python\keras\engine\training.py", line 108, in _method_wrapper
    return method(self, *args, **kwargs)
  File "C:\Users\Jone\anaconda3-64\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1379, in evaluate
    tmp_logs = test_function(iterator)
  File "C:\Users\Jone\anaconda3-64\lib\site-packages\tensorflow\python\eager\def_function.py", line 780, in __call__
    result = self._call(*args, **kwds)
  File "C:\Users\Jone\anaconda3-64\lib\site-packages\tensorflow\python\eager\def_function.py", line 807, in _call
    return self._stateless_fn(*args, **kwds)  # pylint: disable=not-callable
  File "C:\Users\Jone\anaconda3-64\lib\site-packages\tensorflow\python\eager\function.py", line 2829, in __call__
    return graph_function._filtered_call(args, kwargs)  # pylint: disable=protected-access
  File "C:\Users\Jone\anaconda3-64\lib\site-packages\tensorflow\python\eager\function.py", line 1843, in _filtered_call
    return self._call_flat(
  File "C:\Users\Jone\anaconda3-64\lib\site-packages\tensorflow\python\eager\function.py", line 1923, in _call_flat
    return self._build_call_outputs(self._inference_function.call(
  File "C:\Users\Jone\anaconda3-64\lib\site-packages\tensorflow\python\eager\function.py", line 545, in call
    outputs = execute.execute(
  File "C:\Users\Jone\anaconda3-64\lib\site-packages\tensorflow\python\eager\execute.py", line 59, in quick_execute
    tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError:  assertion failed: [predictions must be <= 1] [Condition x <= y did not hold element-wise:] [x (keras_model/TensorGraph/transpose_1:0) = ] [[[0]][[0]][[1.03317153]]...] [y (Cast_6/x:0) = ] [1]
         [[{{node assert_less_equal/Assert/AssertGuard/else/_106/assert_less_equal/Assert/AssertGuard/Assert}}]] [Op:__inference_test_function_2642]

Function call stack:
test_function

What is the difference to use the argument solver in the connections (with solver.weights = True) and the argument transform in NengoDL to make a neuron-to-neuron connection? The argument solver change the connection to use connection weights instead of decoders, and the transform it is just a weight initialization method, this is right? Do I have an issue to use both methods/arguments together in NengoDL?

I don’t know how to visualize the output during training using the tensorflow functions, just before and after training. If I use the sim.run() function, then perform the training with the sim.fit() function and then run the sim.run() function again, I’m simulating the network before and after the training, right? But I don’t know how to visualize the network output during training using sim.fit() function. I just use TensorBoard to only view metrics and connection weights during training, as shown in the posted code example.