Supervised learning with nengo, mnist example, focus on reload weights and simulator

General Discussion
1

Hello everyone,
I am a beginner with nengo, but I have experiences with ANN framework like tensorflow, keras. I have 2 questions:

  1. When I train the nengo network with batch-input, how can I first reload the weights of communication channel(decoder), and continue the training based on the weights.

  2. I not clear with the simulation time in nengo simulator, in my case(mnist) I found no differences with I set the time to 1 or 100, the acc of prediction is the same. What I want is calculate the loss in simulation, if the loss is smaller than the threshold, I will stop the simulation.

My mnist-classification code is there, this code is implemented based on the code someone write in nengo_extra.
If it is ok, please write a new function show me how to do it.

I think this case if pretty general for the beginner of nengo, just like we start to learn artificial neural network, if we solve this mnist problem, we can solve a lot of problem just change the shape of input and output.

I quickly explain how I train the network and use the network to do prediction,
When I train the network, I connect the input neuron to the output node, and set eval_points=train_data (image),
function=train_targets(label) in connection.
When I do prediction, I connect input.neuron to the output node, and use the trained weights as decoder, directly set the transform=decoder.

import nengo
import numpy as np

from keras.datasets import mnist
from keras.utils import np_utils
from vision import Gabor, Mask
from sklearn.metrics import accuracy_score


class Q_network:

    def __init__(self, input_shape, output_shape, nb_hidden, decoder):
        '''
        :param input_shape: the input dimension without batch_size
        :param output_shape: the output dimension without batch_size
        :param nb_hidden: the number of neurons in ensemble
        :param decoder: the path to save weights of connection channel
        '''
        self.input_shape = input_shape
        self.output_shape = output_shape
        self.nb_hidden = nb_hidden
        self.decoder = decoder

    def encoder_initialization(self, way="default"):
        if way=="random":
            encoders = np.random.normal(0, 1, size=(self.nb_hidden, self.input_shape))
        else:
            rng = np.random.RandomState(self.output_shape)
            encoders = Gabor().generate(self.nb_hidden, (self.output_shape, self.output_shape), rng=rng)
            encoders = Mask((28, 28)).populate(encoders, rng=rng, flatten=True)
        return encoders

    def train_network(self, train_data, train_targets, simulation_time):

        encoders = self.encoder_initialization()
        solver = nengo.solvers.LstsqL2(reg=0.01)

        model = nengo.Network(seed=3)
        with model:
            input_neuron = nengo.Ensemble(n_neurons=self.nb_hidden,
                                           dimensions=self.input_shape,
                                           neuron_type=nengo.LIFRate(),
                                           intercepts=nengo.dists.Choice([-0.5]),
                                           max_rates=nengo.dists.Choice([100]),
                                           eval_points=train_data,
                                           encoders=encoders,
                                           )
            output = nengo.Node(size_in=self.output_shape)
            conn = nengo.Connection(input_neuron,
                                    output,
                                    synapse=None,
                                    eval_points=train_data,
                                    function=train_targets,
                                    solver=solver
                                    )
            conn_weights = nengo.Probe(conn, 'weights', sample_every=1.0)

        with nengo.Simulator(model) as sim:
            sim.run(simulation_time)

        # save the connection weights after training
        np.save(self.decoder, sim.data[conn_weights][-1].T)


    def predict(self, input):
        encoders = self.encoder_initialization()

        try:
            decoder = np.load(self.decoder)
        except IOError:
            decoder = np.zeros((self.nb_hidden, self.output_shape))

        model = nengo.Network(seed=3)
        with model:
            input_neuron = nengo.Ensemble(n_neurons=self.nb_hidden,
                                          dimensions=self.input_shape,
                                          neuron_type=nengo.LIFRate(),
                                          intercepts=nengo.dists.Choice([-0.5]),
                                          max_rates=nengo.dists.Choice([100]),
                                          encoders=encoders,
                                          )
            output = nengo.Node(size_in=self.output_shape)
            conn = nengo.Connection(input_neuron.neurons,
                                    output,
                                    synapse=None,
                                    transform=decoder.T
                                    )
        with nengo.Simulator(model) as sim:
            _, acts = nengo.utils.ensemble.tuning_curves(input_neuron, sim, inputs=input)

        return np.dot(acts, sim.data[conn].weights.T)
2

sim.run() isn’t performing the training, it is performing what you might call prediction/inference (running the optimized network). The optimization is performed when you create the Simulator. So actually the function you’ve written called predict is performing the training (with nengo.Simulator(model) as sim) and inference (_, acts = nengo.utils.ensemble.tuning_curves(input_neuron, sim, inputs=input); np.dot(acts, sim.data[conn].weights.T)). The train_network function, and saving/loading the weights, isn’t really necessary, because the decoders computed in the train_network function will be the same as if you just computed them within the predict function.

With the NEF, optimal decoders are computed for the given function (eval points and targets) in a single pass. So there is no need to train something for longer in order to get better accuracy, as you would in gradient descent-based methods. If you want to increase the accuracy of the output, you can increase the number of neurons (nb_hidden).

3

Hello drasmuss,
Thanks for your answer!
I am a beginner for Spiking neural network, so actually I can not use the way deal with artificial neural network to work on SNN. And I rewrite the function:

import nengo
import numpy as np
from vision import Gabor, Mask
from keras.datasets import mnist
from keras.utils import np_utils
from sklearn.metrics import accuracy_score

class mnist_classification:

def __init__(self, input_shape, output_shape, nb_hidden):
    '''
    Spiking neural network as the Q value function approximation
    :param input_shape: the input dimension without batch_size, example: state is 2 dimension, 
    action is 1 dimenstion, the input shape is 3.
    :param output_shape: the output dimension without batch_size, the dimenstion of Q values
    :param nb_hidden: the number of neurons in ensemble
    :param decoder: the path to save weights of connection channel
    '''
    self.input_shape = input_shape
    self.output_shape = output_shape
    self.nb_hidden = nb_hidden

def encoder_initialization(self):
    '''
    encoder is the connection relationship between input and the ensemble
    :return: initialised encoder
    '''
    rng = np.random.RandomState(self.output_shape)
    encoders = Gabor().generate(self.nb_hidden, (1, 1), rng=rng)
    encoders = Mask((self.input_shape, 1)).populate(encoders, rng=rng, flatten=True)
    return encoders

def traning_and_prediction(self, train_data, train_targets, evl_image):
    '''
    training the network useing all training data
    :param train_data: the training input, shape = (nb_samples, dim_samples)
    :param train_targets: the label or Q values shape=(nbm samples, dim_samples)
    :param simulation_time: the time to do the simulation, default = 100s
    :return: 
    '''

    encoders = self.encoder_initialization()
    solver = nengo.solvers.LstsqL2(reg=0.01)

    model = nengo.Network(seed=3)
    with model:
        input_neuron = nengo.Ensemble(n_neurons=self.nb_hidden,
                                      dimensions=self.input_shape,
                                      neuron_type=nengo.LIFRate(),
                                      intercepts=nengo.dists.Choice([-0.5]),
                                      max_rates=nengo.dists.Choice([100]),
                                      eval_points=train_data,
                                      encoders=encoders,
                                      )
        output = nengo.Node(size_in=self.output_shape)
        conn = nengo.Connection(input_neuron,
                                output,
                                synapse=None,
                                eval_points=train_data,
                                function=train_targets,
                                solver=solver
                                )
    # training is done after create the simulator
    with nengo.Simulator(model) as sim:
        # prediction for a single image
        _, acts = nengo.utils.ensemble.tuning_curves(input_neuron, sim, inputs=evl_image)
        return np.dot(acts, sim.data[conn].weights.T)
4

Hello drasmuss,

And I have some further questions:
1, When the simulation is done, this time shows: Building finished in 0:00:20. What this time mean?
2, As you said, the training is already done after the simulator is created, and you mentioned it is not necessary to save the decoder, but I found the training actually takes some time, if I have large data_set, maybe it takes more time. Should I save the encoder and decoder so that I can directly use them for single prediction without training?
3, Just make sure I understand NEF, optimal decoders are computed for the given function (eval points and targets) in a single pass. So what we do is put all of our training data once into the nengo network? In nengo there is no concept of batches?
Because in normal ANN, we input batches of training data into network, and we can also fine-tuning the network based on some pre-trained weights.

best greetings

5

That is the time taken to compute the decoders (training time). Technically it includes all aspects of constructing the model, not just computing decoders, but unless the structure of your network is very complex the decoder time will dominate any other factors.

Nengo actually has a built-in cache system for the decoders, so as long as you are setting the seed (nengo.Network(seed=3)), you should see that when you build the same model a second time (nengo.Simulator(net)), it will be faster. However, if you want to save them manually you can do that too. You can access the optimized decoders via w = sim.data[conn].weights, and then save them in whatever format you’d like. Then if you want to use those decoders to initialize a connection, you can create the connection like conn = nengo.Connection(input_neuron.neurons, output, transform=w, synapse=None). The encoders work in the same way, you can access them via e = sim.data[input_neurons].encoders and use those to initialize an ensemble via nengo.Ensemble(..., encoders=e).

That’s correct. There is no need for “fine-tuning” in this case, since the optimal weights are computed on the first pass.