Importing Trained Weights

Deep Learning
1

A few quick questions:

  1. Is there some clear cut way to import network weights that you trained in another system? I saw this post that seems related: https://github.com/nengo/nengo_extras/issues/35

  2. My understanding is that most or all Nengo ensembles are fully connected. Is there a way to do 1. with different layer types/structures (e.g., 2D convolution)?

  3. Is 1. done any differently for weights of deep networks still trained in a different system?

  4. Is there any convenience code kicking around for doing 1. and/or 2. with systems built in Tensorflow? I saw the work on NengoDL, but I’m less interested in Tensorflow nodes and more interested in simulating spiking versions of networks trained in Tensorflow.

2

Okay. So, this link from another forum post answers 1. for me: https://github.com/nengo/nengo/blob/master/docs/connections.rst#direct-connections

2, 3, and 4 seem to be deeply tied to NengoDL. I literally found a convolution2D operation discussed and then removed in relation to NengoDL here: https://github.com/nengo/nengo/pull/800

I think I’m going to call this solved and just ask a bit more about NengoDL.

3

And, here is the link that better explains how to get spiking neurons functioning with a Tensorflow setup:
http://www.nengo.ai/nengo_dl/examples/spiking_mnist.html

Don’t know how I missed that.

4

So, have you answered your own questions now or do you require further assistance? :slight_smile:

5

Hello,
I also have the same problem.
In nengo_dl, there is no problem, but the question is how to do it in nengo core?
In nengo Core, if you want to do off-line learning, you use nengo.solver to solve the decoder.

        conn = nengo.Connection(input_neuron,
                                output,
                                synapse=None,
                                eval_points=train_data,
                                function=train_targets,
                                solver=solver
                                )

Here eval_points is your training data, function is you label. then you can solve the connection weights(decoder).
My question is, can we here first load the pre-trained weights, then the nengo solver solve the optimization problem.

6

For now, yes! Thanks!

7

Check out this example for an explanation of how to do this in Nengo Core.