NengoDL 3.0.0 released

The NengoDL team is very excited to announce the release of NengoDL 3.0.0.

What is NengoDL?

NengoDL is a backend for Nengo that integrates deep learning methods (supported by the TensorFlow framework) with other Nengo modelling tools. This allows users to optimize their models using deep learning training methods, improves simulation speed (on CPU or GPU), and makes it easy to insert TensorFlow models (such as deep learning architectures) into Nengo networks.

How do I use it?

To use NengoDL, replace instances of nengo.Simulator with nengo_dl.Simulator.

For example, if you have a network called net and you run it as

with nengo.Simulator(net) as sim:
    sim.run(10)

you would change that to

with nengo_dl.Simulator(net) as sim:
    sim.run(10)

and that’s it!

Information on accessing the more advanced features of NengoDL can be found in the documentation.

What’s new?

NengoDL 3.0 is a significant rewrite of NengoDL, designed to take advantage of all the new features in TensorFlow 2.0. One of the most significant changes is that NengoDL has been reworked in many ways to integrate more easily with Keras (which is the new standard in TensorFlow 2.0).

  • Reworked the Simulator.train/loss API into Simulator.compile/fit/evaluate, to align with the Keras Model API.
  • Added a new tool for automatically converting Keras models to native Nengo networks.
  • Changed nengo_dl.tensor_layer to nengo_dl.Layer, which aligns with the Keras functional layer API.

There are a ton of other changes, both user facing and under the hood. See the changelog for all the details, or the migration guide for help upgrading old code to NengoDL 3.0. And all the documentation has been updated, so check out the examples if you would like to see these changes in action.

There’s also a new tips and tricks page, and a new example showing state of the art performance on a complicated RNN task using Legendre Memory Units in NengoDL.

As always with a major release like this, it’s possible (or probable) that there will be some issues we haven’t caught in our testing. Let us know if you run into any issues, and we will get them fixed as fast as we can!

How do I get it?

To install NengoDL, we recommend using pip:

pip install nengo-dl

More detailed installation instructions can be found here.

Where can I learn more?

Where can I get help?

You’re already there! If you have an issue upgrading or have any other questions, please post them in this forum.

1 Like

Hi,
Please tell me how to use Tensor Board for the following networks with Nengo DL(3.0.0 ver).
This code references “OPTIMIZING A SPIKING NEURAL NETWORK”.

%matplotlib inline

from urllib.request import urlretrieve

import nengo
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

import nengo_dl

from tensorflow.keras.callbacks import EarlyStopping
from nengo_dl.callbacks import NengoSummaries, TensorBoard

# load data
(train_images, train_labels), (test_images, test_labels) = (
    tf.keras.datasets.mnist.load_data())

# flatten images
train_images = train_images.reshape((train_images.shape[0], -1))
test_images = test_images.reshape((test_images.shape[0], -1))


with nengo.Network(seed=0) as net:
    # set some default parameters for the neurons that will make
    # the training progress more smoothly
    net.config[nengo.Ensemble].max_rates = nengo.dists.Choice([100])
    net.config[nengo.Ensemble].intercepts = nengo.dists.Choice([0])
    net.config[nengo.Connection].synapse = None
    neuron_type = nengo.LIF(amplitude=0.01)

    # this is an optimization to improve the training speed,
    # since we won't require stateful behaviour in this example
    nengo_dl.configure_settings(stateful=False)

    # the input node that will be used to feed in input images
    inp = nengo.Node(np.zeros(28 * 28))

    # add the first convolutional layer
    x = nengo_dl.Layer(tf.keras.layers.Conv2D(
        filters=32, kernel_size=3))(inp, shape_in=(28, 28, 1))
    x = nengo_dl.Layer(neuron_type)(x)

    # add the second convolutional layer
    x = nengo_dl.Layer(tf.keras.layers.Conv2D(
        filters=64, strides=2, kernel_size=3))(x, shape_in=(26, 26, 32))
    x = nengo_dl.Layer(neuron_type)(x)

    # add the third convolutional layer
    x = nengo_dl.Layer(tf.keras.layers.Conv2D(
        filters=128, strides=2, kernel_size=3))(x, shape_in=(12, 12, 64))
    x = nengo_dl.Layer(neuron_type)(x)

    # linear readout
    out = nengo_dl.Layer(tf.keras.layers.Dense(units=10))(x)

    # we'll create two different output probes, one with a filter
    # (for when we're simulating the network over time and
    # accumulating spikes), and one without (for when we're
    # training the network using a rate-based approximation)
    out_p = nengo.Probe(out, label="out_p")
    out_p_filt = nengo.Probe(out, synapse=0.1, label="out_p_filt")

# add single timestep to training data
train_images = train_images[:, None, :]
train_labels = train_labels[:, None, None]

# when testing our network with spiking neurons we will need to run it
# over time, so we repeat the input/target data for a number of
# timesteps.
n_steps = 30
test_images = np.tile(test_images[:, None, :], (1, n_steps, 1))
test_labels = np.tile(test_labels[:, None, None], (1, n_steps, 1))

def classification_accuracy(y_true, y_pred):
    return tf.metrics.sparse_categorical_accuracy(y_true[:, -1], y_pred[:, -1])

# note that we use `out_p_filt` when testing (to reduce the spike noise)
sim.compile(loss={out_p_filt: classification_accuracy})
print("accuracy before training:",sim.evaluate(test_images, {out_p_filt: test_labels}, verbose=1)["loss"])

# callbacks
es_cb = EarlyStopping(monitor='val_loss', patience=10, verbose=1, mode='auto')
# TensorBoard
ts_cb = TensorBoard(log_dir="logs/fit", histogram_freq=1)
ns_cb = NengoSummaries("logs/fit", sim, [???])

# run training
sim.compile(
    optimizer=tf.optimizers.RMSprop(0.0001),
    loss={out_p: tf.losses.SparseCategoricalCrossentropy(from_logits=True)}
)
history = sim.fit(train_images, {out_p: train_labels}, epochs=100, validation_split=0.2, callbacks=[es_cb,ts_cb,ns_cb])

I don’t understand nengo_dl.callbacks.NengoSummaries ().
If I can only pass nengo.Ensemble, Neurons, or Connection as the third argument, can I pass “Connection” from TensorNode?

Hi Souichi,

NengoSummaries is just there to make it easy to add TensorBoard summaries for Nengo objects (Ensembles, Neurons, and Connections). In a similar way to how tf.keras.callbacks.TensorBoard makes it easy to plot data from Keras layers. The code is pretty short if you want to take a look, the substantive part is just this function here https://github.com/nengo/nengo-dl/blob/master/nengo_dl/callbacks.py#L77. So it reads the values of the parameters associated with those objects (e.g. Ensemble encoders or Connection weights), and then simply adds a histogram summary for each of those parameters (using tf.summary.histogram).

Yes, you can pass a Connection from a TensorNode. But what is it you are trying to view in TensorBoard? I’m guessing you want to view statistics on the parameters inside the TensorNode (e.g. the tf.keras.layers.Conv2D kernel). In that case those are just standard TensorFlow parameters, not Nengo objects, so you would use the standard tf.keras.callbacks.TensorBoard callback (with, e.g., histogram_freq=1 if you wanted to plot histograms for your weights).