Training loop using Nengo network

General Discussion
#1

Hi,

so far I have been using sim.fit() for training my models, but my new task requires me to use preloaded batches.
I was wondering if there is a good way to explicitly train on batches like in this post: https://keras.io/guides/writing_a_training_loop_from_scratch/

Thanks :slight_smile:

#2

Hi Julian and welcome to the forum!

The keras example you reference is creating a custom training loop, which you probably don’t need to do.

NengoDL is setup to use batching by default, you should be able to simply set the batch size for the simulator object (nengo_dl.Simulator.minibatch_size). Take a look at the Batch Processing section of the docs.

If you want more control over the dataset, you could also try creating a data generator to provide the training inputs. For example, Tensorflow has tf.data.Dataset and there is more information about inputs and a brief example of this in the Simulator.fit docs:

with nengo.Network() as net:
    a = nengo.Node([0], label="a")
    p = nengo.Probe(a, label="p")

with nengo_dl.Simulator(net) as sim:
    dataset = tf.data.Dataset.from_tensor_slices(
        ({"a": tf.ones((50, 10, 1)),
          "n_steps": tf.ones((50, 1), dtype=tf.int32) * 10},
         {"p": tf.ones((50, 10, 1))})
    ).batch(sim.minibatch_size)

    sim.compile(loss="mse")
    sim.fit(x=dataset)

Keras also has methods for creating data generators if you prefer to integrate that instead, take a look at this post about keras data generators

I hope this helps!

1 Like