Questions about training the converted model with dynamic input pipeline in Nengo-dl

When training my model with Nengo-dl, I used the tensorflow map function to preprocess my dataset. The code below basically shows how do I preprocess my dataset.

##"files" here just mean a tensor of filenames on disk
def preprocess_dataset(files):    
   files_ds =
   output_ds =
   output_ds =
   return output_ds

Since I got many data samples and I want to preprocess my data more efficiently with the map function, my training data is constructed in the form of MapDataset when I feed them to I have used a simple keras model and converted it with the nengo-dl.converter. This error below arises when I use method. This problem arises because that the method tries to get the batch_size and time_steps information using the .shape method, however this is not supported by the MapDataset I used for training.

Traceback (most recent call last):
  File "D:/snn models/spiking_models/", line 386, in <module>
  File "D:\Anaconda3\envs\myvenv3\lib\site-packages\nengo\utils\", line 179, in __call__
    return self.wrapper(self.__wrapped__, self.instance, args, kwargs)
  File "D:\Anaconda3\envs\myvenv3\lib\site-packages\nengo_dl\", line 67, in require_open
    return wrapped(*args, **kwargs)
  File "D:\Anaconda3\envs\myvenv3\lib\site-packages\nengo_dl\", line 855, in fit
    x_val = self._generate_inputs(x_val, n_steps=n_steps)
  File "D:\Anaconda3\envs\myvenv3\lib\site-packages\nengo_dl\", line 1826, in _generate_inputs
    data_batch, data_steps = next(iter(data.values())).shape[:2]
AttributeError: 'MapDataset' object has no attribute 'shape'

So I wonder what type of dataset does the method in Nengo-dl supports? Does this mean that I can only use dataset that has the “shape” attribute? Since my dataset is large and I need to use the Map function to efficiently preprocess it, I wonder how can I specify the data_batch and data_steps information explicitly, preventing the from using the .shape method? Thank in advance for any replies. :watermelon:

Hi @fgh, and welcome to the Nengo forums! :smiley:

For NengoDL specifically, the function is pretty much a wrapper around the underlying TensorFlow function. This applies to other functions like sim.evaluate, sim.train, sim.compile, sim.evaluate, etc., as well. As such, any data format that works with the respective TensorFlow functions should also work with NengoDL’s equivalents.

As for your issue, I forwarded it to the NengoDL developer, and they seem to think that it may be an issue with the way you are passing the dataset object to the function. However, without an example of the code you are using it’s hard to definitely confirm this is the case. If you could provide some example code (or some minimal working example) that produces this issue, that would be great!

Our dev believes that you may be feeding the data in as a tuple, where according to the TensorFlow documentation, if you were to use a TensorFlow dataset as input for the validation data, you’ll just want to pass in the dataset (not a tuple). See:

Data on which to evaluate the loss and any model metrics at the end of each epoch. The model will not be trained on this data. Thus, note the fact that the validation loss of data provided using validation_split or validation_data is not affected by regularization layers like noise and dropout. validation_data will override validation_split. validation_data could be:
A tuple (x_val, y_val) of Numpy arrays or tensors.
A tuple (x_val, y_val, val_sample_weights) of NumPy arrays.
A Python generator or keras.utils.Sequence returning (inputs, targets) or (inputs, targets, sample_weights). validation_data is not yet supported with tf.distribute.experimental.ParameterServerStrategy.

If you are still encountering the error when you pass in just the dataset object, you could try converting the dataset to a NumPy array. I’m not too familiar with this process, though this stack overflow thread may help.

1 Like

Thank you very much for your reply! I looked again at this problem today, and after reading the nengo document again I managed to come up with a solution. So let me just answer my question here, and after that, I want to ask a question about a new error after I changed the input data format.

The MapDataset I provided in the above quesiton is in the form of (batchsize, n_steps, node.size_out) for the training label and (batchsize, n_steps, probe.size_in), but according to the nengo-dl document of, I am using a dynamic input pipeline when training with the tensorflow Mapdataset rather than a static numpy array version of training data, so the input data form should be in the following format according to the nengo-dl document.

Example for dynamic input on the Nengo-dl document:

with nengo.Network() as net:
    a = nengo.Node([0], label="a")
    p = nengo.Probe(a, label="p")

with nengo_dl.Simulator(net) as sim:
      dataset =
        ({"a": tf.ones((50, 10, 1)),
          "n_steps": tf.ones((50, 1), dtype=tf.int32) * 10},
         {"p": tf.ones((50, 10, 1))})


Therefore, I added the third map function to change the format of my dataset to produce the same format shown above in the nengo-dl document. (n_steps = 1)

def data_dict(data, label_id):
dict_sim_input = {'input_1': data, 'n_steps': tf.ones(1, dtype=tf.int32)}
dict_sim_output = {'dense.0': label_id}
return dict_sim_input, dict_sim_output

The naming of keys (‘input1’, ‘dense.0’) comes from the output of the following function, from my understanding these functions can give me the names of the corresponding nodes and probes for the converted model.

print("input node", nengo_dl.Converter(model).inputs[model.input])
print("output probe", nengo_dl.Converter(model).outputs[model.output])
(the output is ‘input1’ and ‘dense.0’ respectively)

And the map function I used to get the Mapdataset is as follows:

def preprocess_data(files):
    files_ds =
    output_ds =
    output_ds =
    output_ds =,)
return output_ds

After doing this, the dataset I feed to is in the following format(after batched), from my understanding it is the same format as shown in the nengo-dl document.

({'input_1': TensorSpec(shape=(None, 1, 4096), dtype=tf.float32, name=None), 
'n_steps': TensorSpec(shape=(None, 1), dtype=tf.int32, name=None)}, 
{'dense.0': TensorSpec(shape=(None, 1, 1), dtype=tf.int64, name=None)})

However, the following error messages show up when I train the model with the above data, I just put the last few lines here

ValueError: in user code:
    File "D:\Anaconda3\envs\myvenv3\lib\site-packages\keras\engine\", line 183, in assert_input_compatibility
        raise ValueError(f'Missing data for input "{name}". '

ValueError: Missing data for input "conv2d.0.bias". You passed a data dictionary with keys ['input_1', 'n_steps']. Expected the following keys: ['input_1', 'conv2d.0.bias', 'conv2d_1.0.bias', 'dense.0.bias', 'n_steps']

From my understanding, besides the training data, the asks me for other dictionary keys, and these dictionary keys are parameters of the convolutional layers in my model. The code for my model and simulation is as follows:

inputs = keras.Input(shape=(inp_width, inp_height, 1))
Conv1 = layers.Conv2D(32, 3, strides=2, activation="relu")(inputs)
Average1 = layers.AveragePooling2D(5)(Conv1)
Conv2 = layers.Conv2D(32, 3, activation="relu")(Average1)
Flatten1 = layers.Flatten()(Conv2)
outputs = layers.Dense(units=num_labels)(Flatten1)
model = keras.Model(inputs=inputs, outputs=outputs, name="conv_model")

converter = nengo_dl.Converter(model)
print("inputs", nengo_dl.Converter(model).inputs[model.input])
print("outputs", nengo_dl.Converter(model).outputs[model.output])

with nengo_dl.Simulator(, minibatch_size=64) as sim:
    # run training
    # save the parameters to file

However, if I use a static numpy version for input data, the model can be trained. Train_data (not batched) is in the shape of (number of samples. n_steps, node.size_out), train label is in the shape of (number of samples, n_steps, probe.size_in).

The code for training the model with a static numpy array:

with nengo_dl.Simulator(, minibatch_size=64) as sim:
    # run training
        {converter.inputs[inputs]: train_data},
        {converter.outputs[outputs]: train_label},
            {converter.inputs[inputs]: val_data},
            {converter.outputs[outputs]: val_label},
    save the parameters to file

The question I want to ask is, I don’t know why I have to manually specify the bias of my convolutional layers when I use the Mapdataset, while I can train the model if I use a big static numpy array version of my dataset. Could you please provide some information about this? Thx in advance for your reply!

There’s actually some information in the NengoDL documentation for the fit function that outlines exactly what is needed (or assumed) for the type of input you provide. Namely:

Under the hood, NengoDL uses nengo.Node objects to implement the bias weights. Nengo nodes are also used to provide input to and get output from the network itself. You can sort of consider the bias values as “inputs” to the network, so having the nodes implement the biases as well as doing “double-duty” providing input/output to the network make sense.

With “static” data, because everything is predetermined at the start of the training process, NengoDL can do some behind-the-scenes “magic” (the part about introspection) to figure out how to set the bias values according to how they have been configured in the network architecture. However, when you use a generator, it’s possible to modify the data while the training / simulation is running, so what NengoDL does is to avoid doing any introspection on any inputs. Since the bias values are inputs to the network, the side effect of this is that if you are using a dynamic inputs, you need to include the bias input values in the generated data input.

Unfortunately, it’s very hard (or impossible) to figure out what is “static” and what is “dynamic” when using generators. It might be possible to add another set of user-defined parameters to tell NengoDL which inputs are static and which are dynamic, but that adds a bunch of extra complexity, and it opens the door for unexpected behaviour to occur. Adding additional parameters also breaks the symmetry of NengoDL’s function definition with TensorFlow’s's function definition. For these reasons, our devs decided it would be best (and more straightforward) for the user to specify the values for all of the input nodes in the case when generators are used.

1 Like

Thank you for the clear clarification :+1: :+1: I will manually specify the bias parameters required by this function.

1 Like