Thank you very much for your reply! I looked again at this problem today, and after reading the nengo document again I managed to come up with a solution. So let me just answer my question here, and after that, I want to ask a question about a new error after I changed the input data format.
The MapDataset I provided in the above quesiton is in the form of (batchsize, n_steps, node.size_out) for the training label and (batchsize, n_steps, probe.size_in), but according to the nengo-dl document of sim.fit(), I am using a dynamic input pipeline when training with the tensorflow Mapdataset rather than a static numpy array version of training data, so the input data form should be in the following format according to the nengo-dl document.
Example for dynamic input on the Nengo-dl document:
with nengo.Network() as net:
a = nengo.Node([0], label="a")
p = nengo.Probe(a, label="p")
with nengo_dl.Simulator(net) as sim:
dataset = tf.data.Dataset.from_tensor_slices(
({"a": tf.ones((50, 10, 1)),
"n_steps": tf.ones((50, 1), dtype=tf.int32) * 10},
{"p": tf.ones((50, 10, 1))})
).batch(sim.minibatch_size)
sim.compile(loss="mse")
sim.fit(x=dataset)
Therefore, I added the third map function to change the format of my dataset to produce the same format shown above in the nengo-dl document. (n_steps = 1)
def data_dict(data, label_id):
dict_sim_input = {'input_1': data, 'n_steps': tf.ones(1, dtype=tf.int32)}
dict_sim_output = {'dense.0': label_id}
return dict_sim_input, dict_sim_output
The naming of keys (‘input1’, ‘dense.0’) comes from the output of the following function, from my understanding these functions can give me the names of the corresponding nodes and probes for the converted model.
print("input node", nengo_dl.Converter(model).inputs[model.input])
print("output probe", nengo_dl.Converter(model).outputs[model.output])
(the output is ‘input1’ and ‘dense.0’ respectively)
And the map function I used to get the Mapdataset is as follows:
def preprocess_data(files):
files_ds = tf.data.Dataset.from_tensor_slices(files)
output_ds = files_ds.map(
map_func=first_map_function
)
output_ds = output_ds.map(
map_func=second_map_function,
)
output_ds = output_ds.map(map_func=data_dict,)
return output_ds
After doing this, the dataset I feed to sim.fit() is in the following format(after batched), from my understanding it is the same format as shown in the nengo-dl document.
element_spec=
({'input_1': TensorSpec(shape=(None, 1, 4096), dtype=tf.float32, name=None),
'n_steps': TensorSpec(shape=(None, 1), dtype=tf.int32, name=None)},
{'dense.0': TensorSpec(shape=(None, 1, 1), dtype=tf.int64, name=None)})
However, the following error messages show up when I train the model with the above data, I just put the last few lines here
ValueError: in user code:
File "D:\Anaconda3\envs\myvenv3\lib\site-packages\keras\engine\input_spec.py", line 183, in assert_input_compatibility
raise ValueError(f'Missing data for input "{name}". '
ValueError: Missing data for input "conv2d.0.bias". You passed a data dictionary with keys ['input_1', 'n_steps']. Expected the following keys: ['input_1', 'conv2d.0.bias', 'conv2d_1.0.bias', 'dense.0.bias', 'n_steps']
From my understanding, besides the training data, the sim.fit() asks me for other dictionary keys, and these dictionary keys are parameters of the convolutional layers in my model. The code for my model and simulation is as follows:
inputs = keras.Input(shape=(inp_width, inp_height, 1))
Conv1 = layers.Conv2D(32, 3, strides=2, activation="relu")(inputs)
Average1 = layers.AveragePooling2D(5)(Conv1)
Conv2 = layers.Conv2D(32, 3, activation="relu")(Average1)
Flatten1 = layers.Flatten()(Conv2)
outputs = layers.Dense(units=num_labels)(Flatten1)
model = keras.Model(inputs=inputs, outputs=outputs, name="conv_model")
model.summary()
converter = nengo_dl.Converter(model)
print("inputs", nengo_dl.Converter(model).inputs[model.input])
print("outputs", nengo_dl.Converter(model).outputs[model.output])
with nengo_dl.Simulator(converter.net, minibatch_size=64) as sim:
# run training
sim.compile(
optimizer=tf.optimizers.Adam(0.003),
loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=[tf.metrics.sparse_categorical_accuracy],
)
sim.fit(
train_data,
validation_data=val_data,
epochs=20,
)
# save the parameters to file
sim.save_params("./keras_to_snn_params")
However, if I use a static numpy version for input data, the model can be trained. Train_data (not batched) is in the shape of (number of samples. n_steps, node.size_out), train label is in the shape of (number of samples, n_steps, probe.size_in).
The code for training the model with a static numpy array:
with nengo_dl.Simulator(converter.net, minibatch_size=64) as sim:
# run training
sim.compile(
optimizer=tf.optimizers.Adam(0.003),
loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=[tf.metrics.sparse_categorical_accuracy],
)
sim.fit(
{converter.inputs[inputs]: train_data},
{converter.outputs[outputs]: train_label},
validation_data=(
{converter.inputs[inputs]: val_data},
{converter.outputs[outputs]: val_label},
),
epochs=20,
)
save the parameters to file
sim.save_params("./keras_to_snn_params")
The question I want to ask is, I don’t know why I have to manually specify the bias of my convolutional layers when I use the Mapdataset, while I can train the model if I use a big static numpy array version of my dataset. Could you please provide some information about this? Thx in advance for your reply!
