Hi, thx for the great work with Nengo & NengoDL
I tried to reproduce the code available @ NengoDL page : https://www.nengo.ai/nengo-dl/examples/spiking-mnist.html
i decided to add another layer based on Batch normalization (in order to build blocks: BNRELuCONV)
when i add the layer, the graph get built:
with nengo.Network() as net:
# set some default parameters for the neurons that will make
# the training progress more smoothly
net.config[nengo.Ensemble].max_rates = nengo.dists.Choice([100])
net.config[nengo.Ensemble].intercepts = nengo.dists.Choice([0])
neuron_type = nengo.LIF(amplitude=0.01)
# we'll make all the nengo objects in the network
# non-trainable. we could train them if we wanted, but they don't
# add any representational power. note that this doesn't affect
# the internal components of tensornodes, which will always be
# trainable or non-trainable depending on the code written in
# the tensornode.
nengo_dl.configure_settings(trainable=False)
# the input node that will be used to feed in input images
inp = nengo.Node([0] * 28 * 28)
# add the first convolutional layer
x = nengo_dl.tensor_layer(
inp, tf.layers.conv2d, shape_in=(28, 28, 1), filters=32,
kernel_size=3)
x = nengo_dl.tensor_layer(
x, tf.layers.batch_normalization)
# apply the neural nonlinearity
x = nengo_dl.tensor_layer(x, neuron_type)
# add another convolutional layer
x = nengo_dl.tensor_layer(
x, tf.layers.conv2d, shape_in=(26, 26, 32),
filters=64, kernel_size=3)
x = nengo_dl.tensor_layer(x, neuron_type)
# add a pooling layer
x = nengo_dl.tensor_layer(
x, tf.layers.average_pooling2d, shape_in=(24, 24, 64),
pool_size=2, strides=2)
# another convolutional layer
x = nengo_dl.tensor_layer(
x, tf.layers.conv2d, shape_in=(12, 12, 64),
filters=128, kernel_size=3)
x = nengo_dl.tensor_layer(x, neuron_type)
# another pooling layer
x = nengo_dl.tensor_layer(
x, tf.layers.average_pooling2d, shape_in=(10, 10, 128),
pool_size=2, strides=2)
# linear readout
x = nengo_dl.tensor_layer(x, tf.layers.dense, units=10)
# we'll create two different output probes, one with a filter
# (for when we're simulating the network over time and
# accumulating spikes), and one without (for when we're
# training the network using a rate-based approximation)
out_p = nengo.Probe(x)
out_p_filt = nengo.Probe(x, synapse=0.1)
BUT then when i follow the instructions:
I see this error message (when training):
(sorry it’s rather long)
NotFoundError: Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
Key batch_normalization/beta not found in checkpoint
[[node save/RestoreV2 (defined at /Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/nengo_dl/simulator.py:996) ]]
Caused by op ‘save/RestoreV2’, defined at:
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/runpy.py”, line 193, in _run_module_as_main
“main”, mod_spec)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/runpy.py”, line 85, in _run_code
exec(code, run_globals)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/ipykernel_launcher.py”, line 16, in
app.launch_new_instance()
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/traitlets/config/application.py”, line 658, in launch_instance
app.start()
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/ipykernel/kernelapp.py”, line 505, in start
self.io_loop.start()
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tornado/platform/asyncio.py”, line 132, in start
self.asyncio_loop.run_forever()
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/asyncio/base_events.py”, line 421, in run_forever
self._run_once()
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/asyncio/base_events.py”, line 1425, in _run_once
handle._run()
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/asyncio/events.py”, line 127, in _run
self._callback(*self._args)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tornado/ioloop.py”, line 758, in _run_callback
ret = callback()
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tornado/stack_context.py”, line 300, in null_wrapper
return fn(*args, **kwargs)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tornado/gen.py”, line 1233, in inner
self.run()
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tornado/gen.py”, line 1147, in run
yielded = self.gen.send(value)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/ipykernel/kernelbase.py”, line 357, in process_one
yield gen.maybe_future(dispatch(*args))
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tornado/gen.py”, line 326, in wrapper
yielded = next(result)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/ipykernel/kernelbase.py”, line 267, in dispatch_shell
yield gen.maybe_future(handler(stream, idents, msg))
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tornado/gen.py”, line 326, in wrapper
yielded = next(result)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/ipykernel/kernelbase.py”, line 534, in execute_request
user_expressions, allow_stdin,
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tornado/gen.py”, line 326, in wrapper
yielded = next(result)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/ipykernel/ipkernel.py”, line 294, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/ipykernel/zmqshell.py”, line 536, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/IPython/core/interactiveshell.py”, line 2843, in run_cell
raw_cell, store_history, silent, shell_futures)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/IPython/core/interactiveshell.py”, line 2869, in _run_cell
return runner(coro)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/IPython/core/async_helpers.py”, line 67, in _pseudo_sync_runner
coro.send(None)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/IPython/core/interactiveshell.py”, line 3044, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/IPython/core/interactiveshell.py”, line 3209, in run_ast_nodes
if (yield from self.run_code(code, result)):
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/IPython/core/interactiveshell.py”, line 3291, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File “”, line 17, in
sim.load_params("./mnist_params")
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/nengo_dl/simulator.py”, line 996, in load_params
tf.train.Saver(vars).restore(self.sess, path)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tensorflow/python/training/saver.py”, line 832, in init
self.build()
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tensorflow/python/training/saver.py”, line 844, in build
self._build(self._filename, build_save=True, build_restore=True)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tensorflow/python/training/saver.py”, line 881, in _build
build_save=build_save, build_restore=build_restore)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tensorflow/python/training/saver.py”, line 513, in _build_internal
restore_sequentially, reshape)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tensorflow/python/training/saver.py”, line 332, in _AddRestoreOps
restore_sequentially)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tensorflow/python/training/saver.py”, line 580, in bulk_restore
return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tensorflow/python/ops/gen_io_ops.py”, line 1572, in restore_v2
name=name)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py”, line 788, in _apply_op_helper
op_def=op_def)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tensorflow/python/util/deprecation.py”, line 507, in new_func
return func(*args, **kwargs)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tensorflow/python/framework/ops.py”, line 3300, in create_op
op_def=op_def)
File “/Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/tensorflow/python/framework/ops.py”, line 1801, in init
self._traceback = tf_stack.extract_stack()
NotFoundError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
Key batch_normalization/beta not found in checkpoint
[[node save/RestoreV2 (defined at /Users/assansanogo/Desktop/PYTHON_ANACONDA/anaconda2/envs/neuromorphics/lib/python3.5/site-packages/nengo_dl/simulator.py:996) ]]
What am I doing wrong? thx for your input 
