NotImplementedError: Cannot convert a symbolic Tensor (TensorGraph/while/iteration_0/ElementwiseIncBuilder/meshgrid/Size:0) to a numpy array

Without any apparent changes in my code I’ve started getting the following error when trying to use the NengoDL simulator with my custom neuron class:

Traceback (most recent call last):
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/IPython/core/interactiveshell.py”, line 3418, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File “”, line 1, in
runfile(’/Users/thomastiotto/PycharmProjects/Learning-to-approximate-functions-using-niobium-doped-strontium-titanate-memristors/experiments/mPES.py’, wdir=’/Users/thomastiotto/PycharmProjects/Learning-to-approximate-functions-using-niobium-doped-strontium-titanate-memristors/experiments’)
File “/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py”, line 197, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File “/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py”, line 18, in execfile
exec(compile(contents+"\n", file, ‘exec’), glob, loc)
File “/Users/thomastiotto/PycharmProjects/Learning-to-approximate-functions-using-niobium-doped-strontium-titanate-memristors/experiments/mPES.py”, line 214, in
cm = nengo_dl.Simulator( model, seed=seed, dt=timestep, progress_bar=progress_bar, device=device )
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/nengo_dl/simulator.py”, line 526, in init
self._build_keras(progress)
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/nengo/utils/magic.py”, line 181, in call
return self.wrapper(self.wrapped, self.instance, args, kwargs)
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/nengo_dl/simulator.py”, line 50, in with_self
output = wrapped(*args, **kwargs)
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/nengo_dl/simulator.py”, line 549, in _build_keras
outputs = self.tensor_graph(inputs, stateful=self.stateful, progress=progress)
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py”, line 946, in call
return self._functional_construction_call(inputs, args, kwargs,
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py”, line 1085, in _functional_construction_call
outputs = self._keras_tensor_symbolic_call(
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py”, line 817, in _keras_tensor_symbolic_call
return self._infer_output_signature(inputs, args, kwargs, input_masks)
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py”, line 858, in _infer_output_signature
outputs = call_fn(inputs, *args, **kwargs)
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py”, line 620, in wrapper
return func(*args, **kwargs)
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/nengo_dl/tensor_graph.py”, line 488, in call
self._build_loop(sub) if self.use_loop else self._build_no_loop(sub)
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/nengo_dl/tensor_graph.py”, line 638, in _build_loop
loop_vars = tf.while_loop(
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/tensorflow/python/util/deprecation.py”, line 605, in new_func
return func(*args, **kwargs)
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/tensorflow/python/ops/control_flow_ops.py”, line 2489, in while_loop_v2
return while_loop(
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/tensorflow/python/ops/control_flow_ops.py”, line 2687, in while_loop
return while_v2.while_loop(
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/tensorflow/python/ops/while_v2.py”, line 192, in while_loop
body_graph = func_graph_module.func_graph_from_py_func(
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py”, line 990, in func_graph_from_py_func
func_outputs = python_func(func_args, **func_kwargs)
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/tensorflow/python/ops/while_v2.py”, line 178, in wrapped_body
outputs = body(
_pack_sequence_as(orig_loop_vars, args))
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/nengo_dl/tensor_graph.py”, line 615, in loop_body
loop_i = self._build_inner_loop(loop_i, update_probes, progress)
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/nengo_dl/tensor_graph.py”, line 758, in _build_inner_loop
self.signals.scatter(
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/nengo_dl/signals.py”, line 372, in scatter
result = tf.tensor_scatter_nd_update(var, dst.tf_indices_nd, val)
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/nengo_dl/signals.py”, line 212, in tf_indices_nd
tf.meshgrid(
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py”, line 201, in wrapper
return target(*args, **kwargs)
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/tensorflow/python/ops/array_ops.py”, line 3552, in meshgrid
mult_fact = ones(shapes, output_dtype)
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py”, line 201, in wrapper
return target(*args, **kwargs)
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/tensorflow/python/ops/array_ops.py”, line 3120, in ones
output = _constant_if_small(one, shape, dtype, name)
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/tensorflow/python/ops/array_ops.py”, line 2804, in _constant_if_small
if np.prod(shape) < 1000:
File “<array_function internals>”, line 5, in prod
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/numpy/core/fromnumeric.py”, line 3030, in prod
return _wrapreduction(a, np.multiply, ‘prod’, axis, dtype, out,
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/numpy/core/fromnumeric.py”, line 87, in _wrapreduction
return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
File “/Users/thomastiotto/miniforge3/envs/tf24/lib/python3.8/site-packages/tensorflow/python/framework/ops.py”, line 852, in array
raise NotImplementedError(
NotImplementedError: Cannot convert a symbolic Tensor (TensorGraph/while/iteration_0/meshgrid/Size:0) to a numpy array. This error may indicate that you’re trying to pass a Tensor to a NumPy call, which is not supported

My code can be found here and I cannot see what the problem might be, given that it was working perfectly until the last time I tested it:

My frontend class AdaptiveLIFLateralInhibition extends LIF and everything works when using the Nengo Core simulator.
I also have a AdaptiveLIFLateralInhibitionBuilder class for the NengoDL implementation, which I register with the default SimNeuronsBuilder.

Edit: this error has also appeared for my custom mPES learning rule:

Thanks in advance!

Hi @Tioz,

Assuming your code and environment hasn’t changed, it is almost impossible to have code that was working suddenly fail to work. Typically, something has changed between the working and non-working instances. I’m no stranger to encountering this issue (I’ve done this multiple times in the past), so here are some debugging tips:

  • Double check that the code you are using hasn’t changed. If the code has been committed to a Github repository, clone a completely new instance of the repository and test that code.
  • Check that the file you are saving to is the file actually being used (done this multiple times myself).
  • Check that the environment hasn’t changed.
  • If you don’t have the original environment around, create a new environment with the required packages and roll back versions of the installed packages until your code starts working. In this case, I would try rolling back NengoDL and TensorFlow as both of those have been recently updated.

As for the specific issue with your code, I tried to take a look at your code by following the links you posted, but I am getting 404 errors. It seems like you deleted the memristor_nengo folder in a commit you made a few hours ago.

I’m absolutely sure that my code has not changed so I was wondering if updates to NengoDL might be the cause.

On a theoretical level, what could be the cause of the error:

NotImplementedError: Cannot convert a symbolic Tensor (TensorGraph/while/iteration_0/meshgrid/Size:0) to a numpy array. This error may indicate that you’re trying to pass a Tensor to a NumPy call, which is not supported

Sorry about the code! I’ve been reorganising it and moved it to a new repository. Here are the NengoDL build function and OpBuilder for my learning rule and here is the OpBuilder for my neuron, which I register here.

Thanks again!

I manage to get it to work and found the culprit!
It seems that having numpy-1.20.1 breaks things. I’m not sure if NengoDL installed this dependency but I’d double check on your side :slight_smile:
I realised this might have been a problem while installing packages manually and getting the following warning:
tensorflow 2.4.1 requires numpy~=1.19.2, but you have numpy 1.20.1 which is incompatible.

I tested, and downgrading to numpy-1.19.2 solved the issue. :smile:
My working env right now is:

nengo                  3.1.0
nengo-dl               3.4.0
numpy                  1.19.2
tensorflow             2.4.1
tensorflow-estimator   2.4.0
1 Like

NengoDL’s Numpy requirement is numpy>=1.16.0. I just tested it in a new Python environment, and doing pip install nengo-dl installs numpy==1.19.5, which follows the TensorFlow requirements.

Hello there. I’m adding to this thread because I just got a similar error when running the Coming from Nengo to NengoDL example. The error is occurring when I get to the fifth code block where it runs the same Nengo network in NengoDL, and it says all that needs to be done is switch nengo.Simulator to nengo_dl.Simulator.

The environment I’m using has Numpy 1.20.1 installed. If I understand you correctly, I should be using Numpy 1.16.0 when running this example? Or is there something else I should be considering?

Thanks.

No. You should be using Numpy 1.19.2 or 1.19.5, which should be the version installed when you install tensorflow.

Thank you.

Saved my day as well. Downgrading numpy to 1.19.2 helped. Following is my setting:

tf.__version__, nengo_dl.__version__, nengo.__version__, np.__version__
('2.2.0', '3.4.0', '3.1.0', '1.19.2')