Hello everyone,

I am including below a small script to reproduce the issue I am facing.

```
import nengo
import numpy as np
import tensorflow as tf
import nengo_dl
seed = 0
np.random.seed(seed)
tf.random.set_seed(seed)
def get_model(include_kr=False):
inp = tf.keras.Input(shape=(28, 28, 1))
# convolutional layers
if include_kr:
conv0 = tf.keras.layers.Conv2D(
filters=32,
kernel_size=3,
activation=tf.nn.relu,
kernel_regularizer=tf.keras.regularizers.l2(1e-3),
)(inp)
conv1 = tf.keras.layers.Conv2D(
filters=64,
kernel_size=3,
strides=2,
activation=tf.nn.relu,
kernel_regularizer=tf.keras.regularizers.l2(1e-3),
)(conv0)
else:
conv0 = tf.keras.layers.Conv2D(
filters=32,
kernel_size=3,
activation=tf.nn.relu,
)(inp)
conv1 = tf.keras.layers.Conv2D(
filters=64,
kernel_size=3,
strides=2,
activation=tf.nn.relu,
)(conv0)
flatten = tf.keras.layers.Flatten()(conv1)
# fully connected layer.
if include_kr:
dense = tf.keras.layers.Dense(units=32, activation="relu",
kernel_regularizer=tf.keras.regularizers.l2(1e-3))(flatten)
dense = tf.keras.layers.Dense(units=64, activation="relu",
kernel_regularizer=tf.keras.regularizers.l2(1e-3))(dense)
else:
dense = tf.keras.layers.Dense(units=32, activation="relu")(flatten)
dense = tf.keras.layers.Dense(units=64, activation="relu")(dense)
# output layer.
dense = tf.keras.layers.Dense(units=10, activation="softmax")(dense)
model = tf.keras.Model(inputs=inp, outputs=dense)
model.summary()
return model
```

As you can see, depending on the value of `include_kr`

, the `get_model()`

returns model with or without the kernel regularizers in both Conv layers. Upon converting the model and inspecting the output of `converter.net.all_nodes`

, I see two different outputs as below. Is this an expected behaviour? Why are there `bias_relay`

nodes (in Nengo-DL v3.4.0 which are `unlabeled`

nodes (in Nengo-DL v3.2.0) ) as well as nodes with different labels (i.e. “0.bias” appended to layer names)?

With `include_kr=False`

:

```
model = get_model()
converter = nengo_dl.Converter(model)
converter.net.all_nodes
```

Output:

```
[<Node "input_1" at 0x2abef3fa8150>,
<Node "conv2d.0.bias" at 0x2abef157c410>,
<Node "conv2d.0.bias_relay" at 0x2abef3fc64d0>,
<Node "conv2d_1.0.bias" at 0x2abef3fc6a50>,
<Node "conv2d_1.0.bias_relay" at 0x2abef3fc6ad0>,
<TensorNode "dense_2.0" at 0x2abef154d110>,
<Node "dense_2.0.bias" at 0x2abefff284d0>]
```

With `include_kr=True`

:

```
model = get_model(include_kr=True)
converter = nengo_dl.Converter(model)
converter.net.all_nodes
```

Output:

```
[<Node "input_1" at 0x2b993bee9910>,
<TensorNode "conv2d" at 0x2b993bf29d10>,
<TensorNode "conv2d_1" at 0x2b993bf4b2d0>,
<TensorNode "dense" at 0x2b993bf4bcd0>,
<TensorNode "dense_1" at 0x2b993bf4bd50>,
<TensorNode "dense_2.0" at 0x2b993bf55390>,
<Node "dense_2.0.bias" at 0x2b993bf559d0>]
```

Please resolve.