Calculate accuracy vs timesteps

General Discussion
1

Dears,

I’m using Nengodl for classification problem.
After training the model, I want to calculate the prediction accuracy versus timesteps.
Then, I feed the validation dataset to the model with each timesteps, and use model.evaluate to calculate the accuracy. However, the code can run only few times then the GPU memory get full and making error. I have two questions:

  1. It seems that the NengoDL still taking GPU memory after finish the model.evaluate. So, the memory is not released. In the next model.evaluate, the memory will get larger and become full after few calling (model.evaluate). How to release the GPU memory after each calling?
  2. Is there any better way to calculate the Accuracy vs Timesteps?

Thank you in advance!

Here is my code:

with nengo.Network(seed=42) as net:
net.config[nengo.Ensemble].max_rates = nengo.dists.Choice([100])
net.config[nengo.Ensemble].intercepts = nengo.dists.Choice([0])
net.config[nengo.Connection].synapse = None
neuron_type = nengo.LIF(amplitude=0.01)

nengo_dl.configure_settings(stateful=False)

input = nengo.Node(np.zeros(80*56))
x = nengo_dl.Layer(tf.keras.layers.Conv2D(32,3,2))(input,shape_in=(80,56,1))
x = nengo_dl.Layer(neuron_type)(x)

x = nengo_dl.Layer(tf.keras.layers.Conv2D(64,3,2))(x,shape_in=(39, 27, 32))
x = nengo_dl.Layer(neuron_type)(x)

x = nengo_dl.Layer(tf.keras.layers.Conv2D(128,3,2))(x,shape_in=(19, 13, 64))
x = nengo_dl.Layer(neuron_type)(x)

x = nengo_dl.Layer(tf.keras.layers.Conv2D(256,3,2))(x,shape_in=(9, 6, 128))
x = nengo_dl.Layer(neuron_type)(x)

output = nengo_dl.Layer(tf.keras.layers.Dense(num_classes))(x)
out_p = nengo.Probe(output,label='out_p')
out_p_filt = nengo.Probe(output,synapse=0.1,label='out_p_filt')

def classification_accuracy(y_true, y_pred):
return tf.metrics.sparse_categorical_accuracy(y_true[:, -1], y_pred[:, -1])

minibatch_size = 100
sim = nengo_dl.Simulator(net, minibatch_size=minibatch_size)
sim.load_params(“models/model_snn”)

#%% ACC vs Timesteps------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

sim.compile(loss={out_p_filt: classification_accuracy})

images = X_valid.reshape((X_valid.shape[0], -1))
ylabels = Y_valid

ACC = []
trange = range(1,31)

for n_steps in trange:
test_images = np.tile(images[:, None, :], (1, n_steps, 1))
test_labels = np.tile(ylabels[:, None, None], (1, n_steps, 1))

acc = sim.evaluate(test_images, {out_p_filt: test_labels}, verbose=1)["loss"]
print('Timesteps = ', n_steps, ' -> Accuracy = ', acc)
ACC.append(acc)

plt.figure(figsize=(8, 4))
plt.plot(trange, ACC,linewidth=2)
plt.legend(labels, loc=“right”)
plt.xlabel(“Timesteps”)
plt.ylabel(“Prediction Accuracy”)
plt.tight_layout()
plt.rc(‘font’, size=14)