Training SNN with random search process

General Discussion
1

Hi guys, I’m training my neural network converted to SNN using nengo_dl. In my problem I need to obtain the best training parameters and so I decided to implement the random search process, since spiking networks do not have support for this… but I am getting the following result, is this normal to happen? What could be happening? Can anyone help me?

Image 01:

image
image1013×613 34.7 KB

Image 02:

image
image1005×610 45.4 KB

It seems that there are times when the random search works (second image) and other times when it doesn’t (first image)… does anyone know or has anyone experienced this?

Thank you.

2

I solved this problem by inserting a Batch Normalization layer into my SNN structure. If you have a problem with network convergence, insert this layer as it helps to minimize the effects of erroneously calculated error gradients across the layers.