Hi iam a newbie here And i am interested to create a Spiking GAN -Generative adversarial Network using spiking neurons instead Can we do that using Nengo
Hi @vishwak1002, welcome to the Nengo forum!
Implementing a spiking GANs is definitely possible with Nengo, though it is not as straightforward as models that have been around longer (like CNNs).
Generally, the main thing that you should look into is NengoDL, which is how we have been using Nengo for most machine learning tasks. Depending on your familiarity with various tools, you might implement the spiking GAN by taking a normal GAN written with keras and converting it with NengoDL. You could also implement it directly in the Nengo API. If you do have familiarity with TensorFlow, then this example should be helpful; if not, you can go straight to an example that doesn’t assume any particular background (e.g., spiking MNIST).
The main thing that is missing is a transposed convolution (deconvolution) operator. @Eric has been doing some work with GANs and in order to implement it with Nengo, he needed to write a custom transform, which he’s let me share here: convolution_transpose.py (5.8 KB)
Hopefully that’s enough to get you started. As you run into issues along the way, feel free to post more specific questions here on the forum and we’ll be able to help further!
1 Like
Thank you sir It was really helpful. iam also curious can we do tasks such as image inpainting or voice generation which we usually do using Normal second generations ANNS
And I have one more question where can we learn theoritically about spiking neuron model…Because it looks like for me like some black box…And i wanna know whats happening in the black box
Yes, while Nengo is particularly well suited for spiking neural networks, you can do anything that you do with normal ANNs with Nengo. On our main examples page the “Standard Neural Networks” section can give you some idea of how that would work, though for larger models like those used for inpainting and voice generation, I suspect you will be using the same NengoDL and Keras conversion tools that I mentioned in my last post.
A lot of spiking neural networks (particularly those that use STDP or other low-level learning rules) are black boxes. The NEF-style networks that Nengo can produce, however, are based on three principles that make clear how neurons represent and transform information dynamically. This technical overview of the NEF is a good starting point for learning about the NEF, as is this recently produced video series.