Hi,everyone!
I am a novice. In the process of learning nengo, I found that nengo was created based on NEF, and tensorflow can be used in nengoDL, so is nengoDL also created based on tensorflow? Is there a containment relationship between nengo and nengoDL?
I would be very grateful if you could answer my question！
Hello @YL0910! Welcome to the community!
Having some experience with Nengo and Nengo-DL I think I can take a glance at your question. You are mostly correct with your understanding. NEF, as the acronym stands for is a framework to model neural connections, neural communication etc. when attempting to simulate (in general) a cognitive task with the help of neuron models. As you might have noticed TensorFlow (TF) generally uses very abstract representation of neurons, namely ReLU, sigmoid etc, but NEF uses detailed computational models of biological neurons.
Nengo is a python library built on top of NEF and SPA to facilitate easy usage of the NEF and SPA theory. More details here and here. Coming to Nengo-DL, it is a simulator specifically designed for implementing and executing deep learning models with nengo objects, for example with spiking neurons. An obvious question would be… why have nengo-dl when nengo is already present? Well because nengo is a more general library for building general neural models (or even a brain model!) whereas Nengo-DL is specifically designed for deep learning tasks, e.g. image recognition, video recognition etc., hence built on top of TF to leverage its (i.e. tensorflow’s) environment.
I believe an example of use cases will make the difference between NEF, Nengo, Nengo-DL, and TF more apparent. If you want to build a spiking neuron model for working memory, you can use Nengo (which in backend would be using NEF principles to realize your model). If you want to train a model in TF environment and then want to test it out with spiking neurons, then use Nengo-DL. And yes… similar to Nengo-DL you have Nengo-Loihi, Nengo-FPGA which are simulators designed to be run on Loihi and FPGA boards. Hope this clarifies!
2 Likes
Thank you very much for your careful and patient answer! I’m currently organizing study notes for nengo, so some questions are still not clear: I know that SPA is the framework for building SPAUN. Is there a relationship between SPA and NEF? Does SPAUN also use NEF?
Hello @YL0910, I am not very much aware of the interaction between NEF and SPA; but I will try to explain my best. SPAUN happens to be implemented using Nengo and it uses both the theories of NEF and SPA. NEF is towards neural representation and computation. SPA is an architecture which uses semantic pointers and more for cognitive modelling, but to realize this architecture we use NEF as the computational substrate. May be @astoecke can help you more with it.
BTW, following links can be helpful (if not already gone through).
a) http://compneuro.uwaterloo.ca/research/spa/semantic-pointer-architecture.html
b) http://compneuro.uwaterloo.ca/files/publications/stewart.2012d.pdf
c) https://www.nengo.ai/nengo-spa/user-guide/spa-intro.html
Glad to see this being discussed @YL0910 and @zerone! There’s a lot going on in the Nengo ecosystem so we definitely struggle to explain where everything fits in.
Our main documentation page has a handy diagram putting certain parts of the Nengo ecosystem in different boxes:
It’s not strictly delineated though, as NengoDL (for example) has some core framework bits as well as a simulation backend. But, in general, I think of it this way:
- Nengo core is an API for building neural networks. It’s a set of lego-like building blocks that you can put together to describe your model. That model can be spiking or non-spiking, can have either direct neuron-to-neuron connections, NEF decoded connections, or both.
- That API is a high-level description of the model that isn’t tied to any hardware in particular. Hardware-specific implementations of those models are built in backends.
- Nengo core has a backend that uses NumPy to implement the model. Since NumPy runs on CPUs, the model runs on a CPU.
- NengoDL has a backend that uses TensorFlow to implement the model. Since TensorFlow runs on both CPUs and GPUs, you can use either with the NengoDL backend.
- NengoLoihi has a backend that uses NxSDK to implement the model. Since NxSDK runs on Loihi hardware, the model runs on Loihi.
Where it gets a little complicated is that NengoDL also can use TensorFlow to optimize the model, which is different from “running” the model, as we normally do with backends. In this way, Nengo and TensorFlow are similar because Nengo uses the NEF to “solve” for connection weight matrices, while TensorFlow uses optimizers and differentiation to learn connection weight matrices (and other model parameters).
As @zerone says, SPA is a cognitive modeling architecture that uses the NEF. It’s all part of the model definition, which is expressed in the frontend, and therefore can be run on any backend (like models using NEF connections). The separation between the Nengo API and the backend implementation is how Nengo is able to run large, complex models on many different types of hardware.
3 Likes