Thank you for your detailed answer. This is exactly what I was worried about :). I have a paper on a deep reinforcement learning model for some robotic project and I wanted to try to convert that paper to SNN (with Nengo).
In that paper, they use SAC (Soft Actor-Critic) network which is a relatively complex model for conversion, which is implemented as DNN. I even already tried to convert a much simpler model here: Converting a simple ANN to SNN - General Discussion - Nengo forum without success.
I know PES, but I don’t think it is much powerful as a deep model. PES can only learn one connection while deep learning learns all the connections in the network every step. So I believe that if I compare the result of the deep SAC model and models with PES. The deep SAC will perform much better. (Maybe I wrong, but that is my feeling). Here is another question that I’ve asked about that dilemma: Hierarchy reinforcement learning vs NengoDL - General Discussion - Nengo forum
So here are my questions:
Do you think that PES can perform as well as DNN?
Can PES replace any RL model out there (DQN, DDQN, SAC, etc.)?
Is there any example of how to implement “big” RL models (DQN, DDQN, SAC, etc.) directly in Nengo (if that thing is necessary…)