I would like to run it with nengo_loihi.Simulator instead of nengo_dl.Simulator what would allow to execute converted keras models on the Loihi chip. I got an error “No neurons marked for execution on-chip. Please mark some ensembles as on-chip” because, as I understand, Nengo Loihi requires among others a presentation time in the model. Is there any possibility or work around which would allow to execute keras/tf models on the Loihi chip?
While creating a network it is mentioned in the example “At this point we could add any other Nengo components we like to the network, and connect them up to the Keras node (for example, if we wanted to take the classified image labels and use them as input to a spiking neural model)”.
Unfortunately it is not possible to run models written directly in TensorFlow/Keras in any simulators other than NengoDL. In order to be cross-compatible, a model has to be written in standard Nengo syntax.
So in order to run a Keras model on Loihi you would need to go through the manual process of building an equivalent Nengo model (using, e.g., nengo.Convolution), and then loading the trained weights from your Keras model into the Nengo model. It should certainly be possible to do that, but it will be a manual process.
nengo-extras does include code for automatically turning a Keras model into a Nengo model, but nengo-loihi has constraints on the number of neurons per ensemble (among others), and it’s unlikely the model generated by nengo-extras will satisfy all those constraints. In the future, we hope to make the process of breaking up large ensembles on Loihi more automatic, but currently you have to do it all manually. Putting convnets on Loihi is an area of active development for us, and we will share our tools for this as we develop them.