There are a lot of factors that affect the amount of time a simulation on Loihi takes.
One of them is IO. If you have a lot of spikes going to the chip, or a lot of things on the chip you need to probe (to get output), this can slow things down. One way to work around this is by setting the biases of neurons on the chip to fire at the correct rates for a given input image, rather than sending spikes from the host to the chip.
Another source of slowness that we’ve recently discovered relates to sending spikes between chips. For example, on a Nahuku32 board, it is significantly slower to send spikes from one chip to another, than to send spikes within a chip. This is an active area of research, and one that we don’t have good solutions for yet. I’ve recently been working on a new allocator that tries to put the model on Loihi in a way that minimizes communication between chips. You could try that out and see if it helps (let me know if you find any bugs, I literally wrote it yesterday). If you really want to minimize inter-chip communication, you could look at extending that allocator with a more optimal algorithm for partitioning the model across chips (nxmetis might be a useful package for helping with this).
It is also possible that your
on_chip=False layer is responsible for some of this slowdown, as you suggest. If you’re using
precompute=True on your
Simulator (which I would highly recommend, as it should be faster), though, then everything happening on the host will execute first, and then Loihi will execute all timesteps together, with no involvement from the host (we “pre-compute” all the inputs to the network, i.e. your input spikes, by running that separately on the host, then run everything on Loihi with those pre-computed spikes). So if you’re using
precompute=True, then any times per timestep reported by NxSDK won’t be affected by having that layer off-chip.
Let us know if you have any success with improving your time per timestep; as I said, getting things to execute as fast as possible on Loihi is still a very active area of research!