Im curious of the effect of arithmetic precision on the neuron models i.e. Nengo uses numpy which uses 64 bit float as default, what effect would changing this to 32 or even 16 bits have.
Ive tried changing everything to work with float32 however could never seem to get it running. If I did get it working I would compare turing curves produced when using float16/32/64 and see what the affect it has.
Any idea on how I could get this working or anyone has done anything similar?
Thanks in advance
Hi @ConorPQuinn, good question! The short answer is that most Nengo models are surprisingly robust to changing the precision of various quantities. You probably would not see anything significant in the tuning curves, though you will be some degraded performance in certain situations (e.g., small number of neurons).
If you want to play around with it, I made this pull request to try it out a few years ago… you can check it out and it should work, but it’s based on a very old version of Nengo so it might take some work to apply those changes to a recent version.
Hi @tbekolay,
Thanks I seem to have ran into the same failures you did with the NaN and infs when using float 16s.
I forgot to mention I was also curious to the affect of fixed point arithmetic on the models. I’m throwing a hail mary asking if you have had any history with this?
Would be very interesting as we could begin to evaluate these models running on different hardware i.e. FPGA with quick fixed point arithmetic hardware.
I’ve done a bit of work with fixed-point neuron models. In particular, the neuron model we use in nengo_spinnaker is fixed-point (S16.15), and that seems to be pretty indistinguishable from the normal nengo float64. However, we never got very far into doing a better job of optimizing the bit range. I made one attempt at starting to explore this here https://github.com/tcstewar/nengo_fixed/blob/master/fixed_ensemble.py but that’s outside of nengo and using a spiking rectified-linear unit, and I didn’t get very far into the analysis.