Reducing weight percision

Hi, I’m trying to experiment with different weight precisions for SNNs. I would like to convert the saved weights I have to 16-bit precision (float16) so that I can use it on the test set for inferencing. Is this possible? I can’t find any documentation on it, I already tried training using float16 and the model will not converge anymore so I figured this was the best option.

Hi @Ricko,

Can you provide some information about what you are using to train the network (Nengo? NengoDL?).

NengoDL

Thanks! I’ve forwarded your question to the NengoDL devs and I’ll update this thread when I get a response. :slight_smile:

Hi @Ricko,

I discussed this with the NengoDL devs, and we have a something you can try (in theory it should work, but haven’t actually tested to see if it works). If you want to have a network trained with one level of precision, then do the inference on another level of precision, it should be possible to do something like this:

  1. Train the model in NengoDL.
  2. Use the sim.save_params function to save the weights to file. This file is in an .npz format, so you should be able to parse it with Numpy. Note that the data in the .npz file isn’t labelled (it’s stored as arr0, arr1, etc.), so some work might be needed to identify what arrays are associated with which weights in the model.
  3. Use Numpy to convert the weights to whatever precision you want, then do the conversion again to get the numbers back into float32 (the conversion back may not actually be necessary).
  4. Save the modified data as an .npz file.
  5. Use sim.load_params to load the modified weights back into the NengoDL simulator.
  6. Run the inference on the modified model.

Let me know if the steps above work for you. :grinning:

It works, thanks!

1 Like

Woo! :raised_hands: