How to remove I/O from Loihi on INRC superhost?

There is a significant amount of I/O that happens when I run code on the INRC superhost, which I suspect is seriously slowing down training time by over a factor of 10 (as compared to using regular Nengo on my laptop).

To run my code, I’m using SLURM=1 PARTITION=loihi_2h python -m name_of_python_file. Is there some option in that command I can use to reduce the verbosity?

Also, how can I view the energy usage and processing time of my network?

Hi @kshivvy, and welcome back to the Nengo forums. :smiley:

Regarding the I/O (or the messages that appear on the screen), those are part of the NxSDK debug messages, and Nengo does not have any control over their appearance. I believe you can control the output by setting the logging level of NxSDK. If you look in the NxSDK documentation: “Logging in NXNet” or the docs/nxlogging.html file, it seems that this will do it:

from nxsdk.logutils.nxlogging import set_verbosity, LoggingLevel
set_verbosity(LoggingLevel.ERROR)

As for the slow simulation speed, I would like to clarify how you are running your simulation. Are you using the sim.run() function or the sim.step() function, or perhaps something else entirely?

1 Like

Thanks for that code snippet! I’ll familiarize myself more with the NxSDK documentation.

I’m calling sim.run() repeatedly for training. Does that cause the model to have to be reloaded and reinitialized onto the board every time?

For NengoLoihi, yes, it does. If you can provide a snippet of your code (or explain how you perform the training process), perhaps I can suggest an alternative approach.