There was another user who was working on this here Nengo RL performance with standard OpenAI cart pole problem, and a paper written about integrating Nengo and OpenAI here https://www.aaai.org/ocs/index.php/FLAIRS/FLAIRS19/paper/download/18312/17429. So those may provide a helpful starting point for your project.
The basic idea is that you use a nengo.Node to encapsulate the calls to OpenAI. A Node can execute arbitrary Python code, so as long as you can write the interaction with OpenAI as a function that receives a value (the input to the Node), does some computation, and then returns a value (the output of the Node), that’s basically all there is to interfacing an external library with Nengo.