Adding a "Groups" parameter for Convolutions in Nengo-Loihi?

Backends Loihi
1

I want to add support for the groups parameter for nengo-loihi. (This would be a stepping stone in supporting DepthWiseConvolutions in nengo-loihi.)

I have gotten sufficiently far along in modifying the code that I can support a groups parameter in regular nengo and nengo-dl.

I believe that modifying this function here is the key to achieving this goal since it formats the convolutional weights for nengo-loihi. Does anybody have an pointers about how this function works or other pointers on how to modify nengo-loihi to support a groups parameter for 2d convolutions?

2

I’m not sure if it makes sense to use groups as a stepping stone for depth-wise convolutions; my guess is it’d be easier to just implement depth-wise convolutions (since that’s a special-case of the complete grouped implementation, with groups=1).

The function you want to modify is this one (I’m not sure if that’s the one you were looking at or not). The best way to understand how it works is by looking at the test for it. Specifically, the latter part of that test (from the conv2d_loihi_weights call and beyond) actually takes the outputs of the function and applies them to an image, so you can see how they’re all being used.

I think you can actually keep a lot of conv2d_loihi_weights the same; the main part you want to modify is this last if block. Currently, it adds entries to weights and indices that have shape (n_channels, len(row_inds) * len(col_inds) * n_filters); furthermore, the indices are the same for every channel. What you want (assuming you’re just implementing depth-wise convolution) is for these entries to have shape (n_channels, len(row_inds) * len(col_inds)), and for the indices to be different for every channel to target a different output channel for each input channel.

1 Like
3

I think I have had some success implementing a groups convolution in nengo-loihi. I started off by considering the most simple case which is the depthwise case as you mentioned.

However, to fully incorporate this change into nengo-loihi it is necessary to edit several classes and helper functions in nengo such as nengo._vendor.npconv2d.conv2d.conv_2d, nengo._vendor.npconv2d.conv2d.extract_sliding_windows
as well as various Transform classes and Builder Functions.

Do I need any special permissions to create a feature branch in nengo, nengo-dl, or nengo-loihi so I can share this result?

4

Hi @mjurado3,

Yeah, you need to be part of the Nengo GitHub organization to create a feature branch on one of the Nengo repositories. For external (non-ABR) users, we advise that you create a fork of the repo, make a feature branch in said fork, then make a pull request (in the original Nengo repo) with the forked code (see the GitHub docs here).

1 Like
5

I just submitted a pull request for the nengo about grouped convolution support. (The delay was caused by editing the unit tests).

I also have a fork for nengo-loihi with one modification.
But sadly the unit tests do not pass when groups > 1 and channels_last = False. If anyone has any recommendations please feel free to reply to this thread.

If anyone else has been wondering how depthwise convolutions could look in nengo see this example:
depthwise-convolution-nengo (94.9 KB)

Finally, thank you @Eric and @xchoo for the help

6

I thought I would revive this thread once more to mention that I edited a few more files to make this change. Specifically split_block.py.

The code still fails when groups > 1 and channels_last = False though.

1 Like