Handling of aedat2.0 DVS format

General Discussion
1

Dear all,

I am pretty much new in terms of Nengo usage, hopefully somebody in this forum have had the experiences with various DVS (Dynamic Vision Sensor) file formats.

Namely, I have tried to to read the MNIST-DVS events by usage of “nengo_loihi.dvs.DVSFileChipProcess” as it has been shown in the example.
However, the events read by the “DVSFileChipProcess” always missing y-axis which is set to zero which should not be the case:

Event: 15247
x: 4
y: 0
t: 1240570
p: 0
v: 0

Event: 15248
x: 4
y: 0
t: 1240581
p: 1
v: 0

Event: 15249
x: 4
y: 0
t: 1240590
p: 0
v: 0

Since DVS-MNIST is in aedat2.0 format and Nengo implementation supports the same format where is the mismatch?

a) The Nengo aedat2.0 implementation

b) Here is another implementation of the dataloader which provides
which provides good output, for example:
Evnt: 15000
x: 66
y: 69
t: 1214176
p: 1.0

Evnt: 16500
x: 75
y: 55
t: 1307373
p: -1.0

It seems to me that in current Nengo impelmentation for aedat2.0 DVS format
contains additional fields apart of (y-axis,x-axis,p-polarity,t-time stamp) which are not present in
aedat2.0 format, am I right? And what would be the remedy for such situation?

fields = [
(“type”, ctypes.c_uint64, 1),
(“y”, ctypes.c_uint64, 9),
(“x”, ctypes.c_uint64, 10),
(“polarity”, ctypes.c_uint64, 1),
(“trigger”, ctypes.c_uint64, 1),
(“adc_sample”, ctypes.c_uint64, 10),
(“t”, ctypes.c_uint64, 32),
]

def etuple(e):
return (e.y, e.x, e.polarity, e.trigger, e.t)

Thank you,
With best ragards,
GK.

2

AEDAT 2.0 is a file format that stores 64-bit events, but the meaning of those bits depends on the device used to record the data. This is what’s specified by AEDatEvent._fields_. The implemented _fields_ is for a DAVIS event, as specified here.

However, as that document states,

the address has to be interpreted according to a specific jAER AEChip class’ definition of that address.
So if you’re using e.g. DVS128 (instead of DAVIS), then you’d need to use the DVS128 address format. It’s 16-bit, so I’m not quite sure how it’s aligned within the 32-bits available for address in the AEDAT 2.0 file format.

Basically, you’ll have to modify the _fields_ definition to match whatever device you’re using. For DVS128, it might look something like this:

        _fields_ = [
            ("unused", ctypes.c_uint64, 16),
            ("trigger", ctypes.c_uint64, 1),
            ("y", ctypes.c_uint64, 7),
            ("x", ctypes.c_uint64, 7),
            ("polarity", ctypes.c_uint64, 1),
            ("t", ctypes.c_uint64, 32),
        ]
3

Thanks Eric for the effort, link to the relevant information and implementation suggestion. I’ll experiment a bit with it and post the outcome on the forum.
With best regards,
Goran

4

The modification mentioned above works well for DVS128, meaning that the class “DVSFileChipProcess” with the modifications executed as standalone code in the notebook correctly reads out DVS MNIST frames.

How to incorporate these modifications in the nengo_loihi framework is it by usage of the builders or is there any other way arround?
Many thanks in advance!
G.

5

The latest implementation question is, in the mean time, resolved as follows:

a) In the “nengo_loihi” folder one should find the “dvs.py” python source code file where the Eric’s modification should be implemented.
b) Position the DOS prompt and compile the “dvs.py” file by the command (for example):
python -mmy_compile dvs.py
c) The binary code shall be placed in the “pycache” folder under the name “dvs.cphyton-37.pyc”.
The number “37” might differ, depends on the python version available on your system.
d) Restart the “Anaconda” framework so that the implementation changes take effect otherwise the cached version of the previous “dvs.py” shall be used.

Now the N-MNIST DVS files should be properly read out in the python notebook scripts.
With best regards,
Goran