Hi Howard,
Good question. I’d definitely love to predict EEG and fMRI signals, and you’re entirely right that this would be a wonderful way to validate Nengo models. Most of the time when we’ve been validating Nengo models, we have been focusing on behavioural data (including reaction times) and on spiking activity, but we haven’t done much with anything else. There is one fMRI prediction in the 2013 book (based on the Tower of Hanoi model), and we are currently working on an MEG prediction model and have just started an EEG prediction (both unpublished yet).
The main reason, I think, that we haven’t done more of this is that Nengo/NEF models things at the level of spikes, and it’s unclear exactly how to map that spiking activity to fMRI/EEG/MEG/etc signals. For example, with fMRI, the fMRI signal is indicating the amount of energy consumption in different areas of the brain (assuming oxygenated blood flow correlates to energy consumption). This does not directly map onto spiking activity. For example, if area A has an inhibitory connection to area B, and if area A is firing a lot, then area B will not be spiking at all, but there will be a lot of energy consumption in area B as the inhibitory neurotransmitter that is being used has to get reabsorbed. Indeed, in the fMRI work in the 2013 book, we found the best fit to the fMRI data was if we assumed 80% of the fMRI signal was due to this neurotransmitter re-absorption (and the rest due to spiking activity). But that was fairly ad-hoc and more of a proof-of-concept. I think in order to strongly demonstrate fMRI as a validation method, we’d want a clear and principled method for figuring out the biological energy consumption requirements of a Nengo model. I’d love to sort that out, but haven’t made the time to do it yet.
A similar complication is true for EEG. The EEG signal should be generated by long-distance oriented connections, rather than by short internal recurrent connections. In many Nengo models, there’s isn’t that sort of large-scale modelling happening. However, my personal belief is that, in our SPA models, actions that cause the routing of information from one brain area to another would be good candidates for having visible EEG signals, as these will generally cause a lot of activity along axons that are fairly oriented. So my hope would be to find particular connections in a model that would be fairly long-distance, and when those neurons start firing (i.e. when they are released from inhibition via the b.g./thalamus) then that should correspond to an ERP of some sort. That’s my hope, in any case, and we’re just starting up a collaboration project that would start to explore that, but there are a lot of unknowns involved.
Right now, though, I do hope that LFP and MEG predictions might be a bit easier, as there’s reason to believe that those are a little bit more closely tied to just the spike data. Indeed, there’s a matlab toolkit https://github.com/richardtomsett/vertexsimulator for converting spike data to LFP data. We’re also working on an MEG prediction in work that should hopefully be published soon.
In any case, that’s the current state of validating nengo models using these more large-scale neural measures, rather than spikes. If there’s a particular project in there that really jumps out at you, I’d love to help out how I can; as I said I think you’re entirely right that this would be an excellent step forward for validation.
