Open preprint reviews by Tal Yarkoni

What the Success of Brain Imaging Implies about the Neural Code

Olivia Guest, Bradley C Love

Review posted on 24th August 2016

This is an innovative and very thought-provoking paper that will hopefully be widely read by researchers working with fMRI. I have two general comments with respect to the authors' main thesis:

1. As far as I can tell, the authors don't motivate the decision to focus exclusively on sub-voxel representations. They point out that non-smooth sub-voxel representations would be impossible to detect with fMRI, which is an important observation. But surely non-smooth *supra-voxel* representations would still be easily detectable with fMRI. A priori, there doesn't seem to be a good reason to rule out this kind of representation in the brain. As far as I can tell, representational similarity analyses would still work successfully if the brain were composed of hundreds of functionally discrete tiles that were non-smooth at both the sub-voxel and supra-voxel levels. This doesn't seem like a far-fetched possibility; for example, suppose that when people think about penguins, they're somewhat more likely to think about the unusual climate in which penguins live. Representations of climate may be non-smooth, yet reside in fundamentally different brain circuits from representations of physical shape, size, etc. One consequence would be that neural representations of robins would almost certainly more closely resemble those of sparrows than those of penguins even if there were no spatially graded sub-voxel representations at all in the human brain--simply in virtue of sharing a larger number of salient properties with the former than the latter. Of course, I'm not suggesting that there _aren't_ smooth sub-voxel representations in the brain, but simply that the authors conclusion that "the neural code must be smooth, both at the subvoxel and functional levels" doesn't necessarily follow.

2. Even if one assumes that the signal detected by fMRI is in fact driven entirely by smooth sub-voxel representations, it still wouldn't follow that the neural code must be smooth at the sub-voxel level. All we would be able to conclude is that there is at least *some* component of the signal that is smooth. This would not preclude other neural codes from existing, and in fact, we already have abundant evidence of non-smooth sub-voxel representations. For example, ocular dominance columns clearly exist, and if fMRI is unable to detect them, that reflects a limitation of fMRI, not a generalizable claim about the way the brain represents information. While I'm not a systems neuroscientist, I would imagine that there are any number of examples in the systems neuroscience literature of non-smooth, but highly structured sub-voxel representations that would probably be completely undetectable with fMRI. So I think the authors may want to be more circumspect about the conclusions they draw. Their results don't really show that only a subset of neural coding schemes are plausible; rather they suggest that whatever neural representations fMRI is capable of detecting are likely to stem from either (a) smooth representations (either sub- or supra-voxel) or (b) non-smooth supra-voxel representations. This leaves open the possibility (and it seems like a very real one) that the vast majority of information represented in the brain is not represented in a way that is amenable to detection with fMRI.

Setting these concerns aside, I think this is still a paper that should be of great interest to most cognitive neuroscientists. One point that is made very elegantly here is that the nature of neural representations does not have to be (and probably isn't) uniform across the brain. In particular, the authors put forward a compelling argument for the possibility that brain regions higher in the processing stream--and that are more likely to represent very abstract, multidimensional information--may not be amenable to imaging at all. This point should give many fMRI researchers pause when considering studying, e.g., the representational structure of prefrontal cortex. At the very least, the manuscript raises a number of important questions that should spur further discussion within the neuroimaging community.

show less