Abstract from Haun, A., & Tononi, G. (2019). Why does space feel the way it does? Towards a principled account of spatial experience. Entropy, 21(12), 1160:

"There must be a reason why an experience feels the way it does. A good place to begin addressing this question is spatial experience, because it may be more penetrable by introspection than other qualities of consciousness such as color or pain. Moreover, much of experience is spatial, from that of our body to the visual world, which appears as if painted on an extended canvas in front of our eyes. Because it is ‘right there’, we usually take space for granted and overlook its qualitative properties. However, we should realize that a great number of phenomenal distinctions and relations are required for the canvas of space to feel ‘extended’. Here we argue that, to be experienced as extended, the canvas of space must be composed of countless spots, here and there, small and large, and these spots must be related to each other in a characteristic manner through connection, fusion, and inclusion. Other aspects of the structure of spatial experience follow from extendedness: every spot can be experienced as enclosing a particular region, with its particular location, size, boundary, and distance from other spots. We then propose an account of the phenomenal properties of spatial experiences based on integrated information theory (IIT). The theory provides a principled approach for characterizing both the quantity and quality of experience by unfolding the cause–effect structure of a physical substrate. Specifically, we show that a simple simulated substrate of units connected in a grid-like manner yields a cause–effect structure whose properties can account for the main properties of spatial experience. These results uphold the hypothesis that our experience of space is supported by brain areas whose units are linked by a grid-like connectivity. They also predict that changes in connectivity, even in the absence of changes in activity, should lead to a warping of experienced space. To the extent that this approach provides an initial account of phenomenal space, it may also serve as a starting point for investigating other aspects of the quality of experience and their physical correspondents."

To reference the content of this presentation, please cite Haun, A., & Tononi, G. (2019). Why does space feel the way it does? Towards a principled account of spatial experience. Entropy, 21(12), 1160.

Abstract from Song, C., Haun, A. M., & Tononi, G. (2017). Plasticity in the structure of visual space. Eneuro, 4(3):

"Visual space embodies all visual experiences, yet what determines the topographical structure of visual space remains unclear. Here we test a novel theoretical framework that proposes intrinsic lateral connections in the visual cortex as the mechanism underlying the structure of visual space. The framework suggests that the strength of lateral connections between neurons in the visual cortex shapes the experience of spatial relatedness between locations in the visual field. As such, an increase in lateral connection strength shall lead to an increase in perceived relatedness and a contraction in perceived distance. To test this framework through human psychophysics experiments, we used a Hebbian training protocol in which two-point stimuli were flashed in synchrony at separate locations in the visual field, to strengthen the lateral connections between two separate groups of neurons in the visual cortex. After training, participants experienced a contraction in perceived distance. Intriguingly, the perceptual contraction occurred not only between the two training locations that were linked directly by the changed connections, but also between the outward untrained locations that were linked indirectly through the changed connections. Moreover, the effect of training greatly decreased if the two training locations were too close together or too far apart and went beyond the extent of lateral connections. These findings suggest that a local change in the strength of lateral connections is sufficient to alter the topographical structure of visual space."

Abstract from Grasso, M., Haun, A. M., & Tononi, G. (2021). Of maps and grids. Neuroscience of Consciousness, 2021(2), niab022:

"Neuroscience has made remarkable advances in accounting for how the brain performs its various functions. Consciousness, too, is usually approached in functional terms: the goal is to understand how the brain represents information, accesses that information, and acts on it. While useful for prediction, this functional, information-processing approach leaves out the subjective structure of experience: it does not account for how experience feels. Here, we consider a simple model of how a “grid-like” network meant to resemble posterior cortical areas can represent spatial information and act on it to perform a simple “fixation” function. Using standard neuroscience tools, we show how the model represents topographically the retinal position of a stimulus and triggers eye muscles to fixate or follow it. Encoding, decoding, and tuning functions of model units illustrate the working of the model in a way that fully explains what the model does. However, these functional properties have nothing to say about the fact that a human fixating a stimulus would also “see” it—experience it at a location in space. Using the tools of Integrated Information Theory, we then show how the subjective properties of experienced space—its extendedness—can be accounted for in objective, neuroscientific terms by the “cause-effect structure” specified by the grid-like cortical area. By contrast, a “map-like” network without lateral connections, meant to resemble a pretectal circuit, is functionally equivalent to the grid-like system with respect to representation, action, and fixation but cannot account for the phenomenal properties of space."

Abstract from Haun, A., & Tononi, G. (2024, April 19). The unfathomable richness of seeing. PsyArxiv preprint: https://doi.org/10.31234/osf.io/jmg35:

"Most experts hold that visual experience is remarkably sparse and its apparent richness is illusory. Indeed, we fail to notice the vast majority of what we think we see, and seem to rely instead on a high-level summary of a visual scene. However, we argue here that seeing is much more than noticing, and is in fact unfathomably rich. We distinguish among three levels of visual phenomenology: a high-level description of a scene based on the categorization of “objects,” an intermediate level composed of “groupings” of simple visual features such as colors, and a base-level visual field composed of “spots” and their spatial relations. We illustrate that it is impossible to see the objects that underlie a high-level description without seeing the groupings that compose them, and we cannot see the groupings without seeing the visual field to which they are bound. We then argue that the way the visual field feels—its spatial extendedness—can only be accounted for by a phenomenal structure composed of innumerable distinctions and relations. It follows that most of what we see has no functional counterpart—it cannot be used, reported, or remembered. And yet we see it."

Abstract from Comolatti, R., Grasso, M., & Tononi, G. (2024). Why does time feel the way it does? Towards a principled account of temporal experience. arXiv preprint arXiv:2412.13198:

Time flows, or at least the time of our experience does. Can we provide an objective account of why experience, confined to the short window of the conscious present, encompasses a succession of moments that slip away from now to then--an account of why time feels flowing? Integrated Information Theory (IIT) aims to account for both the presence and quality of consciousness in objective, physical terms. Given a substrate's architecture and current state, the formalism of IIT allows one to unfold the cause-effect power of the substrate, yielding a cause-effect structure. According to IIT, this accounts in full for the presence and quality of experience, without any additional ingredients. In previous work, we showed how unfolding the cause-effect structure of non-directed grids, like those found in many posterior cortical areas, can account for the way space feels--namely, extended. Here we show that unfolding the cause-effect structure of directed grids can account for how time feels--namely, flowing. First, we argue that the conscious present is experienced as flowing because it is composed of phenomenal distinctions (moments) that are directed, and these distinctions are related in a way that satisfies directed inclusion, connection, and fusion. We then show that directed grids, which we conjecture constitute the substrate of temporal experience, yield a cause-effect structure that accounts for these and other properties of temporal experience. In this account, the experienced present does not correspond to a process unrolling in "clock time," but to a cause-effect structure specified by a system in its current state: time is a structure, not a process. We conclude by outlining similarities and differences between the experience of time and space, and some implications for the neuroscience, psychophysics, and philosophy of time.

Preview of abstract from Grasso & Tononi. (Forthcoming). Why do objects feel the way they do?

When we look at the world, we don’t only perceive specs of color, we see objects we have names for: we see three segments and recognize a letter “A”; we see two dots and a curved line and recognize a smiley face. Visual experience is typically characterized by sets of low-level features that are bound together to form objects we recognize—anything from geometric shapes to natural forms, animal shapes, letters, numbers, faces, tools, and so on. Previous work has applied the formalism of IIT to account for the extendedness of visual space (Haun & Tononi 2019) and the feeling of the flow of time (Comolatti, Grasso & Tononi Forthcoming). Here we develop an initial explanation of why objects feel the way they do: how phenomenal objects bind general concepts with particular features (conceptual invariance). 

First we characterize the phenomenology of objects, in which a particular configuration of low-level features (e.g., colors and edges) is experienced as an instance of a general concept (e.g., “face”), which is invariant for many possible configurations. We then apply the formalism of IIT to analyze a plausible neural substrate for conceptual invariance: pyramids of grids, similar to the ones found across levels of the visual cortex. We show how the binding of particular configurations to general concepts can be accounted for in physical terms by the cause–effect structure specified by pyramids of grids: by the causal distinctions (specified by conjunction and disjunction mechanisms) and  relations they form to compose a conceptual hierarchy.

Why aren’t the experiences of space and time considered axioms in IIT?