mental representations definition quizlet

Bibliography

Academic tools.

• Friends PDF Preview
• Author and Citation Info
• Back to Top

Mental Representation

The notion of a “mental representation” is, arguably, in the first instance a theoretical construct of cognitive science. As such, it is a basic concept of the Computational Theory of Mind, according to which cognitive states and processes are constituted by the occurrence, transformation and storage (in the mind/brain) of information-bearing structures (representations) of one kind or another.

However, on the assumption that a representation is an object with semantic properties (content, reference, truth-conditions, truth-value, etc.), a mental representation may be more broadly construed as a mental object with semantic properties. As such, mental representations (and the states and processes that involve them) need not be understood only in computational terms. On this broader construal, mental representation is a philosophical topic with roots in antiquity and a rich history and literature predating the recent “cognitive revolution,” and which continues to be of interest in pure philosophy. Though most contemporary philosophers of mind acknowledge the relevance and importance of cognitive science, they vary in their degree of engagement with its literature, methods and results; and there remain, for many, issues concerning the representational properties of the mind that can be addressed independently of the computational hypothesis.

Though the term ‘Representational Theory of Mind’ is sometimes used almost interchangeably with ‘Computational Theory of Mind’, I will use it here to refer to any theory that postulates the existence of semantically evaluable mental objects, including philosophy's stock in trade mentalia — thoughts, concepts, percepts, ideas, impressions, notions, rules, schemas, images, phantasms, etc. — as well as the various sorts of “subpersonal” representations postulated by cognitive science. Representational theories may thus be contrasted with theories, such as those of Baker (1995), Collins (1987), Dennett (1987), Gibson (1966, 1979), Reid (1764/1997), Stich (1983) and Thau (2002), which deny the existence of such things.

1. The Representational Theory of Mind

2. propositional attitudes.

• 3. Conceptual and Nonconceptual Representation

4. Representationalism and Phenomenalism

6. content determination, 7. internalism and externalism, 8. the computational theory of mind, 9. thought and language, other internet resources, related entries.

The Representational Theory of Mind (RTM) (which goes back at least to Aristotle) takes as its starting point commonsense mental states, such as thoughts, beliefs, desires, perceptions and imagings. Such states are said to have “intentionality” — they are about or refer to things, and may be evaluated with respect to properties like consistency, truth, appropriateness and accuracy. (For example, the thought that cousins are not related is inconsistent, the belief that Elvis is dead is true, the desire to eat the moon is inappropriate, a visual experience of a ripe strawberry as red is accurate, an imaging of George W. Bush with dreadlocks is inaccurate.)

RTM defines such intentional mental states as relations to mental representations, and explains the intentionality of the former in terms of the semantic properties of the latter. For example, to believe that Elvis is dead is to be appropriately related to a mental representation whose propositional content is that Elvis is dead . (The desire that Elvis be dead, the fear that he is dead, the regret that he is dead, etc., involve different relations to the same mental representation.) To perceive a strawberry is, on the representational view, to have a sensory experience of some kind which is appropriately related to (e.g., caused by) the strawberry.

RTM also understands mental processes such as thinking, reasoning and imagining as sequences of intentional mental states. For example, to imagine the moon rising over a mountain is, inter alia , to entertain a series of mental images of the moon (and a mountain). To infer a proposition q from the propositions p and if p then q is ( inter alia ) to have a sequence of thoughts of the form p , if p then q , q .

Contemporary philosophers of mind have typically supposed (or at least hoped ) that the mind can be naturalized — i.e., that all mental facts have explanations in the terms of natural science. This assumption is shared within cognitive science, which attempts to provide accounts of mental states and processes in terms (ultimately) of features of the brain and central nervous system. In the course of doing so, the various sub-disciplines of cognitive science (including cognitive and computational psychology and cognitive and computational neuroscience) postulate a number of different kinds of structures and processes, many of which are not directly implicated by mental states and processes as commonsensically conceived. There remains, however, a shared commitment to the idea that mental states and processes are to be explained in terms of mental representations.

In philosophy, recent debates about mental representation have centered around the existence of propositional attitudes (beliefs, desires, etc.) and the determination of their contents (how they come to be about what they are about), and the existence of phenomenal properties and their relation to the content of thought and perceptual experience. Within cognitive science itself, the philosophically relevant debates have been focused on the computational architecture of the brain and central nervous system, and the compatibility of scientific and commonsense accounts of mentality.

Intentional Realists such as Dretske (e.g., 1988) and Fodor (e.g., 1987) note that the generalizations we apply in everyday life in predicting and explaining each other's behavior (often collectively referred to as “folk psychology”) are both remarkably successful and indispensable. What a person believes, doubts, desires, fears, etc. is a highly reliable indicator of what that person will do; and we have no other way of making sense of each other's behavior than by ascribing such states and applying the relevant generalizations. We are thus committed to the basic truth of commonsense psychology and, hence, to the existence of the states its generalizations refer to. (Some realists, such as Fodor, also hold that commonsense psychology will be vindicated by cognitive science, given that propositional attitudes can be construed as computational relations to mental representations.)

Intentional Eliminativists , such as Churchland, (perhaps) Dennett and (at one time) Stich argue that no such things as propositional attitudes (and their constituent representational states) are implicated by the successful explanation and prediction of our mental lives and behavior. Churchland denies that the generalizations of commonsense propositional-attitude psychology are true. He (1981) argues that folk psychology is a theory of the mind with a long history of failure and decline, and that it resists incorporation into the framework of modern scientific theories (including cognitive psychology). As such, it is comparable to alchemy and phlogiston theory, and ought to suffer a comparable fate. Commonsense psychology is false , and the states (and representations) it postulates simply don't exist. (It should be noted that Churchland is not an eliminativist about mental representation tout court . See, e.g., Churchland 1989.)

Dennett (1987a) grants that the generalizations of commonsense psychology are true and indispensable, but denies that this is sufficient reason to believe in the entities they appear to refer to. He argues that to give an intentional explanation of a system's behavior is merely to adopt the “intentional stance” toward it. If the strategy of assigning contentful states to a system and predicting and explaining its behavior (on the assumption that it is rational — i.e., that it behaves as it should, given the propositional attitudes it should have, given its environment) is successful, then the system is intentional, and the propositional-attitude generalizations we apply to it are true. But there is nothing more to having a propositional attitude than this. (See Dennett 1987a: 29.)

Though he has been taken to be thus claiming that intentional explanations should be construed instrumentally, Dennett (1991) insists that he is a “moderate” realist about propositional attitudes, since he believes that the patterns in the behavior and behavioral dispositions of a system on the basis of which we (truly) attribute intentional states to it are objectively real. In the event that there are two or more explanatorily adequate but substantially different systems of intentional ascriptions to an individual, however, Dennett claims there is no fact of the matter about what the individual believes (1987b, 1991). This does suggest an irrealism at least with respect to the sorts of things Fodor and Dretske take beliefs to be; though it is not the view that there is simply nothing in the world that makes intentional explanations true.

(Davidson 1973, 1974 and Lewis 1974 also defend the view that what it is to have a propositional attitude is just to be interpretable in a particular way. It is, however, not entirely clear whether they intend their views to imply irrealism about propositional attitudes.)

Stich (1983) argues that cognitive psychology does not (or, in any case, should not) taxonomize mental states by their semantic properties at all, since attribution of psychological states by content is sensitive to factors that render it problematic in the context of a scientific psychology. Cognitive psychology seeks causal explanations of behavior and cognition, and the causal powers of a mental state are determined by its intrinsic “structural” or “syntactic” properties. The semantic properties of a mental state, however, are determined by its extrinsic properties — e.g., its history, environmental or intramental relations. Hence, such properties cannot figure in causal-scientific explanations of behavior. (Fodor 1994 and Dretske 1988 are realist attempts to come to grips with some of these problems.) Stich proposes a syntactic theory of the mind, on which the semantic properties of mental states play no explanatory role. (Stich has since changed his views on a number of these issues. See Stich 1996.)

3. Conceptual and Non-Conceptual Representation

It is a traditional assumption among realists about mental representations that representational states come in two basic varieties (cf. Boghossian 1995). There are those, such as thoughts, which are composed of concepts and have no phenomenal (“what-it's-like”) features (“qualia”), and those, such as sensations, which have phenomenal features but no conceptual constituents. (Nonconceptual content is usually defined as a kind of content that states of a creature lacking concepts might nonetheless enjoy. [ 1 ] ) On this taxonomy, mental states can represent either in a way analogous to expressions of natural languages or in a way analogous to drawings, paintings, maps, photographs or movies. Perceptual states such as seeing that something is blue, are sometimes thought of as hybrid states, consisting of, for example, a non-conceptual sensory experience and a belief, or some more integrated compound of conceptual and nonconceptual elements. (There is an extensive literature on the representational content of perceptual experience. See the entry on the contents of perception .)

Disagreement over nonconceptual representation concerns the existence and nature of phenomenal properties and the role they play in determining the content of sensory experience. Dennett (1988), for example, denies that there are such things as qualia at all (as they are standardly construed); while Brandom (2002), McDowell (1994), Rey (1991) and Sellars (1956) deny that they are needed to explain the content of sensory experience. Among those who accept that experiences have phenomenal content, some (Dretske, Lycan, Tye) argue that it is reducible to a kind of intentional content, while others (Block, Loar, Peacocke) argue that it is irreducible. (See the discussion in the next section.)

Some historical discussions of the representational properties of mind (e.g., Aristotle De Anima , Locke 1689/1975, Hume 1739/1978) seem to assume that nonconceptual representations — percepts (“impressions”), images (“ideas”) and the like — are the only kinds of mental representations, and that the mind represents the world in virtue of being in states that resemble things in it. On such a view, all representational states have their content in virtue of their phenomenal features. Powerful arguments, however, focusing on the lack of generality (Berkeley Principles of Human Knowledge ), ambiguity (Wittgenstein 1953) and non-compositionality (Fodor 1981c) of sensory and imagistic representations, as well as their unsuitability to function as logical (Frege 1918/1997, Geach 1957) or mathematical (Frege 1884/1953) concepts, and the symmetry of resemblance (Goodman 1976), convinced philosophers that no theory of mind can get by with only nonconceptual representations construed in this way.

There has also been dissent from the traditional claim that conceptual representations (thoughts, beliefs) lack phenomenology. Chalmers (1996), Flanagan (1992), Goldman (1993), Horgan and Tienson (2002), Jackendoff (1987), Levine (1993, 1995, 2001), McGinn (1991a), Pitt (2004, 2009, 2011, Forthcoming), Searle (1992), Siewert (1998) and Strawson (1994), claim that purely conceptual (conscious) representational states themselves have a (perhaps proprietary) phenomenology. (This view — bread and butter, it should be said, among historical and contemporary Phenomenologists — has been gaining momentum of late among analytic philosophers of mind. See, e.g., the essays in Bayne and Montague 2011 and in Kriegel Forthcoming, Farkas 2008 and Kriegel 2011.) If this claim is correct, the question of what role phenomenology plays in the determination of content rearises for conceptual representation; and the eliminativist ambitions of Sellars, Brandom, Rey, et al. would meet a new obstacle. (It would also raise prima face problems for reductivist representationalism (see the next section), as well as for reductive naturalistic theories of intentional content.)

Among realists about phenomenal properties, the central division is between representationalists (also called “representationists” and “intentionalists”) — e.g., Dretske (1995), Harman (1990), Leeds (1993), Lycan (1987, 1996), Rey (1991), Thau (2002), Tye (1995, 2000, 2009) — and phenomenalists (also called “phenomenists”) — e.g., Block (1996, 2003), Chalmers (1996,2004), Evans (1982), Loar (2003a, 2003b), Peacocke (1983, 1989, 1992, 2001), Raffman (1995), Shoemaker (1990). Representationalists claim that the phenomenal character of a mental state is reducible to a kind of intentional content, naturalistically construed (a la Dretske). Phenomenalists claim that the phenomenal character of a mental state is not so reducible.

The representationalist thesis is often formulated as the claim that phenomenal properties are representational or intentional. However, this formulation is ambiguous between a reductive and a non-reductive claim (though the term ‘representationalism’ is most often used for the reductive claim). (See Chalmers 2004a.) On one hand, it could mean that the phenomenal content of an experience is a kind of intentional content (i.e., the objective qualitative properties it represents). On the other, it could mean that the intrinsic, subjective phenomenal properties of an experience determine an intentional content. Representationalists such as Dretske, Lycan and Tye would assent to the former claim, whereas phenomenalists such as Block, Chalmers, Loar and Peacocke would assent to the latter. (Among phenomenalists, there is further disagreement about whether qualia are intrinsically representational (Loar) or not (Block, Peacocke). (So-called “Ganzfeld” experiences, in which, for example, the visual field is completely taken up with a uniform experience of a single color, are a standard test case: Do Ganzfeld experiences represent anything? It may be that doubts about the representationality of such experiences is simply a consequence of the fact that (outside the laboratory) we never encounter things that would produce them. Supposing we routinely did (and especially if we had names for them), it seems unlikely such skepticism would arise.)

Most (reductive) representationalists are motivated by the conviction that one or another naturalistic explanation of intentionality (see the next section) is, in broad outline, correct, and by the desire to complete the naturalization of the mental by applying such theories to the problem of phenomenality. (Needless to say, most phenomenalists (Chalmers is the major exception) are just as eager to naturalize the phenomenal — though not in the same way.)

The main argument for representationalism appeals to the transparency of experience (cf. Tye 2000: 45–51). The properties that characterize what it's like to have a perceptual experience are presented in experience as properties of objects perceived: in attending to an experience, one seems to “see through it” to the objects and properties it is experiences of . [ 2 ] They are not presented as properties of the experience itself. If nonetheless they were properties of the experience, perception would be massively deceptive. But perception is not massively deceptive. According to the representationalist, the phenomenal character of an experience is due to its representing objective, non-experiential properties. (In veridical perception, these properties are locally instantiated; in illusion and hallucination, they are not.) On this view, introspection is indirect perception: one comes to know what phenomenal features one's experience has by coming to know what objective features it represents. (Cf. also Dretske 1996, 1999.)

In order to account for the intuitive differences between conceptual and sensory representations, representationalists appeal to structural or functional properties. Dretske (1995), for example, distinguishes experiences and thoughts on the basis of the origin and nature of their functions: an experience of a property P is a state of a system whose evolved function is to indicate the presence of P in the environment; a thought representing the property P , on the other hand, is a state of a system whose assigned (learned) function is to calibrate the output of the experiential system. Rey (1991) takes both thoughts and experiences to be relations to sentences in the language of thought, and distinguishes them on the basis of (the functional roles of) such sentences' constituent predicates. Lycan (1987, 1996) distinguishes them in terms of their functional-computational profiles. Tye (2000) distinguishes them in terms of their functional roles and the intrinsic structure of their vehicles: thoughts are representations in a language-like medium, whereas experiences are image-like representations consisting of “symbol-filled arrays.” (Cf. the account of mental images in Tye 1991.)

Phenomenalists tend to make use of the same sorts of features (function, intrinsic structure) in explaining some of the intuitive differences between thoughts and experiences; but they do not suppose that such features exhaust the differences between phenomenal and non-phenomenal representations. For the phenomenalist, it is the phenomenal properties of experiences — qualia themselves — that constitute the fundamental difference between experience and thought. Peacocke (1992), for example, develops the notion of a perceptual “scenario” (an assignment of phenomenal properties to coordinates of a three-dimensional egocentric space), whose content is “correct” (a semantic property) if in the corresponding “scene” (the portion of the external world represented by the scenario) properties are distributed as their phenomenal analogues are in the scenario.

Another sort of representation appealed to by some phenomenalists (e.g., Chalmers (2003), Block (2003)) is what Chalmers calls a “pure phenomenal concept.” A phenomenal concept in general is a concept whose denotation is a phenomenal property, and it may be discurive (‘the color of ripe bananas‘), demonstrative (‘ this color’; Loar 1996)), or even more direct. On Chalmers's view, a pure phenomenal concept is (something like) a conceptual/phenomenal hybrid consisting of a phenomenological “sample” (an image or an occurrent sensation) integrated with (or functioning as) a conceptual component. Phenomenal concepts are postulated to account for the apparent fact (among others) that, as McGinn (1991b) puts it, “you cannot form [introspective] concepts of conscious properties unless you yourself instantiate those properties.” One cannot have a phenomenal concept of a phenomenal property P , and, hence, phenomenal beliefs about P , without having experience of P , because P itself is (in some way) constitutive of the concept of P . (Cf. Jackson 1982, 1986 and Nagel 1974.) (Chalmers (2004b) puts pure phenomenal concepts to use in defending the Knowledge Argument against physicalism. Alter and Walter 2007 is an excellent collection of essays on phenomenal concepts.)

Though imagery has played an important role in the history of philosophy of mind, the important contemporary literature on it is primarily psychological. (McGinn 2004 is a notable recent exception.) In a series of psychological experiments done in the 1970s (summarized in Kosslyn 1980 and Shepard and Cooper 1982), subjects' response time in tasks involving mental manipulation and examination of presented figures was found to vary in proportion to the spatial properties (size, orientation, etc.) of the figures presented. The question of how these experimental results are to be explained kindled a lively debate on the nature of imagery and imagination.

Kosslyn (1980) claims that the results suggest that the tasks were accomplished via the examination and manipulation of mental representations that themselves have spatial properties — i.e., pictorial representations, or images . Others, principally Pylyshyn (1979, 1981a, 1981b, 2003), argue that the empirical facts can be explained in terms exclusively of discursive , or propositional representations and cognitive processes defined over them. (Pylyshyn takes such representations to be sentences in a language of thought.)

The idea that pictorial representations are literally pictures in the head is not taken seriously by proponents of the pictorial view of imagery (see, e.g., Kosslyn and Pomerantz 1977). The claim is, rather, that mental images represent in a way that is relevantly like the way pictures represent. (Attention has been focused on visual imagery — hence the designation ‘pictorial’; though of course there may be imagery in other modalities — auditory, olfactory, etc. — as well. See O'Callaghan 2007 for discussion of auditory imagery.)

The distinction between pictorial and discursive representation can be characterized in terms of the distinction between analog and digital representation (Goodman 1976). This distinction has itself been variously understood (Fodor & Pylyshyn 1981, Goodman 1976, Haugeland 1981, Lewis 1971, McGinn 1989), though a widely accepted construal is that analog representation is continuous (i.e., in virtue of continuously variable properties of the representation), while digital representation is discrete (i.e., in virtue of properties a representation either has or doesn't have) (Dretske 1981). (An analog/digital distinction may also be made with respect to cognitive processes . (Block 1983.)) On this understanding of the analog/digital distinction, imagistic representations, which represent in virtue of properties that may vary continuously (such as being more or less bright, loud, vivid, etc.), would be analog, while conceptual representations, whose properties do not vary continuously (a thought cannot be more or less about Elvis: either it is or it is not) would be digital.

It might be supposed that the pictorial/discursive distinction is best made in terms of the phenomenal/nonphenomenal distinction, but it is not obvious that this is the case. For one thing, there may be nonphenomenal properties of representations that vary continuously. Moreover, there are ways of understanding pictorial representation that presuppose neither phenomenality nor analogicity. According to Kosslyn (1980, 1982, 1983), a mental representation is “quasi-pictorial” when every part of the representation corresponds to a part of the object represented, and relative distances between parts of the object represented are preserved among the parts of the representation. But distances between parts of a representation can be defined functionally rather than spatially — for example, in terms of the number of discrete computational steps required to combine stored information about them. (Cf. Rey 1981.)

Tye (1991) proposes a view of images on which they are hybrid representations, consisting both of pictorial and discursive elements. On Tye's account, images are “(labeled) interpreted symbol-filled arrays.” The symbols represent discursively, while their arrangement in arrays has representational significance (the location of each “cell” in the array represents a specific viewer-centered 2-D location on the surface of the imagined object).

The contents of mental representations are typically taken to be abstract objects (properties, relations, propositions, sets, etc.). A pressing question, especially for the naturalist, is how mental representations come to have their contents. Here the issue is not how to naturalize content (abstract objects can't be naturalized), but, rather, how to specify naturalistic content-determining relations between mental representations and the abstract objects they express. There are two basic types of contemporary naturalistic theories of content-determination, causal-informational and functional . [ 3 ]

Causal-informational theories (Dretske 1981, 1988, 1995) hold that the content of a mental representation is grounded in the information it carries about what does (Devitt 1996) or would (Fodor 1987, 1990a) cause it to occur. [ 4 ] There is, however, widespread agreement that causal-informational relations are not sufficient to determine the content of mental representations. Such relations are common, but representation is not. Tree trunks, smoke, thermostats and ringing telephones carry information about what they are causally related to, but they do not represent (in the relevant sense) what they carry information about. Further, a representation can be caused by something it does not represent, and can represent something that has not caused it.

The main attempts to specify what makes a causal-informational state a mental representation are Asymmetric Dependency Theories (e.g., Fodor 1987, 1990a, 1994) and Teleological Theories (Fodor 1990b, Millikan 1984, Papineau 1987, Dretske 1988, 1995). The Asymmetric Dependency Theory distinguishes merely informational relations from representational relations on the basis of their higher-order relations to each other: informational relations depend upon representational relations, but not vice versa. For example, if tokens of a mental state type are reliably caused by horses, cows-on-dark-nights, zebras-in-the-mist and Great Danes, then they carry information about horses, etc. If, however, such tokens are caused by cows-on-dark-nights, etc. because they were caused by horses, but not vice versa, then they represent horses (or the property horse ).

According to Teleological Theories, representational relations are those a representation-producing mechanism has the selected (by evolution or learning) function of establishing. For example, zebra-caused horse-representations do not mean zebra , because the mechanism by which such tokens are produced has the selected function of indicating horses, not zebras. The horse-representation-producing mechanism that responds to zebras is malfunctioning .

Functional theories (Block 1986, Harman 1973), hold that the content of a mental representation is determined, at least in part, by its (causal, computational, inferential) relations to other mental representations. They differ on whether relata should include all other mental representations or only some of them, and on whether to include external states of affairs. The view that the content of a mental representation is determined by its inferential/computational relations with all other representations is holism ; the view it is determined by relations to only some other mental states is localism (or molecularism ). (The non-functional view that the content of a mental state depends on none of its relations to other mental states is atomism .) Functional theories that recognize no content-determining external relata have been called solipsistic (Harman 1987). Some theorists posit distinct roles for internal and external connections, the former determining semantic properties analogous to sense, the latter determining semantic properties analogous to reference (McGinn 1982, Sterelny 1989).

(Reductive) representationalists (Dretske, Lycan, Tye) usually take one or another of these theories to provide an explanation of the (non-conceptual) content of experiential states. They thus tend to be externalists (see the next section) about phenomenological as well as conceptual content. Phenomenalists and non-reductive representationalists (Block, Chalmers, Loar, Peacocke, Siewert), on the other hand, take it that the representational content of such states is (at least in part) determined by their intrinsic phenomenal properties. Further, those who advocate a phenomenally-based approach to conceptual content (Horgan and Tienson, Kriegel, Loar, Pitt, Searle, Siewert) also seem to be committed to internalist individuation of the content (if not the reference) of such states.

Generally, those who, like informational theorists, think relations to one's (natural or social) environment are (at least partially) determinative of the content of mental representations are externalists , or anti-individualists (e.g., Burge 1979, 1986b, 2010, McGinn 1977), whereas those who, like some proponents of functional theories, think representational content is determined by an individual's intrinsic properties alone, are internalists (or individualists ; cf. Putnam 1975, Fodor 1981b). [ 5 ]

This issue is widely taken to be of central importance, since psychological explanation, whether commonsense or scientific, is supposed to be both causal and content-based. (Beliefs and desires cause the behaviors they do because they have the contents they do. For example, the desire that one have a beer and the beliefs that there is beer in the refrigerator and that the refrigerator is in the kitchen may explain one's getting up and going to the kitchen.) If, however, a mental representation's having a particular content is due to factors extrinsic to it, it is unclear how its having that content could determine its causal powers, which, arguably, must be intrinsic (see Stich 1983, Fodor 1982, 1987, 1994). Some who accept the standard arguments for externalism have argued that internal factors determine a component of the content of a mental representation. They say that mental representations have both “narrow” content (determined by intrinsic factors) and “wide” or “broad” content (determined by narrow content plus extrinsic factors). (This distinction may be applied to the sub-personal representations of cognitive science as well as to those of commonsense psychology. See von Eckardt 1993: 189.)

Narrow content has been variously construed. Putnam (1975), Fodor (1982: 114; 1994: 39ff), and Block (1986: 627ff), for example, seem to understand it as something like de dicto content (i.e., Fregean sense , or perhaps character , à la Kaplan 1989). On this construal, narrow content is context-independent and directly expressible. Fodor (1987) and Block (1986), however, have also characterized narrow content as radically inexpressible . On this construal, narrow content is a kind of proto-content, or content-determinant, and can be specified only indirectly, via specifications of context/wide-content pairings. On both construals, narrow contents are characterized as functions from context to (wide) content. The narrow content of a representation is determined by properties intrinsic to it or its possessor, such as its syntactic structure or its intramental computational or inferential role.

Burge (1986b) has argued that causation-based worries about externalist individuation of psychological content, and the introduction of the narrow notion, are misguided. Fodor (1994, 1998) has more recently urged that a scientific psychology might not need narrow content in order to supply naturalistic (causal) explanations of human cognition and action, since the sorts of cases they were introduced to handle, viz., Twin-Earth cases and Frege cases, are either nomologically impossible or dismissible as exceptions to non-strict psychological laws.

On the most common versions of externalism, though intentional contents are externally determined, mental representations themselves, and the states they partly constitute, remain “in the head.” More radical versions are possible. One might maintain that since thoughts are individuated by their contents, and some thought contents are partially constituted by objects external to the mind, then some thoughts are partly constituted by objects external to the mind. On such a view, a singular thought — i.e., a thought about a particular object — literally contains the object it is about. It is “object-involving.” Such a thought (and the mind that thinks it) thus extend beyond the boundaries of the skull. (This appears to be the view articulated in McDowell 1986, on which there is “interpenetration” between the mind and the world.)

Clark and Chalmers (1998) and Clark (2001, 2005, 2008) have argued that mental representations may exist entirely “outside the head.” On their view, which they call “active externalism,” cognitive processes (e.g., calculation) may be realized in external media (e.g., a calculator or pen and paper), and the “coupled system” of the individual mind and the external workspace ought to count as a cognitive system — a mind —in its own right. Symbolic representations on external media would thus count as mental representations.

Clark and Chalmers's paper has inspired a burgeoning literature on extended, embodied and interactive cognition. (Menary 2010 is a recent collection of essays. See also the entry on embodied cognition .)

The leading contemporary version of the Representational Theory of Mind, the Computational Theory of Mind (CTM), claims that the brain is a kind of computer and that mental processes are computations. According to CTM, cognitive states are constituted by computational relations to mental representations of various kinds, and cognitive processes are sequences of such states.

CTM develops RTM by attempting to explain all psychological states and processes in terms of mental representation. In the course of constructing detailed empirical theories of human and other animal cognition, and developing models of cognitive processes implementable in artificial information processing systems, cognitive scientists have proposed a variety of types of mental representations. While some of these may be suited to be mental relata of commonsense psychological states, some — so-called “subpersonal” or “sub-doxastic” representations — are not. Though many philosophers believe that CTM can provide the best scientific explanations of cognition and behavior, there is disagreement over whether such explanations will vindicate the commonsense psychological explanations of prescientific RTM.

According to Stich's (1983) Syntactic Theory of Mind, for example, computational theories of psychological states should concern themselves only with the formal properties of the objects those states are relations to. Commitment to the explanatory relevance of content , however, is for most cognitive scientists fundamental (Fodor 1981a, Pylyshyn 1984, Von Eckardt 1993). That mental processes are computations, that computations are rule-governed sequences of semantically evaluable objects , and that the rules apply to the symbols in virtue of their content, are central tenets of mainstream cognitive science.

Explanations in cognitive science appeal to a many different kinds of mental representation, including, for example, the “mental models” of Johnson-Laird 1983, the “retinal arrays,” “primal sketches” and “2½-D sketches” of Marr 1982, the “frames” of Minsky 1974, the “sub-symbolic” structures of Smolensky 1989, the “quasi-pictures” of Kosslyn 1980, and the “interpreted symbol-filled arrays” of Tye 1991 — in addition to representations that may be appropriate to the explanation of commonsense psychological states. Computational explanations have been offered of, among other mental phenomena, belief (Fodor 1975, 2008 Field 1978), visual perception (Marr 1982, Osherson, et al. 1990), rationality (Newell and Simon 1972, Fodor 1975, Johnson-Laird and Wason 1977), language learning and use (Chomsky 1965, Pinker 1989), and musical comprehension (Lerdahl and Jackendoff 1983).

A fundamental disagreement among proponents of CTM concerns the realization of personal-level representations (e.g., thoughts) and processes (e.g., inferences) in the brain. The central debate here is between proponents of Classical Architectures and proponents of Connectionist Architectures .

The classicists (e.g., Turing 1950, Fodor 1975, 2000, 2003, 2008, Fodor and Pylyshyn 1988, Marr 1982, Newell and Simon 1976) hold that mental representations are symbolic structures, which typically have semantically evaluable constituents, and that mental processes are rule-governed manipulations of them that are sensitive to their constituent structure. The connectionists (e.g., McCulloch & Pitts 1943, Rumelhart 1989, Rumelhart and McClelland 1986, Smolensky 1988) hold that mental representations are realized by patterns of activation in a network of simple processors (“nodes”) and that mental processes consist of the spreading activation of such patterns. The nodes themselves are, typically, not taken to be semantically evaluable; nor do the patterns have semantically evaluable constituents. (Though there are versions of Connectionism — “localist” versions — on which individual nodes are taken to have semantic properties (e.g., Ballard 1986, Ballard & Hayes 1984).) It is arguable, however, that localist theories are neither definitive nor representative of the connectionist program (Smolensky 1988, 1991, Chalmers 1993).)

Classicists are motivated (in part) by properties thought seems to share with language. Fodor's Language of Thought Hypothesis (LOTH) (Fodor 1975, 1987, 2008), according to which the system of mental symbols constituting the neural basis of thought is structured like a language, provides a well-worked-out version of the classical approach as applied to commonsense psychology. (Cf. also Marr 1982 for an application of classical approach in scientific psychology.) According to the LOTH, the potential infinity of complex representational mental states is generated from a finite stock of primitive representational states, in accordance with recursive formation rules. This combinatorial structure accounts for the properties of productivity and systematicity of the system of mental representations. As in the case of symbolic languages, including natural languages (though Fodor does not suppose either that the LOTH explains only linguistic capacities or that only verbal creatures have this sort of cognitive architecture), these properties of thought are explained by appeal to the content of the representational units and their combinability into contentful complexes. That is, the semantics of both language and thought is compositional : the content of a complex representation is determined by the contents of its constituents and their structural configuration. (See, e.g.,Fodor and Lepore 2002.)

Connectionists are motivated mainly by a consideration of the architecture of the brain, which apparently consists of layered networks of interconnected neurons. They argue that this sort of architecture is unsuited to carrying out classical serial computations. For one thing, processing in the brain is typically massively parallel. In addition, the elements whose manipulation drives computation in connectionist networks (principally, the connections between nodes) are neither semantically compositional nor semantically evaluable, as they are on the classical approach. This contrast with classical computationalism is often characterized by saying that representation is, with respect to computation, distributed as opposed to local : representation is local if it is computationally basic; and distributed if it is not. (Another way of putting this is to say that for classicists mental representations are computationally atomic , whereas for connectionists they are not.)

Moreover, connectionists argue that information processing as it occurs in connectionist networks more closely resembles some features of actual human cognitive functioning. For example, whereas on the classical view learning involves something like hypothesis formation and testing (Fodor 1981c), on the connectionist model it is a matter of evolving distribution of “weights” (strengths) on the connections between nodes, and typically does not involve the formulation of hypotheses regarding the identity conditions for the objects of knowledge. The connectionist network is “trained up” by repeated exposure to the objects it is to learn to distinguish; and, though networks typically require many more exposures to the objects than do humans, this seems to model at least one feature of this type of human learning quite well. (Cf. the sonar example in Churchland 1989.)

Further, degradation in the performance of such networks in response to damage is gradual, not sudden as in the case of a classical information processor, and hence more accurately models the loss of human cognitive function as it typically occurs in response to brain damage. It is also sometimes claimed that connectionist systems show the kind of flexibility in response to novel situations typical of human cognition — situations in which classical systems are relatively “brittle” or “fragile.”

Some philosophers have maintained that connectionism entails that there are no propositional attitudes. Ramsey, Stich and Garon (1990) have argued that if connectionist models of cognition are basically correct, then there are no discrete representational states as conceived in ordinary commonsense psychology and classical cognitive science. Others, however (e.g., Smolensky 1989), hold that certain types of higher-level patterns of activity in a neural network may be roughly identified with the representational states of commonsense psychology. Still others (e.g., Fodor & Pylyshyn 1988, Heil 1991, Horgan and Tienson 1996) argue that language-of-thought style representation is both necessary in general and realizable within connectionist architectures. (MacDonald & MacDonald 1995 collects the central contemporary papers in the classicist/connectionist debate, and provides useful introductory material as well. See also Von Eckardt 2005.)

Whereas Stich (1983) accepts that mental processes are computational, but denies that computations are sequences of mental representations, others accept the notion of mental representation, but deny that CTM provides the correct account of mental states and processes.

Van Gelder (1995) denies that psychological processes are computational. He argues that cognitive systems are dynamic , and that cognitive states are not relations to mental symbols, but quantifiable states of a complex system consisting of (in the case of human beings) a nervous system, a body and the environment in which they are embedded. Cognitive processes are not rule-governed sequences of discrete symbolic states, but continuous, evolving total states of dynamic systems determined by continuous, simultaneous and mutually determining states of the systems' components. Representation in a dynamic system is essentially information-theoretic, though the bearers of information are not symbols, but state variables or parameters. (See also Port and Van Gelder 1995; Clark 1997a, 1997b, 2008.)

Horst (1996), on the other hand, argues that though computational models may be useful in scientific psychology, they are of no help in achieving a philosophical understanding of the intentionality of commonsense mental states. CTM attempts to reduce the intentionality of such states to the intentionality of the mental symbols they are relations to. But, Horst claims, the relevant notion of symbolic content is essentially bound up with the notions of convention and intention. So CTM involves itself in a vicious circularity: the very properties that are supposed to be reduced are (tacitly) appealed to in the reduction.

To say that a mental object has semantic properties is, paradigmatically, to say that it is about , or true or false of, an object or objects, or that it is true or false simpliciter . Suppose I think that ocelots take snuff. I am thinking about ocelots, and if what I think of them (that they take snuff) is true of them, then my thought is true. According to RTM such states are to be explained as relations between agents and mental representations. To think that ocelots take snuff is to token in some way a mental representation whose content is that ocelots take snuff. On this view, the semantic properties of mental states are the semantic properties of the representations they are relations to.

Linguistic acts seem to share such properties with mental states. Suppose I say that ocelots take snuff. I am talking about ocelots, and if what I say of them (that they take snuff) is true of them, then my utterance is true. Now, to say that ocelots take snuff is (in part) to utter a sentence that means that ocelots take snuff. Many philosophers have thought that the semantic properties of linguistic expressions are inherited from the intentional mental states they are conventionally used to express (Grice 1957, Fodor 1978, Schiffer1972/1988, Searle 1983). On this view, the semantic properties of linguistic expressions are the semantic properties of the representations that are the mental relata of the states they are conventionally used to express.

(Others, however, e.g., Davidson (1975, 1982) have suggested that the kind of thought human beings are capable of is not possible without language, so that the dependency might be reversed, or somehow mutual (see also Sellars 1956). (But see Martin 1987 for a defense of the claim that thought is possible without language. See also Chisholm and Sellars 1958.) Schiffer (1987) subsequently despaired of the success of what he calls “Intention Based Semantics.”)

It is also widely held that in addition to having such properties as reference, truth-conditions and truth — so-called extensional properties — expressions of natural languages also have intensional properties, in virtue of expressing properties or propositions — i.e., in virtue of having meanings or senses , where two expressions may have the same reference, truth-conditions or truth value, yet express different properties or propositions (Frege 1892/1997). If the semantic properties of natural-language expressions are inherited from the thoughts and concepts they express (or vice versa, or both), then an analogous distinction may be appropriate for mental representations.

• Almog, J., Perry, J. and Wettstein, H. (eds.), (1989), Themes from Kaplan , New York: Oxford University Press.
• Alter, T. and Walter, S. (2007), Phenomenal Concepts and Phenomenal Knowledge: New Essays on Consciousness and Physicalism , Oxford: Oxford University Press.
• Aristotle, De Anima , in The Complete Works of Aristotle: The Revised Oxford Translation , Oxford: Oxford University Press, 1984.
• Baker, L. R. (1995), Explaining Attitudes: A Practical Approach to the Mind , Cambridge: Cambridge University Press.
• Ballard, D.H. (1986), “Cortical Connections and Parallel Processing: Structure and Function,” The Behavioral and Brain Sciences , 9: 67–120.
• Ballard, D.H and Hayes, P.J. (1984), “Parallel Logical Inference,” Proceedings of the Sixth Annual Conference of the Cognitive Science Society , Rochester, NY.
• Bayne, T. and Montague, M. (eds.), (2011), Cognitive Phenomenology , Oxford: Oxord University Press.
• Beaney, M. (ed.) (1997), The Frege Reader , Oxford: Blackwell Publishers.
• Berkeley, G. Principles of Human Knowledge , in M.R. Ayers (ed.), Berkeley: Philosophical Writings , London: Dent, 1975.
• Block, N. (1983), “Mental Pictures and Cognitive Science,” Philosophical Review , 93: 499–542.
• ––– (1986), “Advertisement for a Semantics for Psychology,” in P.A. French, T.E. Uehling and H.K. Wettstein (eds.), Midwest Studies in Philosophy, Vol. X , Minneapolis: University of Minnesota Press: 615–678.
• ––– (1996), “Mental Paint and Mental Latex,” in E. Villanueva (ed.), Philosophical Issues, 7: Perception : 19–49.
• ––– (2003), “Mental Paint,” in M. Hahn and B. Ramberg (eds.), Reflections and Replies: Essays on the Philosophy of Tyler Burge , Cambridge, Mass.: The MIT Press.
• Block, N. (ed.) (1981), Readings in Philosophy of Psychology, Vol. 2 , Cambridge, Mass.: Harvard University Press.
• ––– (ed.) (1982), Imagery , Cambridge, Mass.: The MIT Press.
• Boghossian, P. A. (1995), “Content,” in J. Kim and E. Sosa (eds.), A Companion to Metaphysics , Oxford: Blackwell, 94–96.
• Brandom, R. (2002), “Non-inferential Knowledge, Perceptual Experience, and Secondary Qualities: Placing McDowell's Empiricism,” in N.H. Smith (ed.), Reading McDowell: On Mind and World , London: Routledge.
• Burge, T. (1979), “Individualism and the Mental,” in P.A. French, T.E. Uehling and H.K.Wettstein (eds.), Midwest Studies in Philosophy, Vol. IV , Minneapolis: University of Minnesota Press: 73–121. (Reprinted, with Postscript, in Burge 2007.)
• ––– (1986a), “Individualism and Psychology,” Philosophical Review , 95: 3–45.
• ––– (1986b), “Intellectual Norms and Foundations of Mind,” The Journal of Philosophy , 83: 697–720.
• ––– (2007), Foundations of Mind: Philosophical Essays, Volume 2 , Oxford: Oxford University Press.
• ––– (2010), Origins of Objectivity , Oxford: Oxford University Press.
• Chalmers, D. (1993), “Connectionism and Compositionality: Why Fodor and Pylyshyn Were Wrong,” Philosophical Psychology , 6: 305–319.
• ––– (1996), The Conscious Mind , New York: Oxford University Press.
• ––– (2003), “The Content and Epistemology of Phenomenal Belief,” in Q. Smith & A. Jokic (eds.), Consciousness: New Philosophical Perspectives , Oxford: Oxford University Press: 220–272.
• ––– (2004a), “The Representational Character of Experience,” in B. Leiter (ed.), The Future for Philosophy , Oxford: Oxford University Press: 153–181.
• ––– (2004b), “Phenomenal Concepts and the Knowledge Argument,” in P. Ludlow, Y. Nagasawa and D. Stoljar (eds.), There's Something About Mary: Essays on Phenomenal Consciousness and Frank Jackson's Knowledge Argument , Cambridge, Mass.: The MIT Press.
• Chisholm, R. and Sellars, W. (1958), “The Chisholm-Sellars Correspondence on Intentionality,” in H. Feigl, M. Scriven and G. Maxwell (eds.), Minnesota Studies in the Philosophy of Science, Vol. II , Minneapolis: University of Minnesota Press: 529–539.
• Chomsky, N. (1965), Aspects of the Theory of Syntax , Cambridge, Mass.: The MIT Press.
• Churchland, P.M. (1981), “Eliminative Materialism and the Propositional Attitudes,” Journal of Philosophy , 78: 67–90.
• ––– (1989), “On the Nature of Theories: A Neurocomputational Perspective,” in W. Savage (ed.), Scientific Theories: Minnesota Studies in the Philosophy of Science , Vol. 14, Minneapolis: University of Minnesota Press: 59–101.
• Clark, A. (1997a), “The Dynamical Challenge,” Cognitive Science , 21: 461–481.
• ––– (1997b), Being There: Putting Brain, Body and World Together Again , Cambridge, MA: The MIT Press.
• ––– (2001), “Reasons, Robots and the Extended Mind,” Mind and Language , 16: 121–145.
• ––– (2005), “Intrinsic Content, Active Memory, and the Extended Mind,” Analysis , 65: 1–11.
• ––– (2008). Supersizing the Mind , Oxford: Oxford University Press.
• Clark, A., and Chalmers, D. (1998), “The Extended Mind,” Analysis , 58: 7–19.
• Collins, A. (1987), The Nature of Mental Things , Notre Dame: Notre Dame University Press.
• Crane, T. (1995), The Mechanical Mind , London: Penguin Books Ltd.
• Davidson, D. (1973), “Radical Interpretation,” Dialectica 27: 313–328.
• ––– (1974), “Belief and the Basis of Meaning,” Synthese , 27: 309–323.
• ––– (1975), “Thought and Talk,” in S. Guttenplan (ed.), Mind and Language , Oxford: Clarendon Press: 7–23.
• ––– (1982), “Rational Animals,” Dialectica , 4: 317–327.
• Dennett, D. (1969), Content and Consciousness , London: Routledge & Kegan Paul.
• ––– (1981), “The Nature of Images and the Introspective Trap,” pages 132–141 of Dennett 1969, reprinted in Block 1981: 128–134.
• ––– (1987), The Intentional Stance , Cambridge, Mass.: The MIT Press.
• ––– (1987a), “True Believers: The Intentional Strategy and Why it Works,” in Dennett 1987: 13–35.
• ––– (1987b), “Reflections: Real Patterns, Deeper Facts, and Empty Questions,” in Dennett 1987: 37–42.
• ––– (1988), “Quining Qualia,” in A.J. Marcel and E. Bisiach (eds.), Consciousness in Contemporary Science , Oxford: Clarendon Press: 42–77.
• ––– (1991), “Real Patterns,” The Journal of Philosophy , 87: 27–51.
• Devitt, M. (1996), Coming to Our Senses: A Naturalistic Program for Semantic Localism , Cambridge: Cambridge University Press.
• Dretske, F. (1969), Seeing and Knowing , Chicago: The University of Chicago Press.
• ––– (1981), Knowledge and the Flow of Information , Cambridge, Mass.: The MIT Press.
• ––– (1988), Explaining Behavior: Reasons in a World of Causes , Cambridge, Mass.: The MIT Press.
• ––– (1995), Naturalizing the Mind , Cambridge, Mass.: The MIT Press.
• ––– (1996), “Phenomenal Externalism, or If Meanings Ain't in the Head, Where are Qualia?”, in E. Villanueva (ed.), Philosophical Issues 7: Perception : 143–158.
• ––– (1999), “The Mind's Awareness of Itself,” Philosophical Studies , 95: 103–124.
• ––– (1998), “Minds, Machines, and Money: What Really Explains Behavior,” in J. Bransen and S. Cuypers (eds.), Human Action, Deliberation and Causation, Philosophical Studies Series 77 , Dordrecht: Kluwer Academic Publishers. Reprinted in Dretske 2000.
• ––– (2000), Perception, Knowledge and Belief , Cambridge: Cambridge University Press.
• Evans, G. (1982), The Varieties of Reference , Oxford: Oxford University Press.
• Farkas, K. (2008), The Subject's Point of View , Oxford: Oxford University Press.
• Field, H. (1978), “Mental representation,” Erkenntnis , 13: 9–61.
• Flanagan, O. (1992), Consciousness Reconsidered , Cambridge, Mass.: The MIT Press.
• Fodor, J.A. (1975), The Language of Thought , Cambridge, Mass.: Harvard University Press.
• ––– (1978), “Propositional Attitudes,” The Monist 61: 501–523.
• ––– (1981), Representations , Cambridge, Mass.: The MIT Press.
• ––– (1981a), “Introduction,” in Fodor 1981: 1–31.
• ––– (1981b), “Methodological Solipsism Considered as a Research Strategy in Cognitive Psychology,” in Fodor 1981: 225–253.
• ––– (1981c), “The Present Status of the Innateness Controversy,” in Fodor 1981: 257–316.
• ––– (1982), “Cognitive Science and the Twin-Earth Problem,” Notre Dame Journal of Formal Logic , 23: 98–118.
• ––– (1987), Psychosemantics , Cambridge, Mass.: The MIT Press.
• ––– (1990a), A Theory of Content and Other Essays , Cambridge, Mass.: The MIT Press.
• ––– (1990b), “Psychosemantics or: Where Do Truth Conditions Come From?” in W.G. Lycan (ed.), Mind and Cognition: A Reader , Oxford: Blackwell Publishers: 312–337.
• ––– (1994), The Elm and the Expert , Cambridge, Mass.: The MIT Press.
• ––– (1998), Concepts: Where Cognitive Science Went Wrong , Oxford: Oxford University Press.
• ––– (2000), The Mind Doesn't Work that Way: The Scope and Limits of Computational Psychology , Cambridge, Mass.: The MIT Press.
• ––– (2003), LOT 2: The Language of Thought Revisited , Oxford: Clarendon Press.
• ––– (2008), The Mind Doesn't Work that Way: The Scope and Limits of Computational Psychology , Cambridge, Mass.: The MIT Press.
• Fodor, J.A. and Lepore, E. (2002), The Compositionality Papers , Oxford: Clarendon Press.
• Fodor, J.A. and Pylyshyn, Z. (1981), “How Direct is Visual Perception?: Some Reflections on Gibson's ‘Ecological Approach’,” Cognition , 9: 207–246.
• ––– (1988), “Connectionism and Cognitive Architecture: A Critical Analysis,” Cognition , 28: 3–71.
• Frege, G. (1884), The Foundations of Arithmetic , trans. J.L. Austin, New York: Philosophical Library (1954).
• ––– (1892), “On Sinn and Bedeutung ”, in Beany 1997: 151–171.
• ––– (1918), “Thought”, in Beany 1997: 325–345.
• Geach, P. (1957), Mental Acts: Their Content and Their Objects , London: Routledge & Kegan Paul.
• Gibson, J.J. (1966), The senses considered as perceptual systems , Boston: Houghton Mifflin.
• ––– (1979), The ecological approach to visual perception , Boston: Houghton Mifflin.
• Goldman, A. (1993), “The Psychology of Folk Psychology,” Behavioral and Brian Sciences , 16: 15–28.
• Goodman, N. (1976), Languages of Art , 2nd ed., Indianapolis: Hackett.
• Grice, H.P. (1957), “Meaning,” Philosophical Review , 66: 377–388; reprinted in Studies in the Way of Words , Cambridge, Mass.: Harvard University Press (1989): 213–223.
• Gunther, Y.H. (ed.) (2003), Essays on Nonconceptual Content , Cambridge, Mass.: The MIT Press.
• Harman, G. (1973), Thought , Princeton: Princeton University Press.
• ––– (1987), “(Non-Solipsistic) Conceptual Role Semantics,” in E. Lepore (ed.), New Directions in Semantics , London: Academic Press: 55–81.
• ––– (1990), “The Intrinsic Quality of Experience,” in J. Tomberlin (ed.), Philosophical Perspectives 4: Action Theory and Philosophy of Mind , Atascadero: Ridgeview Publishing Company: 31–52.
• Harnish, R. (2002), Minds, Brains, Computers , Malden, Mass.: Blackwell Publishers Inc.
• Haugeland, J. (1981), “Analog and analog,” Philosophical Topics , 12: 213–226.
• Heil, J. (1991), “Being Indiscrete,” in J. Greenwood (ed.), The Future of Folk Psychology , Cambridge: Cambridge University Press: 120–134.
• Horgan, T. and Tienson, J. (1996), Connectionism and the Philosophy of Psychology , Cambridge, Mass: The MIT Press.
• ––– (2002), “The Intentionality of Phenomenology and the Phenomenology of Intentionality,” in D.J. Chalmers (ed.), Philosophy of Mind , Oxford: Oxford University Press.
• Horst, S. (1996), Symbols, Computation, and Intentionality , Berkeley: University of California Press.
• Hume, D. (1739), A Treatise of Human Nature , L.A. Selby-Bigg (ed.), rev. P.H. Nidditch, Oxford: Oxford University Press (1978).
• Jackendoff, R. (1987), Computation and Cognition , Cambridge, Mass.: The MIT Press.
• Jackson, F. (1982), “Epiphenomenal Qualia,” Philosophical Quarterly , 32: 127–136.
• ––– (1986), “What Mary Didn't Know,” Journal of Philosophy , 83: 291–295.
• Johnson-Laird, P.N. (1983), Mental Models , Cambridge, Mass.: Harvard University Press.
• Johnson-Laird, P.N. and Wason, P.C. (1977), Thinking: Readings in Cognitive Science , Cambridge University Press.
• Kaplan, D. (1989), “Demonstratives,” in Almog, Perry and Wettstein 1989: 481–614.
• Kosslyn, S.M. (1980), Image and Mind , Cambridge, Mass.: Harvard University Press.
• ––– (1982), “The Medium and the Message in Mental Imagery,” in Block 1982: 207–246.
• ––– (1983), Ghosts in the Mind's Machine , New York: W.W. Norton & Co.
• Kosslyn, S.M. and Pomerantz, J.R. (1977), “Imagery, Propositions, and the Form of Internal Representations,” Cognitive Psychology , 9: 52–76.
• Kriegel, U. (2011), The Sources of Intentionality , Oxford: Oxford University Press.
• Kriegel, U. (ed.) forthcoming, Phenomenal Intentionality: New Essays , Oxford: Oxford University Press.
• Leeds, S. (1993), “Qualia, Awareness, Sellars,” Noûs XXVII: 303–329.
• Lerdahl, F. and Jackendoff, R. (1983), A Generative Theory of Tonal Music , Cambridge, Mass.: The MIT Press.
• Levine, J. (1993), “On Leaving Out What It's Like,” in M. Davies and G. Humphreys (eds.), Consciousness , Oxford: Blackwell Publishers: 121–136.
• ––– (1995), “On What It Is Like to Grasp a Concept,” in E. Villanueva (ed.), Philosophical Issues 6: Content , Atascadero: Ridgeview Publishing Company: 38–43.
• ––– (2001), Purple Haze , Oxford: Oxford University Press.
• Lewis, D. (1971), “Analog and Digital,” Noûs , 5: 321–328.
• ––– (1974), “Radical Interpretation,” Synthese , 27: 331–344.
• Loar, B. (1981), Mind and Meaning , Cambridge: Cambridge University Press.
• ––– (1996), “Phenomenal States” (Revised Version), in N. Block, O. Flanagan and G. Güzeldere (eds.), The Nature of Consciousness , Cambridge, Mass.: The MIT Press: 597–616.
• ––– (2003a), “Transparent Experience and the Availability of Qualia,” in Q. Smith and A. Jokic (eds.), Consciousness: New Philosophical Perspectives , Oxford: Clarendon Press: 77–96.
• ––– (2003b), “Phenomenal Intentionality as the Basis of Mental Content,” in M. Hahn and B. Ramberg (eds.), Reflections and Replies: Essays on the Philosophy of Tyler Burge , Cambridge, Mass.: The MIT Press.
• Locke, J. (1689), An Essay Concerning Human Understanding , P.H. Nidditch (ed.), Oxford: Oxford University Press (1975).
• Lycan, W.G. (1987), Consciousness , Cambridge, Mass.: The MIT Press.
• ––– (1986), Consciousness and Experience , Cambridge, Mass.: The MIT Press.
• MacDonald, C. and MacDonald, G. (1995), Connectionism: Debates on Psychological Explanation , Oxford: Blackwell Publishers.
• Marr, D. (1982), Vision , New York: W.H. Freeman and Company.
• Martin, C.B. (1987), “Proto-Language,” Australasian Journal of Philosophy , 65: 277–289.
• McCulloch, W.S. and Pitts, W. (1943), “A Logical Calculus of the Ideas Immanent in Nervous Activity,” Bulletin of Mathematical Biophysics , 5: 115–33.
• McDowell, J. (1986), “Singular Thought and the Extent of Inner Space,” in P. Pettit and J. McDowell (eds.), Subject, Thought, and Context , Oxford: Clarendon Press: 137–168.
• ––– (1994), Mind and World , Cambridge, Mass.: Harvard University Press.
• McGinn, C. (1977), “Charity, Interpretation, and Belief,” Journal of Philosophy , 74: 521–535.
• ––– (1982), “The Structure of Content,” in A. Woodfield (ed.), Thought and Content , Oxford: Oxford University Press: 207–258.
• ––– (1989), Mental Content , Oxford: Blackwell Publishers.
• ––– (1991), The Problem of Consciousness , Oxford: Blackwell Publishers.
• ––– (1991a), “Content and Consciousness,” in McGinn 1991: 23–43.
• ––– (1991b), “Can We Solve the Mind-Body Problem?” in McGinn 1991: 1–22.
• ––– (2004), Mindsight: Image, Dream, Meaning , Cambridge, Mass.: Harvard University Press.
• Millikan, R. (1984), Language, Thought and other Biological Categories , Cambridge, Mass.: The MIT Press.
• Menary, R. (ed.) (2010), The Extended Mind , Cambridge, Mass.: The MIT Press.
• Minsky, M. (1974), “A Framework for Representing Knowledge,” MIT-AI Laboratory Memo 306 June. (A shorter version appears in J. Haugeland (ed.), Mind Design II , Cambridge, Mass.: The MIT Press (1997).)
• Nagel, T. (1974), “What Is It Like to Be a Bat?” Philosophical Review , 83: 435–450.
• Newell, A. and Simon, H.A. (1972), Human Problem Solving , New York: Prentice-Hall.
• ––– (1976), “Computer Science as Empirical Inquiry: Symbols and Search,” Communications of the Association for Computing Machinery , 19: 113–126.
• O'Callaghan, C. (2007), Sounds , Oxford: Oxford University Press.
• Osherson, D.N., Kosslyn, S.M. and Hollerbach, J.M. (1990), Visual Cognition and Action: An Invitation to Cognitive Science, Vol. 2 , Cambridge, Mass.: The MIT Press.
• Papineau, D. (1987), Reality and Representation , Oxford: Blackwell Publishers.
• Peacocke, C. (1983), Sense and Content , Oxford: Clarendon Press.
• ––– (1989), “Perceptual Content,” in Almog, Perry and Wettstein 1989: 297–329.
• ––– (1992), “Scenarios, Concepts and Perception,” in T. Crane (ed.), The Contents of Experience , Cambridge: Cambridge University Press: 105–35.
• ––– (2001), “Does Perception Have a Nonconceptual Content?” Journal of Philosophy , 99: 239–264.
• Pinker, S. (1989), Learnability and Cognition , Cambridge, Mass.: The MIT Press.
• Pitt, D. (2004), “The Phenomenology of Cognition, Or, What Is it Like to Think That P?” Philosophy and Phenomenological Research , 69: 1–36.
• ––– (2009), “Intentional Psychologism” Philosophical Studies , 146: 117–138.
• ––– (2011), “Introspection, Phenomenality and the Availability of Intentional Content,” in Bayne and Montague 2011.
• –––, forthcoming, “Indexical Thought,” in Kriegel (ed.) forthcoming.
• Port, R., and Van Gelder, T. (1995), Mind as Motion: Explorations in the Dynamics of Cognition , Cambridge, Mass.: The MIT Press.
• Putnam, H. (1975), “The Meaning of ‘Meaning’,” in Philosophical Papers, Vol. 2 , Cambridge: Cambridge University Press: 215–71.
• Pylyshyn, Z. (1979), “The Rate of ‘Mental Rotation’ of Images: A Test of a Holistic Analogue Hypothesis,” Memory and Cognition , 7: 19–28.
• ––– (1981a), “Imagery and Artificial Intelligence,” in Block 1981: 170–194.
• ––– (1981b), “The Imagery Debate: Analog Media versus Tacit Knowledge,” Psychological Review , 88: 16–45.
• ––– (1984), Computation and Cognition , Cambridge, Mass.: The MIT Press.
• ––– (2003), Seeing and Visualizing: It's Not What You Think , Cambridge, Mass.: The MIT Press.
• Raffman, D. (1995), “The Persistence of Phenomenology,” in T. Metzinger (ed.), Conscious Experience , Paderborn: Schönigh/Imprint Academic: 293–308.
• Ramsey, W., Stich, S. and Garon, J. (1990), “Connectionism, Eliminativism and the Future of Folk Psychology,” Philosophical Perspectives , 4: 499–533.
• Reid, T. (1764), An Inquiry into the Human Mind , D.R. Brooks (ed.), Edinburgh: Edinburgh University Press (1997).
• Rey, G. (1981), “Introduction: What Are Mental Images?” in Block 1981: 117–127.
• ––– (1991), “Sensations in a Language of Thought,” in E. Villaneuva (ed.), Philosophical Issues 1: Consciousness , Atascadero: Ridgeview Publishing Company: 73–112.
• Rumelhart, D.E. (1989), “The Architecture of the Mind: A Connectionist Approach,” in M.I. Posner (ed.), Foundations of Cognitive Science , Cambridge, Mass.: The MIT Press: 133–159.
• Rumelhart, D.E. and McClelland, J.L. (1986). Parallel Distributed Processing, Vol. I , Cambridge, Mass.: The MIT Press.
• Schiffer, S. (1987), Remnants of Meaning , Cambridge, Mass.: The MIT Press.
• ––– (1972), “Introduction” (Paperback Edition), in Meaning , Oxford: Clarendon Press (1972/1988): xi-xxix.
• Searle, J.R. (1980), “Minds, Brains, and Programs,” Behavioral and Brain Sciences , 3: 417–424.
• ––– (1983), Intentionality , Cambridge: Cambridge University Press.
• ––– (1984) Minds, Brains, and Science , Cambridge: Harvard University Press.
• ––– (1992), The Rediscovery of the Mind , Cambridge, Mass.: The MIT Press.
• Sellars, W. (1956), “Empiricism and the Philosophy of Mind,” in K. Gunderson (ed.), Minnesota Studies in the Philosophy of Science, Vol. I , Minneapolis: University of Minnesota Press: 253–329.
• Shepard, R.N. and Cooper, L. (1982), Mental Images and their Transformations , Cambridge, Mass.: The MIT Press.
• Shoemaker, S. (1990), “Qualities and Qualia: What's in the Mind?” Philosophy and Phenomenological Research , 50: 109–31.
• Siewert, C. (1998), The Significance of Consciousness , Princeton: Princeton University Press.
• Smolensky, P. (1988), “On the Proper Treatment of Connectionism,” Behavioral and Brain Sciences , 11: 1–74.
• ––– (1989), “Connectionist Modeling: Neural Computation/Mental Connections,” in L. Nadel, L.A. Cooper, P. Culicover and R.M. Harnish (eds.), Neural Connections, Mental Computation Cambridge, Mass.:The MIT Press: 49–67.
• ––– (1991), “Connectionism and the Language of Thought,” in B. Loewer and G. Rey (eds.), Meaning in Mind: Fodor and His Critics , Oxford: Basil Blackwell Ltd.: 201–227.
• Sterelny, K. (1989), “Fodor's Nativism,” Philosophical Studies , 55: 119–141.
• Stich, S. (1983), From Folk Psychology to Cognitive Science , Cambridge, Mass.: The MIT Press.
• ––– (1996), Deconstructing the Mind , New York: Oxford University Press.
• Strawson, G. (1994), Mental Reality , Cambridge, Mass.: The MIT Press.
• ––– (2008), Real Materialism and Other Essays , Oxford: Oxford University Press.
• Thau, M. (2002), Consciousness and Cognition , Oxford: Oxford University Press.
• Turing, A. (1950), “Computing Machinery and Intelligence,” Mind , 59: 433–60.
• Tye, M. (1991), The Imagery Debate , Cambridge, Mass.: The MIT Press.
• ––– (1995), Ten Problems of Consciousness , Cambridge, Mass.: The MIT Press.
• ––– (2000), Consciousness, Color, and Content , Cambridge, Mass.: The MIT Press.
• ––– (2009), Consciousness Revisited , Cambridge, Mass.: The MIT Press.
• Van Gelder, T. (1995), “What Might Cognition Be, if not Computation?”, Journal of Philosophy , XCI: 345–381.
• Von Eckardt, B. (1993), What Is Cognitive Science? , Cambridge, Mass.: The MIT Press.
• ––– (2005), “Connectionism and the Propositional Attitudes,” in C.E. Erneling and D.M. Johnson (eds.), The Mind as a Scientific Object: Between Brain and Culture , Oxford: Oxford University Press.
• Wittgenstein, L. (1953), Philosophical Investigations , trans. G.E.M. Anscombe, Oxford: Blackwell Publishers.

How to cite this entry . Preview the PDF version of this entry at the Friends of the SEP Society . Look up this entry topic at the Indiana Philosophy Ontology Project (InPhO). Enhanced bibliography for this entry at PhilPapers , with links to its database.

• Guide to the Philosophy of Mind , Editor, David Chalmers.
• MindPapers: Online Research in Philosophy , Editors, David Chalmers and David Bourget.
• A Field Guide to the Philosophy of Mind , General editors, Marcho Nani and Massimo Marraffa.
• Dictionary of Philosophy of Mind , Creator and Founding Editor, Chris Eliasmith, University of Waterloo. Chief Editor, Eric Hochstein, University of Waterloo..
• Routledge Encyclopedia of Philosophy , General Editor, Tim Crane.

-->cognition-embodied --> | cognitive science | concepts | connectionism | consciousness: and intentionality | consciousness: representational theories of | folk psychology: as mental simulation | information: semantic conceptions of | intentionality | language of thought hypothesis | -->logic and artificial intelligence --> | materialism: eliminative | mental content: causal theories of | mental content: externalism about | mental content: narrow | mental content: nonconceptual | mental content: teleological theories of | mental imagery | mental representation: in medieval philosophy | mind: computational theory of | neuroscience, philosophy of | perception: the contents of | perception: the problem of | qualia | reference

Acknowledgments

Thanks to Brad Armour-Garb, Mark Balaguer, Dave Chalmers, Jim Garson, John Heil, Jeff Poland, Bill Robinson, Galen Strawson, Adam Vinueza and (especially) Barbara Von Eckardt for comments on earlier versions of this entry.

Copyright © 2012 by David Pitt < dalanpitt @ yahoo . com >

Support SEP

Mirror sites.

View this site from another server:

• Info about mirror sites

The Stanford Encyclopedia of Philosophy is copyright © 2016 by The Metaphysics Research Lab , Center for the Study of Language and Information (CSLI), Stanford University

Library of Congress Catalog Data: ISSN 1095-5054

• Search Menu

Sign in through your institution

• Browse content in Arts and Humanities
• Browse content in Archaeology
• Anglo-Saxon and Medieval Archaeology
• Archaeological Methodology and Techniques
• Archaeology by Region
• Archaeology of Religion
• Archaeology of Trade and Exchange
• Biblical Archaeology
• Contemporary and Public Archaeology
• Environmental Archaeology
• Historical Archaeology
• History and Theory of Archaeology
• Industrial Archaeology
• Landscape Archaeology
• Mortuary Archaeology
• Prehistoric Archaeology
• Underwater Archaeology
• Zooarchaeology
• Browse content in Architecture
• Architectural Structure and Design
• History of Architecture
• Residential and Domestic Buildings
• Theory of Architecture
• Browse content in Art
• Art Subjects and Themes
• History of Art
• Industrial and Commercial Art
• Theory of Art
• Biographical Studies
• Byzantine Studies
• Browse content in Classical Studies
• Classical History
• Classical Philosophy
• Classical Mythology
• Classical Numismatics
• Classical Literature
• Classical Reception
• Classical Art and Architecture
• Classical Oratory and Rhetoric
• Greek and Roman Papyrology
• Greek and Roman Epigraphy
• Greek and Roman Law
• Greek and Roman Archaeology
• Late Antiquity
• Religion in the Ancient World
• Social History
• Digital Humanities
• Browse content in History
• Colonialism and Imperialism
• Diplomatic History
• Environmental History
• Genealogy, Heraldry, Names, and Honours
• Genocide and Ethnic Cleansing
• Historical Geography
• History by Period
• History of Emotions
• History of Agriculture
• History of Education
• History of Gender and Sexuality
• Industrial History
• Intellectual History
• International History
• Labour History
• Legal and Constitutional History
• Local and Family History
• Maritime History
• Military History
• National Liberation and Post-Colonialism
• Oral History
• Political History
• Public History
• Regional and National History
• Revolutions and Rebellions
• Slavery and Abolition of Slavery
• Social and Cultural History
• Theory, Methods, and Historiography
• Urban History
• World History
• Browse content in Language Teaching and Learning
• Language Learning (Specific Skills)
• Language Teaching Theory and Methods
• Browse content in Linguistics
• Applied Linguistics
• Cognitive Linguistics
• Computational Linguistics
• Forensic Linguistics
• Grammar, Syntax and Morphology
• Historical and Diachronic Linguistics
• History of English
• Language Evolution
• Language Reference
• Language Acquisition
• Language Variation
• Language Families
• Lexicography
• Linguistic Anthropology
• Linguistic Theories
• Linguistic Typology
• Phonetics and Phonology
• Psycholinguistics
• Sociolinguistics
• Translation and Interpretation
• Writing Systems
• Browse content in Literature
• Bibliography
• Children's Literature Studies
• Literary Studies (Romanticism)
• Literary Studies (American)
• Literary Studies (Asian)
• Literary Studies (European)
• Literary Studies (Eco-criticism)
• Literary Studies (Modernism)
• Literary Studies - World
• Literary Studies (1500 to 1800)
• Literary Studies (19th Century)
• Literary Studies (20th Century onwards)
• Literary Studies (African American Literature)
• Literary Studies (British and Irish)
• Literary Studies (Early and Medieval)
• Literary Studies (Fiction, Novelists, and Prose Writers)
• Literary Studies (Gender Studies)
• Literary Studies (Graphic Novels)
• Literary Studies (History of the Book)
• Literary Studies (Plays and Playwrights)
• Literary Studies (Poetry and Poets)
• Literary Studies (Postcolonial Literature)
• Literary Studies (Queer Studies)
• Literary Studies (Science Fiction)
• Literary Studies (Travel Literature)
• Literary Studies (War Literature)
• Literary Studies (Women's Writing)
• Literary Theory and Cultural Studies
• Mythology and Folklore
• Shakespeare Studies and Criticism
• Browse content in Media Studies
• Browse content in Music
• Applied Music
• Dance and Music
• Ethics in Music
• Ethnomusicology
• Gender and Sexuality in Music
• Medicine and Music
• Music Cultures
• Music and Media
• Music and Religion
• Music and Culture
• Music Education and Pedagogy
• Music Theory and Analysis
• Musical Scores, Lyrics, and Libretti
• Musical Structures, Styles, and Techniques
• Musicology and Music History
• Performance Practice and Studies
• Race and Ethnicity in Music
• Sound Studies
• Browse content in Performing Arts
• Browse content in Philosophy
• Aesthetics and Philosophy of Art
• Epistemology
• Feminist Philosophy
• History of Western Philosophy
• Metaphysics
• Moral Philosophy
• Non-Western Philosophy
• Philosophy of Language
• Philosophy of Mind
• Philosophy of Perception
• Philosophy of Science
• Philosophy of Action
• Philosophy of Law
• Philosophy of Religion
• Philosophy of Mathematics and Logic
• Practical Ethics
• Social and Political Philosophy
• Browse content in Religion
• Biblical Studies
• Christianity
• East Asian Religions
• History of Religion
• Judaism and Jewish Studies
• Qumran Studies
• Religion and Education
• Religion and Health
• Religion and Politics
• Religion and Science
• Religion and Law
• Religion and Art, Literature, and Music
• Religious Studies
• Browse content in Society and Culture
• Cookery, Food, and Drink
• Cultural Studies
• Customs and Traditions
• Ethical Issues and Debates
• Hobbies, Games, Arts and Crafts
• Natural world, Country Life, and Pets
• Popular Beliefs and Controversial Knowledge
• Sports and Outdoor Recreation
• Technology and Society
• Travel and Holiday
• Visual Culture
• Browse content in Law
• Arbitration
• Browse content in Company and Commercial Law
• Commercial Law
• Company Law
• Browse content in Comparative Law
• Systems of Law
• Competition Law
• Browse content in Constitutional and Administrative Law
• Government Powers
• Judicial Review
• Local Government Law
• Military and Defence Law
• Parliamentary and Legislative Practice
• Construction Law
• Contract Law
• Browse content in Criminal Law
• Criminal Procedure
• Criminal Evidence Law
• Sentencing and Punishment
• Employment and Labour Law
• Environment and Energy Law
• Browse content in Financial Law
• Banking Law
• Insolvency Law
• History of Law
• Human Rights and Immigration
• Intellectual Property Law
• Browse content in International Law
• Private International Law and Conflict of Laws
• Public International Law
• IT and Communications Law
• Jurisprudence and Philosophy of Law
• Law and Politics
• Law and Society
• Browse content in Legal System and Practice
• Courts and Procedure
• Legal Skills and Practice
• Legal System - Costs and Funding
• Primary Sources of Law
• Regulation of Legal Profession
• Medical and Healthcare Law
• Browse content in Policing
• Criminal Investigation and Detection
• Police and Security Services
• Police Procedure and Law
• Police Regional Planning
• Browse content in Property Law
• Personal Property Law
• Restitution
• Study and Revision
• Terrorism and National Security Law
• Browse content in Trusts Law
• Wills and Probate or Succession
• Browse content in Medicine and Health
• Browse content in Allied Health Professions
• Arts Therapies
• Clinical Science
• Dietetics and Nutrition
• Occupational Therapy
• Operating Department Practice
• Physiotherapy
• Radiography
• Speech and Language Therapy
• Browse content in Anaesthetics
• General Anaesthesia
• Clinical Neuroscience
• Browse content in Clinical Medicine
• Acute Medicine
• Cardiovascular Medicine
• Clinical Genetics
• Clinical Pharmacology and Therapeutics
• Dermatology
• Endocrinology and Diabetes
• Gastroenterology
• Genito-urinary Medicine
• Geriatric Medicine
• Infectious Diseases
• Medical Toxicology
• Medical Oncology
• Pain Medicine
• Palliative Medicine
• Rehabilitation Medicine
• Respiratory Medicine and Pulmonology
• Rheumatology
• Sleep Medicine
• Sports and Exercise Medicine
• Community Medical Services
• Critical Care
• Emergency Medicine
• Forensic Medicine
• Haematology
• History of Medicine
• Browse content in Medical Skills
• Clinical Skills
• Communication Skills
• Nursing Skills
• Surgical Skills
• Browse content in Medical Dentistry
• Oral and Maxillofacial Surgery
• Paediatric Dentistry
• Restorative Dentistry and Orthodontics
• Surgical Dentistry
• Medical Ethics
• Medical Statistics and Methodology
• Browse content in Neurology
• Clinical Neurophysiology
• Neuropathology
• Nursing Studies
• Browse content in Obstetrics and Gynaecology
• Gynaecology
• Occupational Medicine
• Ophthalmology
• Otolaryngology (ENT)
• Browse content in Paediatrics
• Neonatology
• Browse content in Pathology
• Chemical Pathology
• Clinical Cytogenetics and Molecular Genetics
• Histopathology
• Medical Microbiology and Virology
• Patient Education and Information
• Browse content in Pharmacology
• Psychopharmacology
• Browse content in Popular Health
• Caring for Others
• Complementary and Alternative Medicine
• Self-help and Personal Development
• Browse content in Preclinical Medicine
• Cell Biology
• Molecular Biology and Genetics
• Reproduction, Growth and Development
• Primary Care
• Professional Development in Medicine
• Browse content in Psychiatry
• Addiction Medicine
• Child and Adolescent Psychiatry
• Forensic Psychiatry
• Learning Disabilities
• Old Age Psychiatry
• Psychotherapy
• Browse content in Public Health and Epidemiology
• Epidemiology
• Public Health
• Browse content in Radiology
• Clinical Radiology
• Interventional Radiology
• Nuclear Medicine
• Radiation Oncology
• Reproductive Medicine
• Browse content in Surgery
• Cardiothoracic Surgery
• Gastro-intestinal and Colorectal Surgery
• General Surgery
• Neurosurgery
• Paediatric Surgery
• Peri-operative Care
• Plastic and Reconstructive Surgery
• Surgical Oncology
• Transplant Surgery
• Trauma and Orthopaedic Surgery
• Vascular Surgery
• Browse content in Science and Mathematics
• Browse content in Biological Sciences
• Aquatic Biology
• Biochemistry
• Bioinformatics and Computational Biology
• Developmental Biology
• Ecology and Conservation
• Evolutionary Biology
• Genetics and Genomics
• Microbiology
• Molecular and Cell Biology
• Natural History
• Plant Sciences and Forestry
• Research Methods in Life Sciences
• Structural Biology
• Systems Biology
• Zoology and Animal Sciences
• Browse content in Chemistry
• Analytical Chemistry
• Computational Chemistry
• Crystallography
• Environmental Chemistry
• Industrial Chemistry
• Inorganic Chemistry
• Materials Chemistry
• Medicinal Chemistry
• Mineralogy and Gems
• Organic Chemistry
• Physical Chemistry
• Polymer Chemistry
• Study and Communication Skills in Chemistry
• Theoretical Chemistry
• Browse content in Computer Science
• Artificial Intelligence
• Computer Architecture and Logic Design
• Game Studies
• Human-Computer Interaction
• Mathematical Theory of Computation
• Programming Languages
• Software Engineering
• Systems Analysis and Design
• Virtual Reality
• Browse content in Computing
• Business Applications
• Computer Security
• Computer Games
• Computer Networking and Communications
• Digital Lifestyle
• Graphical and Digital Media Applications
• Operating Systems
• Browse content in Earth Sciences and Geography
• Atmospheric Sciences
• Environmental Geography
• Geology and the Lithosphere
• Maps and Map-making
• Meteorology and Climatology
• Oceanography and Hydrology
• Palaeontology
• Physical Geography and Topography
• Regional Geography
• Soil Science
• Urban Geography
• Browse content in Engineering and Technology
• Agriculture and Farming
• Biological Engineering
• Civil Engineering, Surveying, and Building
• Electronics and Communications Engineering
• Energy Technology
• Engineering (General)
• Environmental Science, Engineering, and Technology
• History of Engineering and Technology
• Mechanical Engineering and Materials
• Technology of Industrial Chemistry
• Transport Technology and Trades
• Browse content in Environmental Science
• Applied Ecology (Environmental Science)
• Conservation of the Environment (Environmental Science)
• Environmental Sustainability
• Environmentalist Thought and Ideology (Environmental Science)
• Management of Land and Natural Resources (Environmental Science)
• Natural Disasters (Environmental Science)
• Nuclear Issues (Environmental Science)
• Pollution and Threats to the Environment (Environmental Science)
• Social Impact of Environmental Issues (Environmental Science)
• History of Science and Technology
• Browse content in Materials Science
• Ceramics and Glasses
• Composite Materials
• Metals, Alloying, and Corrosion
• Nanotechnology
• Browse content in Mathematics
• Applied Mathematics
• Biomathematics and Statistics
• History of Mathematics
• Mathematical Education
• Mathematical Finance
• Mathematical Analysis
• Numerical and Computational Mathematics
• Probability and Statistics
• Pure Mathematics
• Browse content in Neuroscience
• Cognition and Behavioural Neuroscience
• Development of the Nervous System
• Disorders of the Nervous System
• History of Neuroscience
• Invertebrate Neurobiology
• Molecular and Cellular Systems
• Neuroendocrinology and Autonomic Nervous System
• Neuroscientific Techniques
• Sensory and Motor Systems
• Browse content in Physics
• Astronomy and Astrophysics
• Atomic, Molecular, and Optical Physics
• Biological and Medical Physics
• Classical Mechanics
• Computational Physics
• Condensed Matter Physics
• Electromagnetism, Optics, and Acoustics
• History of Physics
• Mathematical and Statistical Physics
• Measurement Science
• Nuclear Physics
• Particles and Fields
• Plasma Physics
• Quantum Physics
• Relativity and Gravitation
• Semiconductor and Mesoscopic Physics
• Browse content in Psychology
• Affective Sciences
• Clinical Psychology
• Cognitive Psychology
• Cognitive Neuroscience
• Criminal and Forensic Psychology
• Developmental Psychology
• Educational Psychology
• Evolutionary Psychology
• Health Psychology
• History and Systems in Psychology
• Music Psychology
• Neuropsychology
• Organizational Psychology
• Psychological Assessment and Testing
• Psychology of Human-Technology Interaction
• Psychology Professional Development and Training
• Research Methods in Psychology
• Social Psychology
• Browse content in Social Sciences
• Browse content in Anthropology
• Anthropology of Religion
• Human Evolution
• Medical Anthropology
• Physical Anthropology
• Regional Anthropology
• Social and Cultural Anthropology
• Theory and Practice of Anthropology
• Browse content in Business and Management
• Business Ethics
• Business Strategy
• Business History
• Business and Technology
• Business and Government
• Business and the Environment
• Comparative Management
• Corporate Governance
• Corporate Social Responsibility
• Entrepreneurship
• Health Management
• Human Resource Management
• Industrial and Employment Relations
• Industry Studies
• Information and Communication Technologies
• International Business
• Knowledge Management
• Management and Management Techniques
• Operations Management
• Organizational Theory and Behaviour
• Pensions and Pension Management
• Public and Nonprofit Management
• Social Issues in Business and Management
• Strategic Management
• Supply Chain Management
• Browse content in Criminology and Criminal Justice
• Criminal Justice
• Criminology
• Forms of Crime
• International and Comparative Criminology
• Youth Violence and Juvenile Justice
• Development Studies
• Browse content in Economics
• Agricultural, Environmental, and Natural Resource Economics
• Asian Economics
• Behavioural Finance
• Behavioural Economics and Neuroeconomics
• Econometrics and Mathematical Economics
• Economic History
• Economic Systems
• Economic Methodology
• Economic Development and Growth
• Financial Markets
• Financial Institutions and Services
• General Economics and Teaching
• Health, Education, and Welfare
• History of Economic Thought
• International Economics
• Labour and Demographic Economics
• Law and Economics
• Macroeconomics and Monetary Economics
• Microeconomics
• Public Economics
• Urban, Rural, and Regional Economics
• Welfare Economics
• Browse content in Education
• Adult Education and Continuous Learning
• Care and Counselling of Students
• Early Childhood and Elementary Education
• Educational Equipment and Technology
• Educational Strategies and Policy
• Higher and Further Education
• Organization and Management of Education
• Philosophy and Theory of Education
• Schools Studies
• Secondary Education
• Teaching of a Specific Subject
• Teaching of Specific Groups and Special Educational Needs
• Teaching Skills and Techniques
• Browse content in Environment
• Applied Ecology (Social Science)
• Climate Change
• Conservation of the Environment (Social Science)
• Environmentalist Thought and Ideology (Social Science)
• Management of Land and Natural Resources (Social Science)
• Natural Disasters (Environment)
• Pollution and Threats to the Environment (Social Science)
• Social Impact of Environmental Issues (Social Science)
• Sustainability
• Browse content in Human Geography
• Cultural Geography
• Economic Geography
• Political Geography
• Browse content in Interdisciplinary Studies
• Communication Studies
• Museums, Libraries, and Information Sciences
• Browse content in Politics
• African Politics
• Asian Politics
• Chinese Politics
• Comparative Politics
• Conflict Politics
• Elections and Electoral Studies
• Environmental Politics
• Ethnic Politics
• European Union
• Foreign Policy
• Gender and Politics
• Human Rights and Politics
• Indian Politics
• International Relations
• International Organization (Politics)
• Irish Politics
• Latin American Politics
• Middle Eastern Politics
• Political Behaviour
• Political Economy
• Political Institutions
• Political Methodology
• Political Communication
• Political Philosophy
• Political Sociology
• Political Theory
• Politics and Law
• Politics of Development
• Public Policy
• Public Administration
• Qualitative Political Methodology
• Quantitative Political Methodology
• Regional Political Studies
• Russian Politics
• Security Studies
• State and Local Government
• UK Politics
• US Politics
• Browse content in Regional and Area Studies
• African Studies
• Asian Studies
• East Asian Studies
• Japanese Studies
• Latin American Studies
• Middle Eastern Studies
• Native American Studies
• Scottish Studies
• Browse content in Research and Information
• Research Methods
• Browse content in Social Work
• Addictions and Substance Misuse
• Adoption and Fostering
• Care of the Elderly
• Child and Adolescent Social Work
• Couple and Family Social Work
• Direct Practice and Clinical Social Work
• Emergency Services
• Human Behaviour and the Social Environment
• International and Global Issues in Social Work
• Mental and Behavioural Health
• Social Justice and Human Rights
• Social Policy and Advocacy
• Social Work and Crime and Justice
• Social Work Macro Practice
• Social Work Practice Settings
• Social Work Research and Evidence-based Practice
• Welfare and Benefit Systems
• Browse content in Sociology
• Childhood Studies
• Community Development
• Comparative and Historical Sociology
• Disability Studies
• Economic Sociology
• Gender and Sexuality
• Gerontology and Ageing
• Health, Illness, and Medicine
• Marriage and the Family
• Migration Studies
• Occupations, Professions, and Work
• Organizations
• Population and Demography
• Race and Ethnicity
• Social Theory
• Social Movements and Social Change
• Social Research and Statistics
• Social Stratification, Inequality, and Mobility
• Sociology of Religion
• Sociology of Education
• Sport and Leisure
• Urban and Rural Studies
• Browse content in Warfare and Defence
• Defence Strategy, Planning, and Research
• Land Forces and Warfare
• Military Administration
• Military Life and Institutions
• Naval Forces and Warfare
• Other Warfare and Defence Issues
• Peace Studies and Conflict Resolution
• Weapons and Equipment

• < Previous chapter
• Next chapter >

3 Attitudes and Approaches to Representation

• Published: September 1990
• Cite Icon Cite
• Permissions Icon Permissions

This chapter discusses sceptical, empirical, and rational views regarding mental representations. It begins with scepticism because the concerns of the sceptics must be understood if representational theorists are to avoid serious theoretical and empirical errors. Empiricism follows because modern rationalism arose as a reaction to perceived shortcomings in the empiricists' approach and is understandable only against that background. It is shown that the three approaches contact each other in rather curious and surprising ways.

Personal account

• Sign in with email/username & password
• Get email alerts
• Save searches
• Purchase content
• Activate your purchase/trial code
• Add your ORCID iD

Institutional access

Sign in with a library card.

• Sign in with username/password
• Recommend to your librarian
• Institutional account management
• Get help with access

Access to content on Oxford Academic is often provided through institutional subscriptions and purchases. If you are a member of an institution with an active account, you may be able to access content in one of the following ways:

IP based access

Typically, access is provided across an institutional network to a range of IP addresses. This authentication occurs automatically, and it is not possible to sign out of an IP authenticated account.

Choose this option to get remote access when outside your institution. Shibboleth/Open Athens technology is used to provide single sign-on between your institution’s website and Oxford Academic.

• Click Sign in through your institution.
• Select your institution from the list provided, which will take you to your institution's website to sign in.
• When on the institution site, please use the credentials provided by your institution. Do not use an Oxford Academic personal account.
• Following successful sign in, you will be returned to Oxford Academic.

If your institution is not listed or you cannot sign in to your institution’s website, please contact your librarian or administrator.

Enter your library card number to sign in. If you cannot sign in, please contact your librarian.

Society Members

Society member access to a journal is achieved in one of the following ways:

Sign in through society site

Many societies offer single sign-on between the society website and Oxford Academic. If you see ‘Sign in through society site’ in the sign in pane within a journal:

• Click Sign in through society site.
• When on the society site, please use the credentials provided by that society. Do not use an Oxford Academic personal account.

If you do not have a society account or have forgotten your username or password, please contact your society.

Sign in using a personal account

Some societies use Oxford Academic personal accounts to provide access to their members. See below.

A personal account can be used to get email alerts, save searches, purchase content, and activate subscriptions.

Some societies use Oxford Academic personal accounts to provide access to their members.

Viewing your signed in accounts

Click the account icon in the top right to:

• View your signed in personal account and access account management features.
• View the institutional accounts that are providing access.

Signed in but can't access content

Oxford Academic is home to a wide variety of products. The institutional subscription may not cover the content that you are trying to access. If you believe you should have access to that content, please contact your librarian.

For librarians and administrators, your personal account also provides access to institutional account management. Here you will find options to view and activate subscriptions, manage institutional settings and access options, access usage statistics, and more.

Our books are available by subscription or purchase to libraries and institutions.

• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Rights and permissions
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• HHS Author Manuscripts

The representational structure of mental states generalizes across target people and stimulus modalities

Miriam e. weaverdyck.

a Department of Psychology, Princeton University, Princeton, NJ 08544, United States

Mark A. Thornton

Diana i. tamir.

b Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, United States

Author Contributions

Associated Data

Each individual experiences mental states in their own idiosyncratic way, yet perceivers can accurately understand a huge variety of states across unique individuals. How do they accomplish this feat? Do people think about their own anger in the same ways as another person’s anger? Is reading about someone’s anxiety the same as seeing it? Here, we test the hypothesis that a common conceptual core unites mental state representations across contexts. Across three studies, participants judged the mental states of multiple targets, including a generic other, the self, a socially close other, and a socially distant other. Participants viewed mental state stimuli in multiple modalities, including written scenarios and images. Using representational similarity analysis, we found that brain regions associated with social cognition expressed stable neural representations of mental states across both targets and modalities. Together, these results suggest that people use stable models of mental states across different people and contexts.

1. Introduction

Every person – from a romantic partner to a complete stranger – has a unique mind with unique experiences, expressions, and behaviors. This poses a challenge: How do we understand any one person’s idiosyncratic mind? One possibility is that people rely on core mental state concepts that are consistent across people. This approach would allow perceivers to generalize their understanding of mental states–including emotions and cognitive states–across people and situations, rather than laboriously constructing unique models for each mind. If people consistently apply the same core concepts when thinking about mental states, we should see this stable core across diverse targets. We should also observe stability regardless of how perceivers receive information about that person (e.g., through pictures or stories). Here, we explore this possibility by examining the stability of mental state representations in the brain across diverse contexts.

Do people approach each person anew, representing their emotional landscape idiosyncratically? This strategy might be appropriate, given that every person is unique. One’s best friend has a unique face, traits, behavioral patterns, and relationship history that differentiates them from every other unique individual in this world (e.g., Thornton et al., 2018 , Todorov et al., 2007 , Trope and Liberman, 2010 ). Each person’s individuality shapes others’ understandings of that person’s mind and internal states (e.g., Epley, 2008 , Harris and Fiske, 2011 , Haslam and Loughnan, 2014 , Thornton et al., 2019 , Thornton et al., 2018 ). Indeed, we likely consider emotions differently depending on who is experiencing them. A close friend’s joy may be seen as much more positive than a stranger’s joy ( Claypool et al., 2007 ); an enemy’s joy may even be a negative experience ( Cikara and Fiske, 2011 ). These differences in how people consider emotional experiences become particularly stark when comparing one’s own experiences to others’. People represent their own experiences more richly than others’ ( Thornton et al., 2019 ). These differences in experience may arise, in part, due to the type of information available from each (e.g., interoceptive signals and introspection about one’s own mind versus only perceptible cues from others). Thus, when considering someone else’s mind, the uniqueness of that person may overwhelm any similarities in how states are experienced in general.

However, an alternative way to make sense of others’ unique minds would be to apply core concepts of mental states that generalize across people. That is, people may operate upon concepts of mental states that are largely universal ( Thornton et al., 2020 ). Universality in this case does not mean that everyone has the same understanding of mental state concepts. Rather, it means that people have a similar framework they use to understand mental states; individuals’ idiosyncratic representations could still exist within this framework. For example, grief is intense and negative, no matter who is experiencing that grief. If someone tells a friend about a tragic loss, that friend, understanding the nature of grief, can empathize with their experience ( de Vignemont and Singer, 2006 ). Despite any idiosyncrasies that the current situation may impose, both people have a core understanding of what grief is . This shared understanding of emotions has been the basis of much affective research, which implies that there are universal emotions that span age, race, and culture (e.g., Barrett et al., 2007 , Jackson et al., 2019 , Thornton et al., 2020 ). It may be, then, that everyone uses stable, generalizable concepts of mental states when considering what someone else is experiencing. These concepts would define both the primary features of each state individually, as well as how two states relate to one another (e.g., how similar they are).

Functional magnetic resonance imaging (fMRI) provides a unique method for testing if people use a common mental state model across all targets. Specific regions of the brain, including the medial prefrontal cortex (MPFC), anterior temporal lobe (ATL), and temporoparietal junction (TPJ), have been shown to encode and process information related to others’ minds, including representing others’ mental states ( Schurz et al., 2020 ). FMRI allows us to compare neural representations of individual mental state concepts without having to ask the participant, who may not be able to provide an accurate response regarding their understanding of others’ minds ( Nisbett and Wilson, 1977 ). By examining how the brain represents mental state concepts across individuals and modalities using these techniques, we can map the abstract representational space comprised of these mental states. These representations can be extracted for each target individual, and then compared across targets to test for commonalities and distinctions in the models.

Even if people use the same model for everyone’s mental states, each person offers insight into their state in different ways. For example, we can learn about friends experiencing grief through multiple channels. One friend may text you that they are having a rough day; another friend may look at you with sad, tired eyes; a third friend may cry to you over the phone. Past research using isolated expressions of emotion suggests that the human brain encodes affect similarly across auditory (e.g., voice inflection) and visual (e.g., facial expression, body language) modalities ( Bedny et al., 2008 , Chikazoe et al., 2014 , Peelen et al., 2010 ). However, to date, much of neuroimaging research on mentalizing has relied primarily on lexical stimuli (e.g., Meyer et al., 2012 , Mitchell et al., 2002 , Saxe and Kanwisher, 2003 , Skerry and Saxe, 2014 , Tamir et al., 2016 , Theriault et al., 2020 , Thornton et al., 2019 , Thornton et al., 2019 ). Since language and social cognition overlap significantly in the brain (and are both largely distinct from perceptual regions), this may contribute to findings that there is some stable structure to neural representations of mental states ( Thornton et al., 2019 ). If humans do use generalizable core concepts of mental states across modalities, then we should see similar neural representations of each mental state across visual and lexical stimuli. That is, watching a friend in agony should generate the same neural representation as reading a description of it. However, since pictures and phrases necessarily contain different types of information, it is also possible that people process these mental states uniquely depending on how the information is presented. Here we test how people represent supramodal mental states to see if core mental state concepts generalize across modalities.

Here, we tested two specific hypotheses about how people represent mental state concepts. First, we expect that the structure of neural representations associated with mental states should remain consistent across multiple targets – from the self to dissimilar others. Second, we expect that neural patterns associated with mental states should remain consistent across modalities – including lexical and pictorial stimuli.

2. Material and methods

To test our hypotheses, we combined data from three fMRI studies in which participants considered different individuals experiencing a variety of mental states using either lexical or pictorial stimuli. By analyzing across studies, we can assess the stability of mental state representations across (a) target people and (b) stimulus modalities. Moreover, this approach provides a severe test of the generalizability, since participants, fMRI scanners, time, and institutions differ between studies as well.

2.1. Use of published data

In this manuscript, we use datasets that have been analyzed and published in previous manuscripts. Each study’s materials and analyses are freely available on the Open Science Framework. Here, we refer to these as Study 1 ( https://osf.io/3qn47/ ; Tamir et al., 2016 , Thornton et al., 2018 ), Study 2 and Study 3 (studies 2 and 1 in https://osf.io/hp5wc/ , respectively; Thornton et al., 2019 ).

2.2. Participants

Participants in Study 1 ( N = 20; 16 female, 4 male; 18-27 years, M = 22.7 years) were recruited from the Harvard University Study Pool. The participants in Study 2 ( N = 35; 23 female, 12 male; 18-31 years, M = 21 years) and Study 3 ( N = 30; 14 female, 15 male, 1 nonbinary; 19-27 years, M = 20 years) were recruited from the Princeton University Credit and Paid Study Pools. The sample size in Study 1 was determined via Monte Carlo simulation parameterized based on effect sizes in previous studies of similar design ( Tamir et al., 2016 ). The sample sizes for Studies 2 and 3 were determined a priori to be able to detect the smallest effect found in Study 1 (namely, the relationship between neural representations of mental state and behavioral ratings of valence) with 95% power ( Thornton et al., 2019 ). All participants were right-handed or ambidextrous, fluent English-speakers, reported no history of neurological problems, had normal or corrected-to-normal vision, and were screened for standard MRI exclusion criteria. Participants were compensated with university credit or monetary payment. All data were collected in accordance with the Harvard University Committee on the Use of Human Subjects (Study 1) or Princeton University Institutional Review Board (Studies 2 and 3).

2.3. Experimental design

In all three studies, participants made judgments about a wide range of mental states. On each trial, participants were shown a prompt with a mental state word (e.g., “peacefulness”) at the top of the screen, followed by two scenarios pre-tested to elicit that mental state. Participants chose which of the two scenarios, presented in either lexical or pictorial form, would most likely elicit the current mental state in a particular target person.

There were four targets across all three studies. In Study 1, participants chose for a generic other (no target description was given beyond “another person”); in Studies 2 and 3, participants chose for a specific target individual (shown along with the mental state prompt) at various levels of psychological distance from the subject: self, close other, or far other ( Fig. 1 , Table 1 ). Self-trials were cued with the word “SELF ”; socially close and far targets were cued with the name of that person (e.g., “KATE”). Close targets were nominated by the participant to be likable, familiar, and similar to themselves (i.e., socially close). Far targets were introduced to the participant via a short biographical statement (see Supplement S9 for example bios). The experimenters created these fictional targets to be dissimilar to the participant in terms of their college major, religiosity, political party and attitudes, and extracurricular activities. Thus, the far target was both unfamiliar and dissimilar to the participant, thereby maximizing the social distance between the target and the participant.

In all three studies, participants were shown a prompt with the current mental state (e.g., peacefulness) followed by two scenarios that elicit that mental state. On each trial, participants had to decide which of the two scenarios would most likely elicit the current state in the current target. In Study 1, they chose for a generic other; in Studies 2 and 3, participants chose for a specific target. Self trials were cued with the word “SELF”; socially close and far targets were cued with the name of that person (e.g., “KATE”). Close targets were nominated by the participant, while far targets were created by the experimenter to be unfamiliar and dissimilar to the participant. Scenarios were presented as a short phrase in Studies 1 and 2, and as images in Study 3.

Experimental design parameters.

An initial list of 166 mental state words was reduced to an optimized set of 60 states ( Tamir et al., 2016 ). These 60 states were selected for minimal redundancy and to uniformly cover the conceptual space of mental states along many different psychological dimensions ( Tamir et al., 2016 ). All 60 states were used in Study 1; Studies 2 and 3 consisted of subsets of these 60 ( Table 1 ; see Table S1 for full list of state words) that sampled the entire space based on the variance observed in Study 1. Participants saw each target-mental state pair once per run.

Each mental state word and target cue (if applicable) was followed by two scenarios likely to elicit that state in the average person. The two scenarios were drawn from a unique set associated with that mental state. The modality of the scenario stimuli differed by study ( Fig. 1 ): In Studies 1 and 2, scenarios were presented as short text phrases (e.g., “watching the sun rise,” “engaging in meditation”). In Study 3, scenarios were presented as images. These images sometimes consisted of a person experiencing the mental state (e.g., a picture of a person meditating), and sometimes only showed the scenario (e.g., a picture of the sun rising). Participants’ task was to choose which scenario would most likely elicit the mental state in the target person.

There was no “correct” choice, and there were a large number of possible scenario combinations across studies. For this reason, we chose not to analyze scenario choice as related to neural responses in the current project. Response rates were used in the original studies to exclude participants ( Tamir et al., 2016 , Thornton et al., 2019 , Thornton et al., 2018 ). In the current datasets, response rates were very high ( M Study1 = 92.44%, SD Study1 = 9.19; M Study2 = 96.31%, SD Study2 = 4.24; M Study3 = 96.71%, SD Study3 = 4.78) participants chose the right-hand option approximately 50% of the time ( M Study1 = 51.43%, SD Study1 = 5.80; M Study2 = 49.75%, SD Study2 = 4.57; M Study3 = 52.33%, SD Study3 = 5.06), and participants had reasonable reaction times ( M Study1 = 2.35s, SD Study1 = 0.35; M Study2 = 1.99s, SD Study2 = 0.33; M Study3 = 1.77s, SD Study3 = 0.25), suggesting that participants were alert and focused on the task. We also examined how frequently participants chose the same scenario for different targets in Studies 2 and 3 (where more than one target was presented). We found that the similarity in which scenarios were chosen for different targets aligned with the psychological distance of the target. That is, when choosing which scenario would most likely elicit a mental state, people most often chose the same option for the self and close targets, while they were most likely to choose different scenarios for the self and far targets (see Supplement S8 ).

Each trial was followed by jittered fixation drawn from approximate Poisson distributions with intervals equal to the study’s TR and the following means: M 1 = 1.67s, M 2 = 1.53s, M 3 = 1.4 s. Study 1 included 0.25 s of fixation in addition to the jittered fixation time.

2.4. FMRI data acquisition and analysis

2.4.1. preprocessing.

fMRI data from each study were preprocessed similarly ( Table 2 ). Data from all studies underwent coregistration and normalization to the 2 mm isotropic ICBM 152 template with SPM8 (default implementation in Study 1, DARTEL in Studies 2 and 3). FSL’s slicetime correction and unwarping were applied in Studies 2 and 3. No explicit smoothing was applied at this stage.

MRI acquisition parameters for all three studies.

2.4.2. General linear model contrasts

Preprocessed fMRI data were entered into general linear models (GLM) using SPM 8 and the wrapper package SPM8w ( https://github.com/ddwagner/SPM8w ) in Study 1, and SPM12 via SPM12w ( https://github.com/wagner-lab/spm12w ) in Studies 2 and 3. Boxcar regressors were created for each condition of interest (i.e., every target-state pairing). Note, while Study 2 trials were grouped by target person, no blocked regressors were included. In Study 2, the similarity matrices produced from the GLM contrasts were corrected for spurious correlations arising from the blocked design ( Thornton et al., 2019 ). There were no other substantial correlations among the regressors of interest ( Fig. S5 ). Trial onsets aligned with the presentation of the prompt, and a trial ended when the participant made a choice or when time maxed out (see Table 1 Scenario Duration). These regressors were convolved with a canonical hemodynamic response function and entered into the GLM along with covariates of no interest, including run means and trends, and six head motion parameters. GLM analyses resulted in one contrast map per target-state pair (Study 1: 1 modality × 1 target × 60 states = 60 maps; Study 2: 1 modality × 3 targets × 25 states = 75 maps; Study 3: 1 modality × 2 targets × 30 states = 60 maps) per participant. These maps represent how participants thought about each target person experiencing each state in each modality condition. These resulting patterns of contrast values were used in all subsequent analyses.

2.4.3. Representational, similarity analysis

Our primary analyses tested the stability of mental state representations in the brain. To do so, we used representational similarity analysis (RSA) and suppressed intercept linear mixed models, as follows. RSA reflects the overall structure of mental state representations by measuring how similar each mental state representation is to every other mental state representation. We can thus use RSA to compare neural representations ( Kriegeskorte et al., 2008 ). Specifically, we can look at the structure people apply for each target, and compare these structures across targets. If people use similar representational structures across different targets, this would mean that they apply similar mental state concepts across targets. We tested whether these structures were significantly similar to each other using suppressed intercept mixed models. These models include a coefficient that reflects the extent of this structural similarity that is not dependent on individual neural patterns being consistent across participants.

All analyses were conducted across the entire brain using a functional whole-brain parcellation ( https://identifiers.org/neurovault.collection:2099 ). The 200 parcels in this functional parcellation were defined based on meta-analytic coactivations in Neurosynth ( neurosynth.org ; de la Vega et al., 2016 ). This functional parcellation divides the brain into discrete regions using a data-driven approach to determine which voxels respond similarly throughout past research. As such, we avoid imposing artificial region shapes and sizes via the searchlight approach and reduce the number of multiple comparisons that we need to correct for. This, in turn, increases our statistical power and reduces the computational demand of running mixed models at every point in the brain. For each participant, within each parcel, we generated target-specific representational similarity matrices (RSMs) by calculating the Pearson correlation between every pair of mental state patterns within a single target ( Fig. 2b ). This resulted in one 60 × 60 matrix for the one (generic) target in Study 1, three 25 × 25 matrices for the three (self, close, and far) targets in Study 2, and two 30 × 30 matrices for the two (self and far) targets in Study 3, per parcel, per subject. These RSMs are a measurement of how people think about mental state concepts for each target.

Within each participant and neural parcel, (a) neural response patterns elicited by each target-state pair were extracted and (b) compared via Pearson correlation. These correlation coefficients were then organized into target-specific RSMs. Each of these first-order RSMs was (c) Spearman correlated with every other RSM, within and across studies. These values were arctan transformed and organized into a second-order RSM. This second-order neural RSM was entered into suppressed intercept models ( Fig. 3 ).

Studies 2 and 3 included different subsets of the mental states included in Study 1. To allow for comparison across studies, we expanded all matrices to 60 × 60 with empty rows for mental states that were not shown in that study. This allowed us to calculate the Spearman rank correlation between each of these first-order RSMs (which consists of continuous correlation values) to create a single second-order RSM across all targets, participants, and studies ( Fig. 2c ; see Fig. S6 for the average second-order RSM). Note that, because mental states were imbalanced across the three datasets, Study 1 may have been driving the structure of these first-order mental state geometries. However, when we exclude Study 1, we found highly congruent results ( Fig. S1 ). The final second-order RSM consisted of 185 rows and columns: one per target per participant per modality/study (20 participants × 1 target × 1 modality + 35 participants × 3 targets × 1 modality + 30 participants × 2 targets × 1 modality = 185).

The first-order RSMs reflect the overall structure, or geometry, of the mental state representational space that a participant held for a particular target in a particular modality. The second-order RSM shows the similarity of these geometries across targets, participants, studies, and modalities. This second-order RSM thus allows us to answer the following questions: Did the same participant think about two different targets’ states similarly? Did a participant in Study 2 (lexical stimuli) think about the far target’s mental states similarly to how a participant in Study 3 (pictorial stimuli) thought about the far target’s mental states?

We conducted three analyses to test if neural representations of mental states remain stable across different (i) targets, (ii) modalities, and (iii) targets and modalities, respectively. Each analysis must be run separately rather than including them all into one model. This is because we use suppressed intercept models, which allows us to model similarity (rather than differences) across targets and/or modalities. If we were to combine all three analyses into one, the model would become rank-deficient.

All three stability analyses proceeded similarly. For example, for the target analyses, if people use the same mental state concepts when thinking about different targets, then we should see similar (i.e., correlated) first-order RSMs associated with different targets. Each correlation value between two targets’ first-order RSMs corresponds to a cell in the second-order RSM ( Fig. 2b – c ). Thus, we can test how similarly people think about different targets’ mental states by looking at the mean of the cells in the second-order RSM that refer to first-order RSMs from different targets. That is, we measure the average correlation value between first-order RSMs of different targets and test if that value is greater than 0. If so, this suggests a common structure to the mental state spaces (i.e., first-order RSMs) even across different targets.

We implemented this test by applying suppressed intercept linear mixed effects models. In this analysis, we try to explain how similar the representational structure of mental states is based primarily on the target person. To test for this effect of interest–namely, if there is a similar structure across different target people–we included two binary fixed effect predictors for same-target vs. different-target data ( Fig. 3a ). The same-target predictor has 1 s in cells that compare data from the same target and 0s in all other cells. The different-target predictor is the inverse of the same-target predictor and includes 1 s in cells that compare first-order RSMs from different targets and 0s elsewhere.

In separate suppressed-intercept linear mixed effects models, we tested which brain regions hold stable representations of mental state concepts across (a) targets or (b) modalities. (c) The second-order neural RSM ( Fig. 2c ) was the dependent variable in each mixed-effects model with individual predictors for same and different targets/modalities, and random effects for subject, study, and modality/target. (d) Because we suppressed the intercept (set equal to 0), the different targets/modalities coefficient (blue with asterisk) reflected the mean correlation values of cells that compare first-order RSMs from different targets/modalities (dark cells). That is, it measured the average similarity of mental state representations across targets/modalities. If this coefficient was significantly greater than 0, then the relevant brain region showed stable structures of mental state representation across targets/modalities.

Unlike a typical linear (mixed) model, in which the beta reflects the difference between the levels of a categorical variable, our suppressed intercept model (i.e., the intercept is set to 0) coefficients represent the means of each condition. In other words, each beta corresponds to the mean of all cells (in the second-order RSM) that correspond with a 1 in that predictor RSM (rather than the difference between cells with a 1 and cells with a 0; Fig. 3d ). Because we are not modeling the intercept, it is necessary to include both of these fixed effects in order to model the within-target similarity and the between-target similarity, and thereby also the overall mean of the data (see Supplement S7 ; Fig. S4 ). Thus, in this model, the beta value for the different-target predictor represents the average similarity in the structure of mental state representation between different target people. By running significance testing (ß > 0) on this value, we determined whether or not the overall structure of mental states was similar across different targets in a given parcel. The model also included random intercepts by participant and study, along with a random intercept to account for mental states in the same modality ( Fig. 3a ).

We tested this model across the entire brain. Specifically, we fit the model in each of the 200 regions of our parcellation (see above for description of parcellation, Section 2.4.3 ) to map out where mental state representations are similar across different targets. The resulting p -values were calculated via Satterthwaite approximation for degrees of freedom, and multiple comparisons across the 200 parcels were controlled via Holm correction.

To test the stability of mental state representations across modalities, we conducted the same type of analysis with slightly different predictors: as fixed effects, we included one same-modality predictor, which indicates whether or not a cell compares data from the same modality (1 = same modality, 0 = different modalities; Fig. 3b ). We also included one different-modality predictor that reflects whether or not a cell compares data from different modalities (1 = different modalities, 0 = same modality). As above, both terms are necessary to model both within-modality similarity and between-modality similarity. The beta value for the different modality predictor represents how similarly participants’ representational spaces of mental states (i.e., the first-order RSMs) are across studies that used different modalities. We also included random effects to account for mental states within the same target person, participant, and study ( Fig. 3b ). After fitting the model in every parcel, we performed significance testing (using the same procedure described above) on the different-modality coefficient to determine where representations of mental states are stable across different stimulus modalities ( Fig. 3b ).

Finally, we tested if there were regions with stable mental state representations across both targets and modalities using the same process. In this suppressed intercept linear mixed effects model, we included the following predictors: as fixed effects, we included one same-target/modality predictor, which indicates whether or not a cell compares data from the same target or the same modality (1 = same target or same modality, 0 = different targets and different modalities). We also included one different-target/modality predictor that reflects whether or not a cell compares data from different targets and different modalities (1 = different targets and different modalities, 0 = same target or same modality). The beta value for the different-target/modality predictor represents how similarly participants think about mental state concepts across different targets and different modalities. We also included random effects to participant and study. After fitting the model in every parcel, we performed the same significance testing described above on the different-target/modality coefficient to determine where representations of mental states are stable across different target people and stimulus modalities.

3.1. Stability of neural representations across targets

Do people think about mental states in the same way across different targets, or do people think about each target’s mind uniquely? To test this, we compared neural representations of states across different target people. We did so using representational similarity analysis (RSA) and a suppressed intercept linear mixed effects model to quantify the extent to which each parcel represented mental states similarly across target people. Significance testing on this coefficient showed robustly stable representations of mental states across different targets, specifically within regions associated with social cognition and mental state representation. These regions include the ventral medial prefrontal cortex (vMPFC), dorsal medial prefrontal cortex (dMPFC), precuneus, bilateral temporoparietal junction (TPJ), and anterior temporal lobe (ATL; Fig. 4a ; Table 3 ). These regions represent others’ mental states in similar ways regardless of who is thought to be experiencing these states. We found convergent results when analyzing Studies 2 and 3 independently (see Supplement S4 ).

Participants showed stable neural representations of mental states across (a) target people and (b) stimulus modalities in parcels associated with social cognition. A subset of these regions (parts of the MPFC, dlPFC, and bilateral TPJ) showed stable representations across (c) both targets and modalities. In addition, the precuneus showed stable representations across targets. Only significant t -values ( α = .05) after correcting for multiple comparisons are shown.

Parcels that stably represent mental states across targets and modalities.

Note. Only regions with a trending or significant result are displayed. All p -values are corrected for multiple comparisons using the Holm method

3.2. Stability of neural representations across modalities

Do people think about mental states in the same way whether they see the situation or read about it? To test for supramodal representations of mental states, we compared neural representations of states across modalities. We did so using a suppressed intercept linear mixed effects model with RSA to quantify the extent to which parcels represented mental states similarly across stimulus modalities. Similar to the target stability results, we found that large portions of the social brain network, including MPFC, ATL, and bilateral TPJ, showed robustly stable neural representations of mental states across modalities. These results further suggest that these areas encode generalizable representations of mental states ( Fig. 4b ; Table 3 ).

3.3. Stability of neural representations across targets and modalities

Are there brain regions that hold stable representations of mental state concepts across both targets and modalities? Once again, we used a suppressed intercept linear mixed effects model with RSA to quantify the extent to which parcels represented mental states similarly across target people and stimulus modalities. Once again, we found areas of the MPFC, left dlPFC, and TPJ, as well as the dACC and left insula represented mental states similarly across both contextual changes ( Fig. 4c ; Table 3 ).

4. Discussion

Each individual experiences mental states in their own idiosyncratic way. Yet, perceivers are able to accurately understand this huge variety of states across the uniqueness of each individual and context. How do people accomplish this feat? Here we demonstrate that people do so by drawing upon a core model of mental state concepts. Across three studies, we find that people apply the same fundamental understanding of mental states, no matter who is experiencing a particular state or how they take in that state. Together, these findings suggest that people’s knowledge of mental states is encoded consistently across people and modalities.

We found robustly stable representations in a subset of the regions implicated in social cognition known as the default mode network ( Mars et al., 2012 , Mitchell, 2008 , Schurz et al., 2020 , Van Overwalle and Baetens, 2009 ). Specifically, the MPFC, ATL, and TPJ showed robust stability in mental state representations across both targets and modalities ( Fig. 4 ; Table 3 ). All of these parcels are consistent with areas associated with mentalizing processes, suggesting that not only do these parcels support social cognition, they also support generalized mental state concepts across contextual specifics.

That said, the precuneus generalized across targets, but not modalities. Given that the two stimulus modalities we used were words and images, it makes sense that the precuneus, which supports mental imagery processes, might not represent mental states similarly across these different visual modalities. Similarly, a portion of the dorsal MPFC generalized across modalities, but not targets, and encoded target- specific mental state concepts (see Supplement S5 ; Fig. S2 ). The dMPFC has been shown to respond differently to similar and dissimilar others ( Mitchell et al., 2006 , Tamir and Mitchell, 2010 ). The current finding supports the idea that different regions play distinct roles in supporting social cognitive processes, and that the dMPFC in particular, may serve to help individualize targets. This and other regions across the social brain, however, respond similarly across modalities. This finding is particularly striking when considering the vast differences in informational cues that we have for understanding and experiencing our states of mind, compared to someone else’s. While we are the ultimate authorities on our own feelings, we must rely on entirely external cues to understand what another person is feeling, which will never provide enough information for us to understand their experience fully. Yet, our results suggest that we use a generalizable model of how mental states relate to one another across diverse modalities. These findings are congruent with past literature showing that our brain represents supramodal representations of emotions ( Chikazoe et al., 2014 , Peelen et al., 2010 , Skerry and Saxe, 2014 ) and words ( Marinkovic et al., 2003 ). This prior work specifically finds that portions of the MPFC, ATL, and TPJ encode mental states as core concepts, invariant across different contexts. Importantly, the current findings do not preclude the possibility that other regions also have generalized representations of mental state concepts that we could not detect with the present design, or that regions associated with mental state representation can be modulated by contextual features. Rather, our results suggest that after filtering out the context-specific factors, brain regions spanning the default mode network encode a common core structure of how mental states relate to one another across a diverse range of modalities.

These findings suggest that people use a similar core model of emotions for the self and others. We suspect that people develop this generalizable amodal model of mental state representation through the convergence of knowledge about one’s own experiences with observations of others’ experiences. For example, if someone grew up in a place where everyone was quick to anger, then their mental state model would place anger in a more prominent node than someone who grew up in a more placid environment ( Thornton and Tamir, 2021 ). That is, the landscapes of others’ emotions and the cultural norms in which one is embedded define each individual’s context-independent model. This model, in turn, constrains both one’s personal emotional landscape, as well as one’s perception of others’. Likewise, a person’s own experiences can constrain how they perceive others’ experiences. People are often egocentric in their social inferences, including their inferences about others’ emotions ( Trilla et al., 2020 ). For example, an individual who is often quick to anger might overperceive anger in others. This bidirectional pathway between learning and inference has implications for both emotional contagion and clinical research. In clinical psychology, mood disorders affect how a person thinks about their own feelings ( Leppänen, 2006 ). Since our results suggest that people use the same model to understand others’ mental states as their own, mood disorders may impact a person’s social perception of others in the same way.

Our results offer robust evidence for generalization across diverse targets and stimuli. However, we note at least three limitations to generalizability. First, our samples of all three datasets were not diverse in age, demographics, education, location, and more. As such, future researchers should practice caution before generalizing these results to other populations. Future research should explore if these results hold in different cultures, especially since recent research suggests large cultural variability in emotion concepts and expression ( Jackson et al., 2019 ). Second, the stimuli used were relatively artificial (short phrases, static images) compared to the rich sources of information used in everyday life. We did not explore other modalities commonly used when making inferences about others’ minds, including speech and dynamic stimuli. These other types of information sources may provide further insight into when and why people use this generalizable framework. Finally, this project focused on the neural representations of mental state concepts . As such, our results cannot speak to how people think about mental state experiences . First-hand experiences of states may well show greater variation than the generalizable mental state concepts discussed here.

5. Conclusion

When interacting with others, we must consider their perspective, thoughts, and feelings. Here, we found that, while context may modulate processing of this information, we likely draw on core features of these mental states that provide a consistent generalizable amodal model across the varying situations of our daily lives. This shared core can facilitate deeper understanding of each other’s internal lives.

Supplementary Material

Acknowledgments.

The authors would like to thank Elisa Baek for comments on previous versions of this manuscript.

This work was supported by the National Institute of Mental Health R01MH114904 (D.I.T); and the National Science Foundation Graduate Research Fellowship Program (M.E.W.).

Open Practices Statement

All de-identified data and analysis scripts have been made publicly available via the Open Science Framework and are accessible at https://osf.io/z3xs9/ .

Supplementary materials

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.neuroimage.2021.118258 .

• Barrett LF, Mesquita B, Ochsner KN, Gross JJ, 2007. The experience of emotion . Annu. Rev. Psychol 58 ( 1 ), 373–403. doi: 10.1146/annurev.psych.58.110405.085709. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bedny M, Caramazza A, Grossman E, Pascual-Leone A, Saxe R, 2008. Concepts are more than percepts: the case of action verbs . J. Neurosci 28 ( 44 ), 11347–11353. doi: 10.1523/JNEUROSCI.3039-08.2008. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Chikazoe J, Lee DH, Kriegeskorte N, Anderson AK, 2014. Population coding of affect across stimuli, modalities and individuals . Nat. Neurosci 17 ( 8 ), 1114–1122. doi: 10.1038/nn.3749. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cikara M, Fiske ST, 2011. Bounded empathy: neural responses to outgroup targets’ (mis)fortunes . J. Cognit. Neurosci 23 ( 12 ), 3791–3803. doi: 10.1162/jocn_a_00069. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Claypool HM, Hugenberg K, Housley MK, Mackie DM, 2007. Familiar eyes are smiling: on the role of familiarity in the perception of facial affect . Eur. J. Soc. Psychol 37 ( 5 ), 856–866. doi: 10.1002/ejsp.422. [ CrossRef ] [ Google Scholar ]
• de la Vega A, Chang LJ, Banich MT, Wager TD, Yarkoni T, 2016. Large-scale meta-analysis of human medial frontal cortex reveals tripartite functional organization . J. Neurosci 36 ( 24 ), 6553–6562. doi: 10.1523/jneurosci.4402-15.2016. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• de Vignemont F, Singer T, 2006. The empathic brain: how, when and why? Trends Cognit. Sci 10 ( 10 ), 435–441. doi: 10.1016/j.tics.2006.08.008. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Epley N, 2008. Solving the (real) other minds problem . Soc. Personal. Psychol. Compass 2 ( 3 ), 1455–1474. doi: 10.1111/j.1751-9004.2008.00115.x. [ CrossRef ] [ Google Scholar ]
• Harris LT, Fiske ST, 2011. Dehumanized perception . Z. Psychol 219 ( 3 ), 175–181. doi: 10.1027/2151-2604/a000065. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Haslam N, Loughnan S, 2014. Dehumanization and infrahumanization . Annu. Rev. Psychol 65 ( 1 ), 399–423. doi: 10.1146/annurev-psych-010213-115045. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Jackson JC, Watts J, Henry TR, List J, Forkel R, Mucha PJ, …, Lindquist KA, 2019. Emotion semantics show both cultural variation and universal structure . Science 366 ( 6472 ), 1517–1522. doi: 10.1126/science.aaw8160. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kriegeskorte N, Mur M, Bandettini P, 2008. Representational similarity analysis – connecting the branches of systems neuroscience . Front. Syst. Neurosci 2 ( 4 ), 1–28. doi: 10.3389/neuro.06.004.2008. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Leppänen JM, 2006. Emotional information processing in mood disorders: a review of behavioral and neuroimaging findings . Curr. Opin. Psychiatry 19 ( 1 ), 34–39. doi: 10.1097/01.yco.0000191500.46411.00. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Marinkovic K, Dhond RP, Dale AM, Glessner M, Carr V, Halgren E, 2003. Spatiotemporal dynamics of modality-specific and supramodal word processing . Neuron 38 ( 3 ), 487–497. doi: 10.1016/S0896-6273(03)00197-1. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mars RB, Neubert FX, Noonan MAP, Sallet J, Toni I, Rushworth MFS, 2012. On the relationship between the “default mode network” and the “social brain . Front. Hum. Neurosci 6 ( 189 ), 1–9. doi: 10.3389/fnhum.2012.00189. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Meyer ML, Spunt RP, Berkman ET, Taylor SE, Lieberman MD, 2012. Evidence for social working memory from a parametric functional MRI study . Proc. Natl. Acad. Sci 109 ( 6 ), 1883–1888. doi: 10.1073/pnas.1121077109. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mitchell JP, 2008. Contributions of functional neuroimaging to the study of social cognition . Curr. Direct. Psychol. Sci 17 ( 2 ), 142–146. doi: 10.1111/j.1467-8721.2008.00564.x. [ CrossRef ] [ Google Scholar ]
• Mitchell JP, Heatherton TF, Macrae CN, 2002. Distinct neural systems subserve person and object knowledge . Proc. Natl. Acad. Sci 99 ( 23 ), 15238–15243. doi: 10.1073/pnas.232395699. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mitchell JP, Macrae CN, Banaji MR, 2006. Dissociable medial prefrontal contributions to judgments of similar and dissimilar others . Neuron 50 ( 4 ), 655–663. doi: 10.1016/j.neuron.2006.03.040. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Nisbett RE, Wilson TD, 1977. Telling more than we can know: Verbal reports on mental processes . Psychol. Rev 84 ( 3 ), 231–259. doi: 10.1037/0033-295X.84.3.231. [ CrossRef ] [ Google Scholar ]
• Peelen MV, Atkinson AP, Vuilleumier P, 2010. Supramodal representations of perceived emotions in the human brain . J. Neurosci 30 ( 30 ), 10127–10134. doi: 10.1523/JNEUROSCI.2161-10.2010 . [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Saxe R, Kanwisher N, 2003. People thinking about thinking peopleThe role of the temporo-parietal junction in “theory of mind . NeuroImage 19 ( 4 ), 1835–1842. doi: 10.1016/S1053-8119(03)00230-1. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Schurz M, Radua J, Tholen MG, Maliske L, Margulies DS, Mars RB, …, Kanske P, 2020. Toward a hierarchical model of social cognition: a neuroimaging meta-analysis and integrative review of empathy and theory of mind . Psychol. Bull doi: 10.1037/bul0000303. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Skerry AE, Saxe R, 2014. A common neural code for perceived and inferred emotion . J. Neuroscience 34 ( 48 ), 15997–16008. doi: 10.1523/JNEUROSCI.1676-14.2014. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Tamir DI, Mitchell JP, 2010. Neural correlates of anchoring-and-adjustment during mentalizing . Proc. Natl. Acad. Sci 107 ( 24 ), 10827–10832. doi: 10.1073/pnas.1003242107. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Tamir DI, Thornton MA, Contreras JM, Mitchell JP, 2016. Neural evidence that three dimensions organize mental state representation: rationality, social impact, and valence . Proc. Natl. Acad. Sci 113 ( 1 ), 194–199. doi: 10.1073/pnas.1511905112. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Theriault J, Waytz A, Heiphetz L, Young L, 2020. Theory of mind network activity is associated with metaethical judgment: an item analysis . Neuropsychologia 143 ( January ), 107475. doi: 10.1016/j.neuropsychologia.2020.107475. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Thornton MA, & Tamir DI (2021) (n.d.). The organization of social knowledge is tuned for prediction. In Gilead M & Ochsner KN (Eds.), The Neural Bases of Mentalizing . Springer Press. 10.1007/978-3-030-51890-5_14 [ CrossRef ] [ Google Scholar ]
• Thornton MA, Weaverdyck ME, Mildner JN, Tamir DI, 2019. People represent their own mental states more distinctly than others . Nat. Commun 10 ( 2117 ), 1–9. doi: 10.1038/s41467-019-10083-6 . [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Thornton MA, Weaverdyck ME, Tamir DI, 2018. The brain represents people as the mental states they habitually experience . Nat. Commun ( 2019 ) 1–10. doi: 10.1038/s41467-019-10309-7. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Thornton MA, Wolf S, Reilly BJ, Slingerland EG, Tamir DI, 2020. The 3d mind model characterizes how people understand mental states across modern and historical cultures . PsyArXiv doi: 10.31234/osf.io/m5p74. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Todorov A, Gobbini MI, Evans KK, Haxby JV, 2007. Spontaneous retrieval of affective person knowledge in face perception . Neuropsychologia 45 ( 1 ), 163–173. doi: 10.1016/j.neuropsychologia.2006.04.018. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Trilla I, Weigand A, Dziobek I, 2020. Affective states influence emotion perception: Evidence for emotional egocentricity . Psychol. Res doi: 10.1007/s00426-020-01314-3. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Trope Y, Liberman N, 2010. Construal-level theory of psychological distance . Psychol. Rev 117 ( 2 ), 440–463. doi: 10.1037/a0018963. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Van Overwalle F, Baetens K, 2009. Understanding others’ actions and goals by mirror and mentalizing systems: a meta-analysis . NeuroImage 48 ( 3 ), 564–584. doi: 10.1016/j.neuroimage.2009.06.009. [ PubMed ] [ CrossRef ] [ Google Scholar ]

Entity Realism About Mental Representations

• Original Research
• Open access
• Published: 26 October 2019
• Volume 87 , pages 75–91, ( 2022 )

Cite this article

You have full access to this open access article

• Bence Nanay   ORCID: orcid.org/0000-0002-2835-6530 1 , 2  

6021 Accesses

6 Citations

3 Altmetric

Explore all metrics

The concept of mental representation has long been considered to be central concept of philosophy of mind and cognitive science. But not everyone agrees. Neo-behaviorists aim to explain the mind (or some subset thereof) without positing any representations. My aim here is not to assess the merits and demerits of neo-behaviorism, but to take their challenge seriously and ask the question: What justifies the attribution of representations to an agent? Both representationalists and neo-behaviorists tend to take it for granted that the real question about representations is whether we should be realist about the theory of representationalism. This paper is an attempt to shift the emphasis from the debate concerning realism about theories to the one concerning realism about entities. My claim is that regardless of whether we are realist about representational theories of the mind, we have compelling reasons to endorse entity realism about mental representations.

Similar content being viewed by others

Being and Describing: An Entity Realist Appraisal of Internal Realism

Representationalism and rationality: why mental representation is real.

Representation, Internal

Avoid common mistakes on your manuscript.

1 Introduction: Mental Representations

The concept of mental representation is a central concept of cognitive science and of philosophy of mind. When we describe any of our actions, or actions of other people, it is difficult to do so without talking about representations: I opened the window because I thought it was too hot and I did not want to turn on the air conditioner. This reference to thoughts and wants, to beliefs and desires has become the overarching framework for the philosophy of mind of the last 30 years at least: much of recent philosophy of mind is about the nature of these propositional attitudes, their relations to one another and to the brain.

Some philosophers of mind and most cognitive scientists use a much less restrictive concept of representation, which includes much more than just beliefs, desires and other propositional attitudes. Importantly, cognitive scientists (and some philosophers of mind) talk about representations in the perceptual system and representations that allow us to perform actions successfully. In fact, the very origins of cognitive science could be traced to the opposition to behaviorism in positing some kind of representations in the mind (again, not necessarily beliefs or desires).

This emphasis on representations provided a common ground for philosophy of mind and cognitive science that facilitated interaction between the two disciplines. But this emphasis on representations has been more recently questioned by what I will call – with some malice – ‘neo-behaviorist’ views of the mind.

The neo-behaviorists attempt to explain much of what goes on in our mind without any reference to representations. One motivation for the neo-behaviorist approach comes from worries about over-intellectualizing the mind (Hurley 2001 ). If we describe our mental life as the combination of beliefs, desires and other propositional attitudes, this makes it problematic to talk about animal minds and the minds of young children, but also to talk about many of our everyday actions without over-intellectualization. The neo-behaviorists, rather than trying to remedy this issue by being more precise about what kind of representations we use when describing the mind, reject any concept of representations at least when describing simple actions (of children, animals and of adult humans).

Neo-behaviorism comes in different flavors, some more radical than others. Most proponents of this view would accept that some complex, maybe linguistic human behavior could not be described without talking about representations, but insist that the vast majority of our actions and also our perception can be fully explained in non-representationalist terms (Chemero 2009 ; Hutto and Myin 2014 ; see also Ramsey 2007 for a nuanced summary of this approach).

Many neo-behaviorists focus on perception and argue that perception is not a representational process: there are no perceptual representations. As Dana Ballard put it, “the world is the repository of the information needed to act. With respect to the observer, it is stored ‘out there’, and by implication not represented internally in some mental state that exists separately from the stimulus” (Ballard 1996 , p. 111; see also Brooks 1991 ). There are two major alternatives neo-behaviorists give to the representationalist framework. Some of them think of perception as an active process, some form of dynamic interaction with the world, which does not require representations (Noë 2004 ; Hurley 2001 ). Some others take perception to be a non-representational relation to the perceived object (Campbell 2002 ; Martin 2004 ; Travis 2004 ; Brewer 2011 ).

My aim here is not to assess the merits and demerits of these views or of neo-behaviorism in general. I am a representationalist and I will argue for representationalism in this paper. However, I will do so not by criticizing the neo-behaviorist views, but by taking their challenge seriously and asking the question: What justifies the attribution of representations to an agent?

This question is really a question about scientific methodology. Cognitive science posits an unobservable entity: representation (see Bechtel 2016 for a similar way of framing the problem, but see also Thomson and Piccinini 2018 who question this framing). Just as in physics and philosophy of physics, there are debates about when we are justified to posit an unobservable subatomic entity, there should also be a debate in the philosophy of cognitive science and the philosophy of psychology about when we are justified to posit an unobservable Footnote 1 mental entity: mental representation.

2 Realism About Theories Versus Realism About Entities

My aim is to argue that if we apply some of the standard apparatus of the scientific realism debate in the context of the philosophy of cognitive science and philosophy of psychology, we can make real progress on the old questions about whether and when we are entitled to posit mental representations. As we have seen, the entities that are the most important ingredients of explanations in psychology and cognitive science are unobservable entities, just like electrons or subatomic particles. The question then is: how do the considerations about realism concerning unobservable entities we are familiar with from the scientific realism literature apply to them?

As we know from the classic scientific realism literature, there are different realism versus antirealism debates (see Newton-Smith 1978 ; Hacking 1983 ; Chakravartty 2007 ; Nanay 2013b for taxonomies). One can be realist or antirealist about theories. Scientific realism about theories is the view that “scientific theories are either true or false independent of what we know: science at least aims at the truth and the truth is how the world is” (Hacking 1983 , p. 27). In other words, science aims to give us literally true claims about the world and there is a fact of the matter, independent of us, about how the world is. This definition has two conjuncts: one about what science aims to do and the other about the relation between scientific theories and the world. Scientific antirealists can deny either of these two conjuncts.

But this debate about theories is very different from, and logically independent of, the debate about theoretical entities. Realism about entities is the view that theoretical (unobservable) entities that scientific theories postulate really do exist. And, at least on the face of it, we can be realist about a theoretical entity a theory postulates without being realist about the theory that postulates it (see more on how this may work below).

On the rare occasions when philosophers or cognitive scientists defend representationalism, they assume that the view they need to defend is realism about representationalism as a theory. This can take the form of arguing that a representationalist theory of the mind is superior to a non-representationalist theory, in terms of explanatory scope, simplicity or empirical adequacy. And neo-behaviorists also assume this framework in their fight against the concept of representation: they aim to give a theory that covers all the available empirical facts and has the same explanatory scope, but that has an advantage over representationalist theories in terms of simplicity because it can do all that without positing representations (see esp. Chemero 2009 ; Hutto and Myin 2014 and see also Ramsey 2007 for a good summary).

In short, both representationalists and neo-behaviorists tend to take it for granted that the real question about representations is whether we should be realists about the theory of representationalism. This paper is an attempt to shift the emphasis from the debate concerning realism about theories to the one concerning realism about entities. My claim is that regardless of whether we are realist about representational theories of the mind, we have compelling reasons to endorse entity realism about mental representations. And this is more than enough justification for talking about representations when talking about the mind.

3 Entity Realism

Entity realism is the view that, to quote Ian Hacking, “a good many theoretical Footnote 2 entities do really exist” (Hacking 1983 p. 27; see also Cartwright 1983 , p. 89; see also Nanay 2019 ). Not a particularly controversial view these days. What is somewhat more controversial is what methodology entity realism should use. But the real controversy about entity realism comes from Hacking’s (and Cartwright’s) insistence that one can be realist about entities and antirealist about theories.

First, what I take to be the core commitment of entity realism is that the more evidence we have about the causal powers of x, the more reason we have to be realist about x. Taking causal powers as evidence for existence is hardly a very controversial move—the same connection does a lot of work in various metaphysical debates, for example, the one about the causal role of properties (e.g., Shoemaker 1979 , Crane 2009 ). Jaegwon Kim even came up with a catchy label for it: “This we might call ‘Alexander’s Dictum’: to be real is to have causal powers” (Kim 1993 , p. 202; see also Cargile 2003 ).

What is more controversial is how we can find out about the causal powers of an unobservable entity—the methodology of entity realism. And here we get some variations within the entity realist camp—as how one should proceed in this question clearly depends on one’s conception of causation in general. According to Nancy Cartwright, we have reason to endorse realism about entity x if x figures essentially in causal explanations of observable phenomena (Cartwright 1983 , 1999 ). Ian Hacking’s view is slightly different: if we can manipulate x in such a way that this has direct influence on observable phenomena, we have reason to endorse realism about entity x. As his famous one-liner goes, “so far as I’m concerned, if you can spray them, they are real” (Hacking 1983 , p. 23). I prefer another, less famous, but somewhat more informative, one-liner: “When we use entities as tools, as instruments of inquiry, we are entitled to regard them as real” (Hacking 1989 , p. 578). Or, even more informatively:

We are completely convinced of the reality of electrons when we regularly set out to build and often enough succeed in building new kinds of device that use various well-understood causal properties of electrons to interfere in the more hypothetical parts of nature (Hacking 1983 , p. 265).

Hacking and Cartwright give two different ways of cashing out the very same idea: namely, that discovering the causal properties of an entity is what justifies realism about this entity. But they offer very different methodologies for establishing these claims. I will focus on Hacking’s methodology mainly because Cartwright’s relies on a fairly specific stance on the relation between causal explanation and causation that is easier to criticize (see, e.g., Psillos 2008 ).

When we are assessing entity realism, we should use the following methodology then: if we can manipulate ‘well-understood causal properties’ of a certain kind of entity in a way that would have direct observable effects, we can be convinced of the reality of this kind of entity. This, of course, leaves open the question about how we can manipulate unobservable entities (see Giere 1988 , pp. 111–140), a question that will play an important role in Section IV.

These core commitments of entity realism have been criticized both for being too restrictive and for being too inclusive (see Shapere 1993 and Gelfert 2003 , respectively). But the real controversy comes from the proposal that entity realism does not presuppose realism about theories. This is the reason why entity realism was a major shift in the realism versus antirealism debate: it pushed the debate about the truth of theories (or their aiming at the truth) in the background. Again, here is Hacking on the relation between entity realism and the realism versus antirealism debate about theories:

One can believe in some entities without believing in any particular theory in which they are embedded. One can even hold that no general deep theory about the entities could possibly be true for there is no such truth (Hacking 1983 , p. 29).

In short, one can be realist about observable entities and be noncommittal about the realism versus antirealism debate about theories. Entity realism may even be consistent with antirealism about theories. As Erman McMullin sums up, according to entity realism, we may “know that the electron [exists], even though there is no similar assurance as to what it is” (McMullin 1984 , p. 63; see also Clarke 2001 ). Further, Hacking argues that this seems to fit the actual scientific practice very well—scientists do things with unobservable entities without (or before) having any firm theories about them. So an experimental scientist may or may not be realist about theories, but as long as she conducts experiments with the help of unobservable entities, she must be realist about entities.

This tenet of entity realism—that it is consistent with antirealism about theories—has been very controversial (Morrison 1990 ; Resnik 1994 ; Massimi 2004 ; Chakravartty 2007 ; Psillos 1999 , 2008 ; Hardin and Rosenberg 1982 ; Laudan 1984 ; Musgrave 1996 ), so I do not want to rely on in in this paper. But if it is true that entity realism does not imply realism about theories, then one can accept the main claim of this paper, that we should be entity realist about mental representations, while being noncommittal about whether realism about representationalist theories of mind is correct.

However, my main aim is to use the methodology of entity realism to find out more about the ontological commitments of psychology and cognitive science. Those who are unconvinced that entity realism is consistent with antirealism about theories should have no problem going along with the arguments presented in the rest of the paper.

4 Entity Realism About Representations

Mental representations are about some distal entity in the world. They attribute properties to a distal entity: they represent this entity as having certain properties. This can go wrong in two different ways. The represented entity may not exist. In this case, the mental representation misrepresents. Or the represented entity, although it exists, may not have the represented properties. This is also a case of misrepresentation. If the represented entity exists and has the represented properties, then the mental representation is correct.

Mental representations are not necessarily propositional attitudes like beliefs and desires. They are not necessarily syntactically structured. Further, mental representations are not necessarily conscious, and we do not necessarily have particularly reliable introspective access to them.

Representations attribute properties to distal entities in a way that can misrepresent. This allows for talking about representations in the visual system, even in the early visual system, like the primary visual cortex. But the photosensors of the retina will not count as representations—they carry information about the scene in front of our eyes, but they can’t misrepresent. The primary visual cortex can. The concept of representation covers representations of very different kinds, from the ones in the primary visual cortex to beliefs and desires. My claim is that we should be entity realist about at least some of these. And the reason for this is that we can manipulate these representations.

What would constitute a good reason for attributing a mental representation to a subject? Before I address this question, I want to first set aside a very widespread bad reason for attributing mental representations: introspective report. If a subject introspects and says that she has such and such mental representation, this is not a very good reason to attribute this mental representation to her. Why? Because introspective access is very unreliable (see Nisbett and Wilson 1977 for the locus classicus and Schwitzgebel 2008 ; Spener and Bayne 2010 ; Spener 2011 for philosophical summaries). This is a vast literature and I can’t do justice to all the wrinkles here. But a large number of studies from very different subdisciplines and using very different methodology have suggested that introspection is an unreliable guide to what goes on in our mind (see, e.g., Carruthers 2011 for a summary).

What would then constitute a good reason for attributing a mental representation to a subject? This is where entity realism comes in. Hacking’s criterion in this context would be that we should be realist about mental representations if we can manipulate mental representations in such a way that it would have direct observable influence—which, in the present context would mean that it would influence our observable behavior directly. I will argue that we have strong reason to endorse entity realism about mental representations if we follow this methodology. But before I do so, it is important to see that entity realism about mental representations is not an obviously correct view—it has its opponents. In fact, it has many opponents.

Behaviorists deny that there are mental representations. There is sensory input and there is motor output, but there is nothing representational between them (Watson 1930 ). It is important that at least some versions of behaviorism are consistent with the claim that a lot is going on between the sensory input and the motor output and we can learn about some of these processes. In other words, behaviorism is not committed to the caricature idea that the mind is a black box. But whatever mediates between the sensory input and the motor output is not something representational. Skinner, for example, explicitly allows for a neuroscientific description of such mediation:

The organism is, of course, not empty, and it cannot be adequately treated simply as a black box, but we must carefully distinguish between what is known about what is inside and what is merely inferred […] The physiologist of the future will tell us all that can be known about what is happening inside the behaving organism. His account will be an important advance over a behavior analysis, because the latter is necessarily “historical”–that is to say, it is confined to functional relations showing temporal gaps. Something is done today which affects the behavior of the organism tomorrow. No matter how clearly that fact can be established, a step is missing, and we must wait for the physiologist to supply it. He will be able to show how an organism is changed when exposed to contingencies of reinforcement and why the changed organism then behaves in a different way, possibly at a much later date. What he discovers cannot invalidate the laws of a science of behavior, but it will make the picture of human action more nearly complete (Skinner 1974 , pp. 233–237).

But the behaviorist stance towards neuroscience is ambivalent to say the least (see Catania and Harnad 1988 ). Skinner himself is not always as concessive as in the quote above. In the following quote he talks about the relation between sensory input (the ‘first link’), the neural processes (the ‘second link’) and the motor output (the ‘third link’):

Unless there is a weak spot in our causal chain so that the second [neural process] link is not lawfully determined by the first [sensory input], or the third [motor output] by the second, the first and third links must be lawfully related. […] Valid information about the second link may throw light on this relationship but can in no way alter it. (Skinner 1953 , p. 35)

The main point is that regardless of how much role behaviorists envisaged for the neural processes that mediate between sensory input and motor output, these processes are to be described in nonrepresentational terms. Behaviorism would, presumably, need to posit some neural processes (in order to account for various forms of learning, for example), but, it does not need to posit representations.

Behaviorism is not particularly popular today, but, as we have seen, neo-behaviorism is. They claim that sensory input and motor output are so closely intertwined in a dynamic process that we do not need to posit any representations that would mediate between them. The neo-behaviorist, like the old-fashioned behaviorist, would deny entity realism about mental representations.

My aim is to argue that entity realism, when applied to the question about whether mental representations exist gives a fairly straightforward positive answer. Footnote 3 We have seen that according to Ian Hacking’s criterion, if we can manipulate mental representations in a way that would have direct influence on behavior, we would have a strong case for entity realism about mental representations. The problem is that it is far from clear what it would mean to manipulate mental representations. More generally, it is far from clear how we can manipulate unobservable entities. What would be the equivalent of the spraying of electrons in the domain of mental representations?

The general proposal is that if changes in what a mental representation represents directly influence our behavior, this would constitute a reason to accept entity realism about this kind of mental representation. The problem is that it is not clear how we could be certain that we change what this mental representation represents (and not some nonrepresentational processes leading to the behavior).

To put it differently, in order to establish entity realism about mental representations, we need to argue for two claims: First, that changes in a certain kind of mental state directly influence our observable behavior. Second, that these changes in our observable behavior cannot be explained in terms of nonrepresentational processing of the sensory input. I will label these as:

Criterion A: Changes in mental state M directly influence our observable behavior.

Criterion B: The changes in our observable behavior cannot be straightforwardly explained in terms of nonrepresentational processing of the sensory input. Footnote 4

If both criteria are satisfied, we have good reasons for positing representation M.

I have said a lot about Criterion A in the last section. The importance of Criterion B can be made more explicit if we consider a version of Morgan’s Canon. Here is Morgan’s Canon in its original phrasing:

Morgan’s Canon : “In no case may we interpret an action as the outcome of the exercise of a higher psychical faculty if it can be interpreted as the outcome of the exercise of one which stands lower in the psychological scale.” (Morgan 1894/1903 , p. 53)

Morgan’s Canon has been proposed as the correct methodology for attributing mental states to organisms. There may be reasons to doubt the evolutionary arguments in favor of Morgan’s Canon and it is not clear how ‘higher’ and ‘lower’ psychical faculties are supposed to be distinguished (see Sober 1998 , 2005 , Karin-D’Arcy 2005 , Buckner 2013 ). But for the purposes of this paper, I want to accept a much weaker version of Morgan’s Canon as applied to representations (rephrasing Morgan’s original formulation):

Representational Morgan’s Canon : in no case may we interpret an action as the outcome of a representational process if it can be interpreted as the outcome of a nonrepresentational process.

A behaviorist (or neo-behaviorist) move against a realist stance towards mental representations would be to rely on the Representational Morgan’s Canon (which, for the purposes of this paper I take to be uncontroversial) and insist that whatever causes the differences in our behavior is not a representation but some nonrepresentational processing of the sensory input.

The analogy often used in this context is with the heliotropism of some plants (see, e.g., Dennett 1991 , p. 191). Some plants move their leaves or flowers in a way that tracks the location of the Sun. Examples include the snow buttercup ( Ranunculus adoneus ) and the bud (but not the mature plant) of sunflower ( Helianthus annuus ). If we follow the Representational Morgan’s Canon, we should resist the temptation of attributing the representation of the spatial location of the Sun to these plants because there is a well-understood mechanism that explains heliotropism without postulating any representations (Sherry and Galen 1998 ; Galen and Stanton 2003 ; Vanderbrink et al. 2014 ). Because of the Representational Morgan’s Canon, we should not be entity realist about the representation of the location of the Sun in plants (but see Morgan 2014 for some worries about this claim). My aim is to show that even if we accept the Representational Morgan’s Canon, Footnote 5 we have strong reasons to be realist about some mental representations.

The mental representation I want to focus on is one that is directly involved in the successful performance of actions. I will label it ‘motor representation’ here (I called it ‘pragmatic representation’ earlier, see Nanay 2013a ). Motor representations represent those parameters of the situations that are necessary for the successful performance of action. Just what these parameters are is debated: they may include the properties of the objects one acts upon, the properties of one’s own body, one’s bodily movement that is needed to complete the action or maybe the properties of the goal state the action is aimed at (see Jeannerod 1997 ; Nanay 2013a ; Poincaré 1905 /1958; Bach 1978 ; Brand 1984 ; Pacherie 2011 ; Millikan 2004 ; Butterfill and Sinigaglia 2014 for very different proposals about this).

For simplicity, I will assume that motor representations represent simple shape, size and spatial location of the distal objects the action is directed at. Arguably, simple representations of this kind are involved in all the accounts of motor representations I mentioned in the previous paragraph. These properties need to be represented in order for the agent to be able to perform the action at all. Suppose that the action is to pick up a cup. If I didn’t represent the size of the cup, I would have no idea what grip size I should approach it with. If I didn’t represent its spatial location, I would have no idea which direction I should reach out towards. And so on. Motor representations are genuine representations: they can misrepresent. If I represent the shape property of the cup correctly, then I will be more likely to approach it with the appropriate grip size, which makes it more likely that my action will be successful. And if I represent the spatial location of the cup correctly, I will be more likely to reach out in the right direction, which, again, makes it more likely that my action succeeds.

These motor representations do not need to be (and arguably they normally are not) conscious. But then how do we know what property (say, shape property or spatial location property) they attribute to the cup? Clearly not by introspecting. We can infer what shape property this motor representation attributes to the cup from the grip size the agent approaches the cup with. And we can infer what spatial location property it attributes to the cup from the direction of my reaching. In other words, if the shape property the motor representation attributes to the cup changes, this affects my behavior, that is, the grip size of my approaching hand, directly. And if the spatial location property the motor representation attributes changes, this also affects my behavior—the direction I reach out towards (see Jeannerod 1997 for a number of case studies of how intervention on the motor representation leads to observable changes in our behavior). In one famous experiment, in the middle to the performance of the reaching movement the target was changed—either its spatial location or its size. And this influenced the action execution—the reaching movement changed direction in the course of the execution of this action. The subjects are almost always unaware that anything has changed (Paulignan et al. 1991 ; Pelisson et al. 1986 ; Goodale et al. 1986 ).

Thus, Criterion A for entity realism about mental representations is satisfied: we can manipulate the motor representations to bring about direct changes in our observable behavior. But I said nothing about the Criterion B: that it is really the representation that got manipulated. I need to also show that these changes in behavior could not be explained in a purely nonrepresentational way (that is, in terms of the nonrepresentational processing of the sensory input).

And here we need to turn to the cognitive neuroscience of action. There are lots of empirical reasons to think that the grip size of grasping movements is determined by a representation (that is, motor representation) and not some nonrepresentational processing of the sensory input. What I take to be the most impressive piece of evidence is the following (this is by no means an isolated example, see Jeannerod 1997 ; Nanay 2013a for a more systematic treatment).

Two very widely used brands of matches in the UK are “Swan Vestas” and “Scottish Bluebell.” The box of Swan Vestas is 25 percent larger than that of Scottish Bluebell. It was tested whether the brand of the matchboxes influences our grip size when grasping them, and it was found that it does (McIntosh and Lashley 2008 ). When the subjects were grasping the 1.25-scale replica of the Scottish Bluebell box, their grip size was smaller than it was when grasping the normal Swan Vestas of the same size. And when they were grasping the 0.8-scale replica of the Swan Vestas box, their grip size was larger than it was when grasping the normal Scottish Bluebell box. Hence, the recognition of the brand of the matchboxes influences the grip size we approach it with. In a follow-up study, it was also pointed out that this influence is not due to local or short-term learning, but to long-term familiarity with these matchbox brands (Borchers et al. 2011 ; Borchers and Himmelbach 2012 ; Roche et al. 2015 ).

This is going to be difficult to explain in nonrepresentationalist terms. What happens in motor control in this case is an integration of the sensory input with high-level information (about the matchbox’s brand or maybe about some reliably co-occurring features of these matchboxes). This integration is crucial for yielding the grip-size we approach these objects with. And this integration is not the mere nonrepresentational processing of the sensory input—it is the combination of the sensory input with some fairly complex stored information—information about previously experienced matchboxes (Borchers et al. 2011 ; Borchers and Himmelbach 2012 ; Roche et al. 2015 ). Footnote 6

Some hard-core neo-behaviorists may push back at this point and argue that as long as we allow for a non-representational way of explaining learning (including perceptual learning), we may be able to give a non-representational analysis of these match-box experiments. They could argue that while there is information in the system (as a result of past exposure to different kinds of match-boxes, our movement, and our grip size can be explained fully in terms of the processing of the input (of certain color combinations on the box). I am not entirely sure how such an explanation would work, but as I don’t want to exclude the possibility that it might, I want to modify the argument in a way that would be even more difficult for the neo-behaviorist to counter.

We have seen that manipulating motor representations leads to observable effects: if my motor representation attributed a different size property to the object, I would approach it with a different grip size. In other words, motor representations satisfy Criterion A for positing mental representations—the potential problems were with Criterion B. But there is another kind of mental state that satisfies Criterion A: a mental state I call, following Nanay ( 2013a ), ‘pragmatic mental imagery’. I hope to show that pragmatic mental imagery fares (even) better than motor representations when it comes to Criterion B.

Suppose that there is a cup in front of me. I can pick it up while looking at it: in this case, the visual feedback helps me to do so. I can adjust my movements in the light of my visual experience of how my action succeeds: if my initial reach was too forceful, I can adjust its course in response to the visual feedback (some of this happens unconsciously, see Paulignan et al. 1991 ; Pelisson et al. 1986 ; Goodale et al. 1986 ; see also Brogaard 2011 ).

But I can also perform this action, and do so fairly successfully, without looking. I am looking at the cup, close my eyes, count to ten and then reach out to grab it. In this case, it is my mental imagery that guides my action. It is a special kind of mental imagery inasmuch as it attributes very similar properties as motor representations do: egocentric spatial location properties (that allow me to reach out in the appropriate direction), egocentric size properties (that allow me to approach the cup with the appropriate grip size) and so on. And it is also, like motor representation, a genuine representation as it can misrepresent.

Manipulating pragmatic mental imagery leads to observable behavioral changes in the same way as manipulating motor representations leads to observable behavioral changes: if my pragmatic mental imagery attributed a different size property to the cup, I would approach it with a different grip size. So Criterion A is satisfied. How about Criterion B, which proved to be more problematic in the case of motor representations?

Consider the following slightly modified version of the previous example: there is a cup in front of you and you close your eyes, count to ten and try to pick it up, but before you do so, your friend tells you that she moved the cup to the left by 10 cm. You can still perform this action fairly successfully.

If we attribute representations to the subject, we have no problem explaining how this happens: your mental imagery attributes a spatial location property to where the cup was originally and the information you received from your friend modifies this spatial location property leading to the pragmatic mental imagery that represents the correct spatial location property. And this pragmatic mental imagery can guide you to perform the action successfully.

But what could the non-representationalist explanation be? The direction of my reach is directly influenced by a mental state that is the combination of two stimulus-independent informational states: my mental imagery (of where the cup used to be) and the verbally coded information about how my friend moved it. This is the combination of two information-carrying states. Even if neo-behaviorists can somehow explain how an information-carrying state influences behavior (and even if this information is influenced by previous exposure as in the case of motor representations), I see no way in which they could explain how the information from two information-carrying states would be combined without positing representations, especially given that one of these two information-carrying states is linguistically coded, which would be considered to be representational even by the most ardent proponents of neo-behaviorists (see, e.g., Hutto and Myin 2014 ).

It may help to contrast this case with the simpler example of just looking at the cup, closing my eyes, counting to ten and then picking it up with my eyes closed. In this case, the neo-behaviorist could say that I am acting on the sensory input but with a bit of delay—no representation needs to be posited.

But the case I gave above requires a much more complex mental process: the mental imagery of the cup I have when I close my eyes carries information about the cup that is coded in an egocentric frame of reference. The’10 cm to the left’ information, in contrast, is not coded in an egocentric frame of reference. In order to revise my mental imagery and reach out in the appropriate direction, I need to put together these two very differently coded pieces of information.

Integrating two pieces of information that are coded in comparable reference frames (which is what happens in multimodal sensory integration, for example) could be argued to be explicable in non-representationalist terms (but see Nanay 2014 for an argument about how multisensory integration may give us reasons for positing perceptual representations). But I do not see how one can explain the integration between two pieces of information that involves translating linguistically coded information about (allocentric) distances into egocentric information one can act on. Footnote 7

One may worry that the talk about mental imagery in this argument is ontologically suspicious. Didn’t Ryle—himself very much in the behaviorist camp—refute any appeal to mental imagery, after all? In response, it is important to emphasize that the concept of mental imagery at play here is very different from the ‘little pictures in the head’ conception Ryle was making fun of. Mental imagery, in the meantime, has become a scientifically respectable concept that neuroscience has a lot to say about. And the way neuroscientists and psychologists talk about mental imagery has nothing to do with little pictures in the head. Neuroscience considers mental imagery to be perceptual processing that is not triggered by corresponding sensory stimulation in the relevant sense modality (Kosslyn et al. 1995 ; Pearson et al. 2015 ; Nanay 2015 , 2018 ). Mental imagery in this sense is as scientifically and ontologically unproblematic as perception. And, a fortiori, so is pragmatic mental imagery. Footnote 8

Let’s go back to the explanations of the movement of heliotropic plants. This movement can be fully explained in terms of the nonrepresentational processing of the sensory input (Sherry and Galen 1998 ; Galen and Stanton 2003 ; Vanderbrink et al. 2014 ). The only information that is relied on when determining the movement of the plant is carried by the sensory input. So we need to conclude, via the Representational Morgan’s Canon, that there is no need to postulate any representations here. Criterion B is not satisfied.

But in the case of pragmatic mental imagery, the fine-grained movements of action execution are determined by two informational states, which code spatial location information very differently: the mental imagery of where the cup used to be and verbal information from my friend. Given that we need to explain how these two very differently coded pieces of information are combined and used to guide our movement, this rules out any explanation by means of purely nonrepresentational processing of the sensory input. So the Representational Morgan’s Canon does not apply in this case. Criterion B is satisfied.

We can now put together the argument for entity realism about motor representations: changes in the properties the pragmatic mental imagery attributes directly influence our motor behavior (Criterion A). And these changes in motor behavior cannot be explained in terms of nonrepresentational processing of sensory input (Criterion B). In other words, we have good reasons to be entity realist about at least some kinds of representations.

Note that these considerations give us reason to endorse entity realism about mental representations. If it is true that entity realism in general does not entail realism about theories, then they do not give us any reason to endorse representational theories of the mind. As Hacking pointed out, scientists do things with unobservable entities without (or before) having any firm theories about them and this is definitely true of neuroscientists (see Thomson and Piccinini for a related argument). More generally, entity realism about representations only requires that we can causally manipulate representations. This requires attributing some causal power to these representations, but all the other properties of these representations could be left unspecified. And this picture fits the argument above: the argument left it unspecified just how and what motor representations represent (and there are many radically different theories about this, see Butterfill and Sinigaglia 2014 ; Jeannerod 1997 ; Nanay 2013a ; Poincaré 1905 /1958; Bach 1978 ; Brand 1984 ; Pacherie 2011 ; Millikan 2004 ). These theories differ in their account of what motor representations represent (the object’s actual properties vs. a goal state vs. the agent’s potential action) and they also differ in how they connect up with the rest of our mind. But they are all entity realist about motor representations.

Stephen Stich said in 1984 that:

we now have an enormous collection of experimental data which, it would seem, simply cannot be made sense of unless we postulate something like [representations] (Stich 1984 , p. 649).

I am not sure that this was in fact true in 1984—to the extent that would have satisfied an ardent proponent of the Representational Morgan’s Canon. But with the advances of the cognitive neuroscience of action, Stich’s claim is definitely true now.

5 Conclusion: What Kinds of Representation?

The scope of the argument I presented in the last section was very limited: it was about motor representations and pragmatic mental imagery. This argument gives us strong reasons to endorse entity realism about motor representations and even stronger reasons to endorse entity realism about pragmatic mental imagery. Can we generalize this argument to other representations—maybe ones more familiar to us from our folk psychology, like beliefs and desires?

In this brief conclusion, I want to warn against any such generalization. The argument in the last section was based on very specific empirical findings that we could only explain if we posit motor representations or pragmatic mental imagery. It provides no justification for drawing any conclusions about any other kinds of representations (although it could be thought of as providing an example of what kind of evidence someone who wants to argue for entity realism about other kinds of representations should be looking for).

In other words, the conclusion of this paper would be consistent with an all-out entity realist stance about various forms of representations (including beliefs and desires). But it would also be consistent with the classic 1980s eliminativist stance against beliefs and desires (Stich 1983 ; Churchland 1981 , 1986 , 1988 ).

There is a debate in philosophy of perception whether we can represent some of the mental states of other people perceptually. If we can, then in some sense these mental states are observable. Note, however, that this is not the sense of ‘observable’ that is at stake in the scientific realism debate.

I will use the less controversial term ‘unobservable entities’ instead of Hacking’s original ‘theoretical entities’ in what follows.

Somewhat surprisingly, some advocates of neo-behaviorism do appeal to entity realism when they defend the reality of some entities in their nonrepresentational apparatus (see Chemero 2009 , chapter 9, esp. pp. 192–194, where he defends entity realism about ‘affordances’). I find this surprising, because it seems to me that there are vastly stronger empirical reasons to be entity realist about representations (which neo-behaviorists would want to avoid) than about affordances (which some neo-behaviorists endorse).

It is worth noting that Criterion B could be replaced by a much weaker one and we could still get a strong reason for positing representations: that the representationalist explanation is more powerful/simpler/in some sense better than the non-representationalist one. As none of these ways of comparing explanations is unproblematic, if we manage to find evidence for Criterion B, this would be a much stronger reason for positing representations.

Note that not accepting the Representational Morgan’s Canon would make the job of arguing for entity realism about mental representations much easier.

It should also be noted that the matchbox study is not an isolated example. There are many results that show that motor representations are sensitive to various top-down factors: the subject’s attention (Marrett et al. 2011 ), her language skills and lexical recognition (Deng et al. 2012 ; Pulvermuller and Hauk 2005), and her expectations or knowledge (Roche et al. 2015 ).

It should be emphasized that the kind of representations posited for the reasons outlined above would be consistent with the kind of representations that Ramsey 2007 would allow for, so my view is not inconsistent with his.

Thanks for an anonymous referee for pushing me to clarify what I mean by mental imagery.

Bach, K. (1978). A representational theory of action. Philosophical Studies, 34, 361–379.

Google Scholar  

Ballard, D. H. (1996). On the function of visual representation. In K. Akins (Ed.), Perception (pp. 111–131). New York: Oxford University Press.

Bechtel, W. (2016). Investigating neural representations: The tale of place cells. Synthese, 193, 1287–1321.

Borchers, S., & Himmelbach, M. (2012). The recognition of everyday objects changes grasp scaling. Vision Research, 67, 8–13.

Borchers, S. A., Christensen, L. Ziegler, & Himmelbach, M. (2011). Visual action control does not rely on strangers—Effects of pictorial cues under monocular and binocular vision. Neuropsychologia, 49, 556–563.

Brand, M. (1984). Intending and acting . Cambridge: The MIT Press.

Brewer, B. (2011). Perception and its objects . Oxford: Oxford University Press.

Brogaard, B. (2011). Are there unconscious perceptual processes? Consciousness and Cognition, 20, 449–463.

Brooks, R. A. (1991). Intelligence without representation. Artificial Intelligence, 47, 139–159.

Buckner, C. (2013). Morgan’s Canon, meet Hume’s Dictum: Avoiding Anthropofabulation in cross-species comparisons. Biology and Philosophy, 28, 853–871.

Butterfill, S., & Sinigaglia, C. (2014). Intention and motor representation in purposive action. Philosophy and Phenomenological Research, 88, 119–145.

Campbell, J. (2002). Reference and consciousness . Oxford: Oxford University Press.

Cargile, J. (2003). On ‘Alexander’s’ dictum. Topoi, 22, 143–149.

Carruthers, P. (2011). The opacity of mind: An integrative theory of self-knowledge . Oxford: Oxford University Press.

Cartwright, N. (1983). How the laws of physics lie . New York: Oxford University Press.

Cartwright, N. (1999). The dappled world: A study of the boundaries of science . Cambridge: Cambridge University Press.

Catania, C., & Harnad, S. 1988 (eds). The selection of behavior: The operant behaviorism of B. F. skinner: Comments and consequences . Cambridge: Cambridge University Press.

Chakravartty, A. (2007). A metaphysics for scientific realism: Knowing the unobservable . Cambridge: Cambridge University Press.

Chemero, A. (2009). Radical embodied cognitive science . Cambridge: MIT Press.

Churchland, P. M. (1981). Eliminative materialism and the propositional attitudes. Journal of Philosophy, 78, 67–90.

Churchland, P. (1986). Neurophilosophy . Cambridge: MIT Press.

Churchland, P. M. (1988). Folk psychology and the explanation of behaviour. Proceedings of the Aristotelian Society, 62, 209–221.

Clarke, S. (2001). Defensible territory for entity realism. British Journal for the Philosophy of Science, 52, 701–722.

Crane, T. (2009). Causation and determinable properties: on the efficacy of colour, shape and size. In J. Kallestrup & J. Howhy (Eds.), Being reduced . Oxford: Oxford University Press.

Deng, Y., Guo, R., Ding, G., & Peng, D. (2012). Top-down modulations from dorsal stream in lexical recognition: And effective connectivity fMRI study. PLoS ONE, 7, e33337.

Dennett, D. C. (1991). Consciousness explained . Boston: Little, Brown and Co.

Galen, C., & Stanton, M. L. (2003). Sunny-side up: flower heliotropism as a source of parental environmental effects of pollen quality and performance in the snow buttercup, Ranunculus adoneus . American Journal of Botany, 90, 724–729.

Gelfert, A. (2003). Manipulative success and the unreasl. International Studies in the Philosophy of Science, 17, 245–263.

Giere, R. N. (1988). Explaining science: A cognitive approach . Chicago: University of Chicago Press.

Goodale, M. A., Pelisson, D., & Prablanc, C. (1986). Large adjustments in visually guided reaching do not depend on vision of the hand or perception of target displacement. Nature, 320, 748–750.

Hacking, I. (1983). Representing and intervening . Cambridge: Cambridge University Press.

Hacking, I. (1989). Extragalactic reality: the case of gravitational lensing. Philosophy of Science, 56, 555–581.

Hardin, C. L., & Rosenberg, A. (1982). In defence of convergent realism. Philosophy of Science, 49, 604–615.

Hurley, S. L. (2001). Perception and action: Alternative views. Synthese , 129 , 3–40.

Hutto, D. D., & Myin, E. (2014). Radicalizing enactivism . Cambridge, MA: MIT Press.

Jeannerod, M. (1997). The cognitive neuroscience of action . Oxford: Blackwell.

Karin-D’Arcy, M. R. (2005). The modern role of Morgan’s canon in comparative psychology. International Journal of Comparative Psychology, 18, 179–201.

Kim, J. (1993). Supervenience and mind . Cambridge: Cambridge University Press.

Kosslyn, S. M., Behrmann, M., & Jeannerod, M. (1995). The cognitive neuroscience of mental imagery. Neuropsychologia, 33, 1335–1344.

Laudan, L. (1984). Discussion: Realism without the real. Philosophy of Science, 51, 156–162.

Marrett, N. E., de-Wit, L. H., Roser, M., Kentridge, R. W., Milner, A. D., & Lambert, A. J. (2011). Testing the dorsal stream attention hypothesis: electrophysiological correlates and the effects of ventral stream damage. Visual cognition, 19, 1089–1121.

Martin, M. G. F. (2004). The limits of self-awareness. Philosophical Studies , 120 , 37–89.

Massimi, M. (2004). Non-defensible middle ground for experimental realism: Why we are justified to believe in colored quarks. Philosophy of Science, 71, 36–60.

McIntosh, R. D., & Lashley, G. (2008). Matching boxes: Familiar size influences action programming. Neuropsychologia, 46, 2441–2444.

McMullin, E. (1984). A case for scientific realism. In J. Leplin (Ed.), scientific realism . Berkeley: University of California Press.

Millikan, R. G. (2004). Varieties of meaning . Cambridge: The MIT Press.

Morgan C. L. (1894/1903). An introduction to comparative psychology (2 edn). London: Walter Scott.

Morgan, A. (2014). Representations gone mental. Synthese, 191, 213–244.

Morrison, M. (1990). Theory, intervention and realism. Synthese, 82, 1–22.

Musgrave, A. (1996). ‘Realism, truth, and objectivity’. In R. S. Cohen, R. Hilpinen, & Q. Renzong (Eds.), Realism and anti-realism in the philosophy of science (pp. 19–44). Dordrecht: Kluwer.

Nanay, B. (2013a). Between perception and action . Oxford: Oxford University Press.

Nanay, B. (2013b). Singularist semirealism. British Journal for the Philosophy of Science, 64, 371–394.

Nanay, B. (2014). Empirical problems with anti-representationalism. In B. Brogaard (Ed.), Does perception have content? (pp. 39–50). New York: Oxford University Press.

Nanay, B. (2015). Perceptual content and the content of mental imagery. Philosophical Studies, 172, 1723–1736.

Nanay, B. (2018). Multimodal mental imagery. Cortex, 105, 125–134.

Nanay, B. (2019). Entity realism and singularist semirealism. Synthese, 196, 499–517.

Newton-Smith, W. (1978). The underdetermination of theory by data. Proceedings of the Aristotelian Society, supplementary, 52, 71–91.

Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review , 84 , 231–259.

Noë, A. (2004). Action in perception . Cambridge, MA: The MIT Press.

Pacherie, E. (2011). Nonconceptual representations for action and the limits of intentional control. Social Psychology, 42, 67–73.

Paulignan, Y., MacKenzie, C. L., Marteniuk, R. G., & Jeannerod, M. (1991). Selective perturbation of visual input during prehension movements: 1. The effect of changing object position. Experimental Brain Research, 83, 502–512.

Pearson, J., Naselaris, T., Holmes, E. A., & Kosslyn, S. M. (2015). Mental imagery: Functional mechanisms and clinical applications. Trends in Cognitive Sciences, 19, 590–602.

Pelisson, D., Prablanc, C., Goodale, M. A., & Jeannerod, M. (1986). Visual control of reaching movements without vision of the limb: II. Evidence of fast unconscious processes correcting the trajectory of the hand to the final position of a double-step stimulus. Experimental Brain Research, 62, 303–311.

Poincaré, H. (1905). The value of science . New York: Dover.

Psillos, S. (1999). Scientific realism: How science tracks truth . London: Routledge.

Psillos, S. (2008). Cartwright’s realist toil. In S. Hartmann, C. Hoefer, & L. Bovens (Eds.), Nancy cartwright’s philosophy of science (pp. 167–194). London: Routledge.

Ramsey, W. M. (2007). Representation reconsidered . Cambridge: Cambridge University Press.

Resnik, D. (1994). Hacking’s experimental realism. Canadian Journal of Philosophy, 24, 395–412.

Roche, K., Verheij, R., Voudouris, D., Chainay, H., & Smeets, J. B. J. (2015) Grasping an object comfortably: Orientation information is held in memory. Experimental Brain Research , in print.

Schwitzgebel, E. (2008). The Unreliability of Naive Introspection Philosophical Review, 117, 245–273.

Shapere, D. (1993). ‘Astronomy and Anti-Realism’. Philosophy of Science, 60, 134–150.

Sherry, R. A., & Galen, C. (1998). The mechanism of floral heliotropism in the snow buttercup, Ranunculus adoneus . Plant, Cell and Environment, 21, 983–993.

Shoemaker, S. (1979). ‘Causality and Properties’, in Identity,Cause, and Mind Cambridge: Cambridge University Press: 206–233.

Skinner, B. F. (1953). Science and human behavior . New York: Macmillan.

Skinner, B. F. (1974). About behaviorism . New York: Vintage.

Sober, E. (1998). Morgan’s canon. In C. Allen & D. Cummins (Eds.), The evolution of mind (pp. 224–242). Oxford: Oxford University Press.

Sober, E. (2005). Comparative Psychology Meets Evolutionary Biology: Morgan’s Canon and Cladistic Parsimony. In L. Daston & G. Mitman (Eds.), Thinking with animals: New perspectives on anthropomorphism (pp. 85–99). New York: Columbia University Press.

Spener, M. (2011). Using first person data about consciousness. Journal of Consciousness Studies , 18 , 165–179.

Spener, M., & Bayne, T. (2010). Introspective humility. philosophical. Issues, 20, 1–22.

Stich, S. (1983). From folk psychology to cognitive science . Cambridge MA: MIT Press.

Stich, S. (1984). Is Behaviorism Vacuous? Behavioral and Brain Sciences, 7, 647–649.

Thomson, E., & Piccinini, G. (2018). Neural representation observed. Minds and Machines, 28 (1), 191–235.

Travis, C. (2004). The silence of the senses. Mind , 113 , 57–94.

Vanderbrink, J. P., Brown, E. A., Harmer, S. L., & Blackman, B. K. (2014). Turning heads: The biology of solar tracking in sunflower. Plant Science, 224, 20–26.

Watson, J. B. (1930). Behaviorism . New York: W.W. Norton & Company Inc.

Download references

Acknowledgements

This work was supported by the ERC consolidator Grant [726251], the FWO Odysseus Grant [G.0020.12N] and the FWO research Grant [G0C7416N]. Special thanks for comments by Patrick Butlin, Anna Ichino, Alex Geddes, Lu Teng, Manolo Martinez and three anonymous referees.

Author information

Authors and affiliations.

Centre for Philosophical Psychology, University of Antwerp, D 413, 15 Prisstraat, 2000, Antwerp, Belgium

Bence Nanay

Peterhouse, University of Cambridge, Cambridge, CB2 1RD, UK

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Bence Nanay .

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Nanay, B. Entity Realism About Mental Representations. Erkenn 87 , 75–91 (2022). https://doi.org/10.1007/s10670-019-00185-4

Download citation

Received : 27 May 2019

Accepted : 03 October 2019

Published : 26 October 2019

Issue Date : February 2022

DOI : https://doi.org/10.1007/s10670-019-00185-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research

ORIGINAL RESEARCH article

Differences between spatial and visual mental representations.

• SFB/TR 8 Spatial Cognition, Universität Bremen, Bremen, Germany

This article investigates the relationship between visual mental representations and spatial mental representations in human visuo-spatial processing. By comparing two common theories of visuo-spatial processing – mental model theory and the theory of mental imagery – we identified two open questions: (1) which representations are modality-specific, and (2) what is the role of the two representations in reasoning. Two experiments examining eye movements and preferences for under-specified problems were conducted to investigate these questions. We found that significant spontaneous eye movements along the processed spatial relations occurred only when a visual mental representation is employed, but not with a spatial mental representation. Furthermore, the preferences for the answers of the under-specified problems differed between the two mental representations. The results challenge assumptions made by mental model theory and the theory of mental imagery.

1. Introduction

Our everyday behavior relies on our ability to process visual and spatial information. Describing the route to work, taking another person’s perspective, or imagining a familiar face or object all depend on our capability to process and reason with visual and spatial information.

Two main theoretic frameworks of visual and spatial knowledge processing have been proposed in cognitive science: mental model theory ( Johnson-Laird, 1989 , 1998 ; Tversky, 1993 ) and mental imagery ( Finke, 1989 ; Kosslyn, 1994 ; Kosslyn et al., 2006 ). Furthermore, there is also the conception of verbal or propositional mental representations ( Rips, 1994 ; Pylyshyn, 2002 ) that employ a sort of logical inference to reason about visual and/or spatial information. However, considerable evidence indicates that analogical mental representations, i.e., mental models or mental images, can better predict and explain the empirical data, in particular, for spatial reasoning (e.g., Byrne and Johnson-Laird, 1989 ; Kosslyn, 1994 ; Johnson-Laird, 2001 ).

In line with behavioral and neuroscientific evidence (e.g., Ungerleider and Mishkin, 1982 ; Levine et al., 1985 ; Newcombe et al., 1987 ; Farah et al., 1988 ; Courtney et al., 1996 ; Smith and Jonides, 1997 ; Mellet et al., 2000 ; Knauff and Johnson-Laird, 2002 ; Klauer and Zhao, 2004 ), mental model theory and the theory of mental imagery both propose a distinction between spatial and visual mental representations. The theory of mental imagery proposes spatial mental images and visual mental images; mental model theory proposes (spatial) mental models and visual mental images. Research based on the theories has, however, mostly focused on one of the two representations: the investigation of the properties of visual mental images in the theory of mental imagery and the investigation of reasoning with (spatial) mental models in mental model theory. Consequently, the relationship and interaction between the two types of mental representations is left largely unspecified in both theories. Although initial attempts exist (e.g., Schultheis and Barkowsky, 2011 ) to explain how visual and spatial mental representations interact and relate to each other, empirical data on the issue is largely missing. Accordingly, the primary aim of this article is to examine the differences and the relationship between visual and spatial mental representations. To achieve this, we first review how mental model theory on the one hand and the theory of mental imagery on the other hand understand spatial and visual mental representations as well as how they interpret the relationship between them. Even though there is much theoretic and empirical work on both theories, the literature lacks a systematic comparison of the theories. In the following, we present such a comparison. From this comparison, it will become clear that the theories actually propose very similar conceptions of spatial and visual mental representations but that their foci of investigation are mostly on different aspects and include phenomena not investigated within the respectively other theory. We examined these different aspects and used them in our experiments to gain new insights into the open issues of the relationship between visual and spatial mental representations. The results can be applied to complement gaps in the two theories.

2. Theories

2.1. mental model theory.

Mental model theory ( Johnson-Laird, 1998 ) postulates that there are three representational levels involved in human thinking: propositional representations, mental models, and mental images. The relationships between these three levels are hierarchical in the sense that their construction depends on each other. The following example helps to illustrate this point. Three-term series problems ( Johnson-Laird, 1972 ) are common experimental tasks in the study of mental models. They contain two premises and one conclusion that has to be validated or inferred based on the premises. Let the two premises be “A is left of B” and “B is right of C” and let the to-be-drawn conclusion be the relationship between A and C. According to mental model theory the premises are first encoded propositionally. From these propositional premises a mental model of the described configuration is constructed. As it is an essential property of mental models that “the structural relations between the parts of the model are analogous to the structural relations in the world” ( Johnson-Laird, 1998 , p. 447), one valid mental model of our example can be depicted in the following way:

We note that a mental model is a special case of the situation defined by the premises, because it only represents one valid situation with respect to the premises. For our example another mental model that satisfies the premises is:

Just like the situation represented by a mental model is a special case of what is described in the premises, mental model theory poses that a mental image is a special case of a given mental model. The mental image that is constructed from a mental model is one specific instance out of many valid instances described by the model, because the image has to specify, for example, the distance between the entities. The underlying mental model is in contrast invariant with respect to the distances. Summarizing the hierarchical structure of mental model theory, we note that a mental image is one out of many projections of the visualizable aspects of a mental model, and a mental model is one out of many analogically structured configurations that are valid given the propositionally represented premises. This suggests a clear hierarchy in which it is necessary to have the more general representations in order to construct a more specific one.

Mental models are described to be analogically structured, amodal, and abstract, e.g., they can represent abstract, non-visualizable relations such as “smarter than.” In contrast, mental images can only represent “visualizable” information and are modality-specific to visual perception (e.g., Johnson-Laird, 1998 ; Knauff and Johnson-Laird, 2002 ). It has been suggested that the analogical nature of mental models might be generally spatial ( Knauff et al., 2003 ), i.e., even reasoning with abstract relations like “worse than” or “better than” is handled by a spatio-analogical mental model. This view is supported by the association of mental model reasoning with activation in the parietal lobe (e.g., Goel and Dolan, 2001 ; Knauff et al., 2003 ), which is associated with several processes of spatial cognition (for an overview, see Sack, 2009 ). It was found that the use of “visual” relations, e.g., “dirtier than,” in relational reasoning tasks led to activation in the early visual cortex in contrast to tasks with other (abstract) relations, e.g., “worse than” ( Knauff et al., 2003 ). The study also found that “visual” relations led to longer reaction times and it was concluded that tasks using such “visual” relations induce the employment of visual mental images during the mental-model-based reasoning process.

Most of the literature on mental model theory focuses on how mental models explain reasoning. Johnson-Laird and Byrne (1991) state that reasoning according to the mental model theory consists of three stages: (1) the construction of one mental model (construction phase), (2) the inspection of the mental model (inspection phase), and (3) the variation of the mental model (variation phase). Slightly simplified, the reasoning process works as follows. One first mental model is constructed based on the given premises. This model represents one situation that is valid given the premises. This situation is inspected and can yield a possible conclusion. This conclusion is then verified to be valid in all other possible mental models that can be derived from the premises. If a conclusion is not contradicted in the other valid mental models, the conclusion is confirmed. There is much empirical support for this three stage process in human reasoning (e.g., Johnson-Laird, 2001 ). One interesting phenomenon in reasoning with mental models is the occurrence of preferred mental models when there are multiple valid conclusions. An example for such multiple valid conclusions are the two configurations “CAB” and “ACB” of the above example. It can be observed that there are reliable within-subject and between-subject preferences for which model is constructed first out of several valid mental models. This firstly constructed mental model is termed a preferred mental model. As a consequence, if there are several valid conclusions that can be inferred, there is a preference for one conclusion which corresponds to the preferred mental model. Preferred mental models have been investigated in different domains, but in particular in the domain of spatial reasoning (e.g., Rauh et al., 2005 ; Jahn et al., 2007 ; Schultheis and Barkowsky, 2013 ).

2.2. Theory of Mental Imagery

The theory of mental imagery ( Kosslyn, 1994 ; Kosslyn et al., 2006 ) makes a distinction between spatial mental images and visual mental images. These two mental representations differ in the content they represent and are distinct in their anatomical localization. But they are both assumed to have a (at least partially) spatio-analogical structure. Furthermore, there is also a propositional representation referred to as associative memory, which contains propositional descriptions of the structure of an object or a scene. This information can be used to construct spatial and visual mental images. For the latter, however, one needs to further retrieve encoded shape information from another source, i.e., the object-properties-processing subsystem, which can be thought of as a sort of non-analogical visual memory store located in the temporal lobe.

Spatial mental images (sometimes referred to as object maps) are located in the spatial-properties-processing subsystem in the framework of Kosslyn (1994) . They contain information about the location, size, and orientation of entities. The spatial-properties-processing subsystem is (at least partially) placed in the parietal lobe. Given that areas of the parietal lobe are topographically organized ( Sereno et al., 2001 ), it is assumed that spatial mental images are also at least partially spatio-analogical ( Kosslyn et al., 2006 ).

Visual mental images are constructed and processed in a structure called the visual buffer. The visual buffer consists of the topographically organized areas of the visual cortex. Visual mental images are thus assumed to be spatio-analogical or “depictive,” i.e., the metrics of what is represented, e.g., a shape, are reflected in the metrics of the representation. Visual mental images represent shape information, as well as, for example, color and depth.

A difference between spatial and visual mental images is that spatial mental images contain more information, in the sense that the current visual mental image in the visual buffer only contains a “visualized” part of what is represented in the spatial mental image ( Kosslyn et al., 2006 , p. 138). A visual mental image is a specification of a part of a spatial mental image.

Four types of functions are proposed for visual and spatial mental images: generation, inspection, maintenance, and manipulation. The generation of a mental image can either be just the retrieval of a spatial configuration of entities as a spatial mental image if no visual information is necessary for a given task or it can furthermore include the retrieval of shape information to generate a visual mental image in the visual buffer. Note that the visual buffer does not need to be employed for spatial mental images. Kosslyn et al. (2006) states that the processing of spatial and visual mental images occurs in parallel, i.e., the image of a shape is generated while a spatial image is generated. They furthermore state that this parallel processing might not always be useful, as the proper construction of a shape requires information about its spatial properties, i.e., location, size, and orientation which are provided by a respective spatial mental image ( Kosslyn et al., 2006 , p. 143). For the generation of multi-part visual mental images, a corresponding spatial mental image is necessary to guide the placement of shapes in the visual buffer by specifying the location, orientation, and size. The inspection process can make previously implicit information in a visual or spatial image explicit, i.e., new information is inferred. Visual mental images are inspected by shifting an attention window over the visual buffer. Through this inspection visual information, e.g., properties of a shape, as well as spatial information, e.g., spatial relations, can be inferred. It is also possible that new information is inferred from only a spatial mental image. However, no detailed information on the inspection of/inference in spatial mental images is provided by the theory. The function of image maintenance is used to re-construct parts of mental images as the information fades over time. The function of image manipulation allows the imagination of transformations of mental images. The theory posits that such manipulations are realized by altering the object map, i.e., the spatial mental image, underlying the visual mental image. One would, for example, change the location or size of an entity in the spatial mental image to alter the visual mental image that contains the shape information of that entity.

One interesting phenomenon of mental imagery is the observation of spontaneous eye movements during different visual mental imagery tasks. Brandt and Stark (1997) had participants imagine a previously memorized grid pattern and found that the eye movements during imagination reflected the content of the original stimuli. Spontaneous eye movements that reflect the processed spatial relations during mental imagery have since been found, for example, during imagination of natural scenes ( Holsanova et al., 1998 ), during imagination of detailed paintings and detailed descriptions of scenes while facing a white board as well as in total darkness ( Johansson et al., 2006 ), during reasoning with “visual” syllogisms, e.g., “a jar of pickles is below a box of tea bags,” ( Demarais and Cohen, 1998 ), and while listening to verbal descriptions of spatial scenes, e.g., “at the bottom there is a doorman in blue” ( Spivey and Geng, 2001 ). Johansson et al. (2012) report a series of experiments, in which participants were selectively forced to not move their eyes during mental imagery. They found that the suppression of eye movements has an impact on the quantity and quality of mental imagery. Their results strongly indicate a functional role of eye movements during mental imagery.

2.3. Open Questions

The previous two sections are summarized in Table 1 which provides a comparative overview of the two theories. From the comparison of the two theories, a great overlap in the assumptions made and structures and processes proposed by the two theories is evident. Many aspects of the two theories are revealed to be rather similar, perhaps more similar than one would have expected. In particular, they provide very similar descriptions of a spatial and a visual mental representation with respect to information content, localization, and hierarchical structure between the two representations. There are, however, some diverging predictions with respect to the modality of these representations and their role in reasoning. In the following, we discuss these differences and identify two main questions that arise from the comparison of these two theories.

Table 1 . Comparison of mental model theory and the theory of mental images .

The theory of mental imagery states that spatial mental images are processed in a component called the spatial-properties-processing subsystem. This subsystem is explicitly linked to the dorsal processing stream, which processes spatial information during visual perception ( Kosslyn et al., 2006 , p.138). Processing of spatial mental images uses (at least partly) the same processes used during processing of spatial information in visual perception. Mental models on the other hand are commonly assumed to be amodal or multi-modal (e.g., Johnson-Laird and Byrne, 1991 ). Accordingly, mental models are assumed to be used to also reason about abstract, non-spatial, information, e.g., “A is better than B” ( Knauff et al., 2003 ), whereas spatial mental images are assumed to process only spatial information. It has, however, been assumed that abstract information, e.g., “better than,” can be translated into spatial information in mental models ( Knauff et al., 2003 ). An opinion seemingly shared by Kosslyn (1994) , who states that information like “A is smarter than B” can be represented by dots on a line in a spatial mental image which would then correspond to a mental model in the sense of Johnson-Laird ( Kosslyn, 1994 , p. 324). The question that remains is whether the spatial representation, described as a mental model or a spatial mental image, is actually amodal/multi-modal (as claimed by mental model theory) or linked to the modality of visual perception (as seemingly proposed by the theory of mental imagery). Results pointing either way would help refining the theories.

Another open issue is the theories’ seemingly different prediction on the role of the spatial mental representation in reasoning. Unfortunately, both theories remain vague regarding the details of how spatial and visual representations interact during reasoning. In mental model theory it is often explicitly stated that it is mental models and not mental images that underlie human reasoning ( Knauff and Johnson-Laird, 2002 ; Knauff et al., 2003 ). The automatic generation of mental images through “visual” relations, e.g., “the dog is dirtier than the cat” is even considered to impede the reasoning process that happens on the level of mental models ( Knauff and Johnson-Laird, 2002 ). Of course, mental images can be important for reasoning if certain visual information is necessary, but it is not described how such visual information would be interpreted by nor how it would be transferred into the mental model for further reasoning. In the theory of mental imagery, it is made clear that visual mental images play a major role in reasoning: “[I]magery plays a critical role in many types of reasoning” ( Kosslyn, 1994 , p.404). And, contrasting mental model theory, visual mental images are assumed to be much more than just the provider of visual information for spatial mental images, in general, and particularly in reasoning ( Kosslyn, 1994 ; Kosslyn et al., 2006 ). The inspection of visual mental images constructed in the visual buffer can lead to new insights and is thus directly involved in the reasoning processes. According to Kosslyn et al. (2006) a visual mental image is generated using an underlying spatial mental image. However, the concrete role of the spatial mental image in the reasoning process is never elaborated in a way that would suggest that the spatial mental image is of specific importance to reasoning or even that it might be the actual reasoning component (as proposed in mental model theory).

Summarizing, we pointed out two main open issues regarding the differences between spatial mental representations and visual mental representations: (1) whether the spatial mental representation is rather amodal/multi-modal or whether it is also directly linked to visual perception like the visual mental representation; (2) to which extent the two mental representations are involved in reasoning, i.e., whether the spatial mental representation is the primary reasoning component or not.

3. Experiments

The comparison of the two theories, furthermore, showed that there are phenomena which have mostly been investigated only within the framework of one of the two theories. Preferences in under-specified problems have so far only been investigated within the framework of mental model theory while eye movements have been a focus of investigation almost only with mental images. In the presented experiments, we investigated to which extent these two phenomena are transferable to the respectively other type of mental representation. That is, we checked for spontaneous eye movements during reasoning with a spatial mental representation, i.e., a (spatial) mental model, and we checked for possible preferences when employing a visual mental representation, i.e., a visual mental image. In the following, we describe how the investigation of these phenomena informs us about the open questions stated in Section 3.3.

The tasks used in the experiments are three-term series relational reasoning problems about orientation knowledge. The two experiments differed only in their instructions which were formulated so that they induced the employment of a spatial mental representation in the first experiments and a visual mental representation in the second experiment.

We assume that we will confirm the findings of the literature that systematic eye movements occur during the second experiment (employing a visual mental representation) and that there are significant preferences in the answers of the participants in the first experiment (employing a spatial mental representation). The apparent functional role of eye movements during visual mental imagery provides strong evidence that visual mental representations are linked to processes of visual perception. These spontaneous eye movements reflect the spatial relations of the processed information. Both mental model theory and the theory of mental imagery assume spatial relations to be represented by a spatial mental representation, which supports the construction of a visual mental representation by providing the required spatial information. We tested whether such eye movements along the processed spatial relations would occur during employment of only a spatial mental representation, i.e., without the representation of visual content. The investigation of eye movements in this context can inform us about the question of the modality of spatial mental representations: if systematic eye movements occur during reasoning with spatial mental representations, then this would be a strong indication that mental models are not amodal, but are, in fact, linked to attentional processes of visual perception. A lack of systematic eye movements during reasoning with spatial mental representations, on the other hand, would support the assumption of mental model theory that mental models are amodal. More specifically, this would indicate that the processes of spatial mental representations do not employ the overt attentional processes of visual perception as it is the case for visual mental representations.

Preferred mental models are preferences for certain answers to under-specified reasoning problems that have been found for reasoning with mental models. These preferences are assumed to emerge because participants first construct one, perhaps the most parsimonious, mental model out of several valid models (e.g., Rauh et al., 2005 ). Visual mental images are also assumed to “depict” just one situation at a time; in fact it is hard to imagine how a “depictive” representation could represent more than one situation simultaneously. There are three possible outcomes for our investigation of such preferences for reasoning with visual mental representations: (1) we find no significant preferred answers, (2) we find different preferences for the two mental representations, or (3) we find the same preferences in reasoning with both mental representations. Finding no significant preferences in the answers when a visual mental representation is employed would strongly indicate that the assumption that visual mental representations build upon corresponding spatial mental representations is incorrect. Furthermore, this would indicate that the construction of visual mental representations can be subject to very strong individual differences. Such a finding seems unlikely and would not be predicted by any of the two theories. Should we find the same preferences in both experiments, i.e., for reasoning with both a spatial and a visual mental representation, the assumption of a hierarchal relationship between the two mental representations would be supported. This would strongly suggest that indeed the spatial configuration of a visual mental representation is taken from an underlying spatial mental representation. Should we find different preferences for the two mental representations, refinements of both mental model theory and the theory of mental imagery would be required to explain this disparity. In particular, such a finding would challenge the two theories to elaborate on their assumption that the construction of visual mental representations depends on an underlying spatial mental representation. Additionally, the strong claim made by mental model theory that reasoning is realized by spatial mental representations and not visual mental representations would without additional hypotheses be contradicted by this result.

In the following, the materials and methods employed in both conducted experiments are described.

3.1. Materials and Apparatus

The tasks used in the experiments are under-specified three-term series problems about orientation knowledge, specifically cardinal directions. We chose these problems because problems of this type, i.e., three-term series relational spatial reasoning, have been used in several studies of mental model theory before (e.g., Knauff et al., 2003 ; Byrne and Johnson-Laird, 1989 ; Schultheis et al., in revision). We use an eight-sector model of cardinal directions, i.e., the eight directions are north, north-east, east, south-east, south, south-west, west, and north-west. The problems are of the following form:

Premise 1: A is [direction 1] of B, e.g., A is north of B

Premise 2: B is [direction 2] of C, e.g., B is east of C

Conclusion: As seen from A, where is C?

The premises provide two spatial relations between three entities and the third spatial relation has to be inferred. In general, these problems are under-specified, i.e., there can be more than one correct conclusion given the premises. We used two classes of these problems, which we term 45° problems and 90° problems. These problems can be visualized as triangles with one of the three edges missing. This missing edge corresponds to the to-be-inferred spatial relation. We used all possible combinations in which the two given edges form either a 45° or a 90° angle. Figure 1 depicts an overview of all these problems.

Figure 1. The 16 different types of problems used in the experiments . The upper eight are 45° problems and the lower eight are 90° problems.

We can identify all possible correct solutions for the two problem sets. The 90° problems have three possible solutions and the 45° problems have 4 possible solutions. The different configurations leading to these solutions are depicted in Figure 2 . To distinguish the different solutions, we classify the underlying mental representations based on a visualization of the solution as triangles. In this context we use the term “model” to describe the underlying mental configuration, whether it might be a spatial mental representation or a visual mental representation. Models with very different distances for the given spatial relations are termed distorted models (DM) ; models with roughly equal distances for the given relations are termed equal-distance models (EDM) . The remaining valid solution for the 45° problems, the third solution in Figure 2 , always leads to one of the four main cardinal directions being inferred and is therefore termed cardinal model (CM) .

Figure 2. Possible valid models for a 45° problem are depicted as 1, 2, 3, and 4 . Possible valid models for a 90° problem are depicted as 5, 6, and 7. The models 1, 4, 5, and 7 are termed distorted models (DM) because the distances between the entities vary a lot from each other. The models 2 and 6 have equal distances and are termed equal-distance models (EDM). The model 3 is termed cardinal model (CM) because the to-be-inferred relation corresponds to one of the main cardinal directions, i.e., north, east, south, or west.

There are 16 different possible problems. We used them all twice with different letters resulting in a total of 32 problems. The 16 different problems consist of eight 45° and eight 90° problems, as depicted in Figure 1 .

Participants wore a head-mounted SensoMotoric Instruments (SMI) iView X HED eye tracker with a 200 Hz sampling rate to record their eye movements. To prevent expectancy effects, participants were told that the experiment investigates the size of their pupils. A post-experimental questionnaire verified that participants were not aware of the eye tracking.

3.2. Procedure

3.2.1. instructions.

The two experiments used slightly different instructions, so that they conformed with the usual instructions of both studies on mental models as well as studies on visual mental images. At the same time, the minimal change between the experiments helped to keep the tasks as similar as possible and minimize any differences besides the induced mental representation.

The instructions of the first experiment did not contain any suggestions to use visualization or visual information, but simply asked participants to infer the missing relation as fast and as accurately as possible. It is in line with previous experimental studies to assume the employment of mental models, i.e., a spatial mental representation, based on the fact that no visual information is required, given or asked for in the task (e.g., Johnson-Laird, 2001 ; Knauff and Johnson-Laird, 2002 ; Jahn et al., 2007 ).

The instructions of the second experiment only differed slightly from those of the first one. The participants were told that the letters represent cities that are to be imagined as little red squares with the respective letter next to them, which are all placed on a map. This slight variation made the instructions conform with those of several other visual mental imagery studies, i.e., using phrases such as “imagine […]” or “try to mentally see […]” (e.g., Kosslyn et al., 1983 ; Chambers and Reisberg, 1985 ; Borst et al., 2006 ).

In both experiments participants were asked to work as accurately and as fast as possible.

3.2.2. Setup

Participants were seated facing a blank white wall at a distance of approximately 1 m. Their hands were placed on their legs under a table holding a computer mouse in the one hand and a small ball in the other one. This was to prevent participants from using their fingers as an aid to solve the tasks. The eye tracker was mounted on the participant’s head and calibrated. All initial instructions of the experiment were projected on the white wall.

3.2.3. Learning phase

The experiment started with a learning phase to familiarize the participants with the cardinal directions. The learning phase consisted of acoustically presented statements and an answer screen with a question. Each statement was of the form “ K is [direction] of U .” After 4 s the answer screen appeared, which depicted the reference entity U surrounded by the numbers 1 to 8 in a counterclockwise circular order together with the question “ As seen from U, where is K? ” The eight numbers represented the eight cardinal directions (1 = north, 2 = north-west, 3 = west, … 8 = north-east). Participants answered by naming the respective number. In case of an incorrect answer, the correct answer was projected on the wall. The training phase ended as soon as each of the eight cardinal directions was recognized correctly twice in a row.

3.2.4. Problem trials

Participants were presented with a total of 48 trials. Out of those the first four were pre-trials intended to familiarize the participants with the form and procedure of the problems. Out of the remaining 44 trials, 12 were designed as filler trials. These filler trials differed in the order in which the entities were presented: AB, AC, BC, e.g., “ A is north of B; A is west of C; B is? of C ,” in contrast to the order of the remaining 32 problem trials: AB, BC, CA, e.g., “ A is north of B; B is east of C; C is? of A .” The filler trials served a double purpose. First, they were meant to prevent memory effects due to the identical order of all problem trials. Second, filler trials were employed to identify those time intervals in which participants show eye movements along the given directions. We elaborate on this method in Section 4.3. After the presentation of the four pre-trials, the remaining 44 trials were presented in randomized order.

3.2.5. Presentation

All premises and questions were presented acoustically. There was no projection on the white wall during the premises; after the conclusion phase an answer screen was projected onto the wall. Participants used the mouse to trigger the acoustic presentation of the first premise in each trial. As soon as they understood the statement, they clicked again for the presentation of the second premise. Similarly, they triggered the acoustic presentation of the question after having understood the second premise. Only after participants found an answer, they clicked the mouse again making the answer screen appear. The answer screen was the same as the one used in the learning phase. Participants verbally gave their answer by naming the number associated with the resulting direction. Participants continued to the next trial by clicking the mouse again.

The participants took between 35 and 50 min to complete the experiment.

3.3. Processing of the Eye Tracking Data

We processed the eye tracking data to identify whether eye movements occurred along the spatial relations given in each trial. We employed the same method for both experiments.

The raw eye tracking data collected by the iView X software was first converted using the IDF Event Detector to generate a list of fixations made by the participant. Saccades were calculated automatically from the sequence and coordinates of the participant’s fixations. Using the starting and ending coordinates of each saccade, we classified them into one of eight categories corresponding to the eight cardinal directions used in the trials. All possible angles of a saccade, interpreted as a vector in a Cartesian plane, were uniformly mapped to the set of cardinal directions. Each direction corresponds to a range of angles on the degree circle with each direction taking up (360°/8) = 45°. For example, north corresponded to all angles in the range of 0° ± (45°/2) = 0° ± 22.5° = [337.5°; 22.5°]. Note that the eye movements classified in this way are relative eye movements, i.e., the absolute coordinates do not matter. This is reasonable considering that participants moved their head during trials and that arbitrary eye movements occurred in between. Given this classification, we were able to investigate a possible coupling between the given direction and observed eye movements during a trial. If eye movements are linked to the processing of spatial relations, we expected eye movements to occur not only along the given direction, but also along the opposite one. Assuming a mental representation of, for example, A being north of B, it is plausible to not only expect attention shifts from A to B but also from B to A during inspection as well as construction of the representation. Thus, we always compared the absolute number of observed saccades to the sum of saccades made along the given and the opposite direction. For the first premise, we used the given direction, e.g., for the premise A is north of B we looked for saccades along the north-south axis. For the second premise, we used the direction given in the first premise as well as the new direction given in the second premise, e.g., for B is west of C , we looked for north-south (from premise 1) and for east-west. For the conclusion phase, we used the direction (and its opposite) that was given as the answer by the participant. We applied a binomial test with a probability of 1/4 to test whether the two expected directions were above chance for each participant for the first premise and the conclusion. For the second premise we applied a binomial test with a probability of 1/2 to test whether the four expected directions (two directions from each relation of the two premises) were above chance. For each phase we then applied a binomial test with a probability of 0.05 to check whether the number of participants showing significant eye movements is significantly above chance. The probability of 0.05 corresponds to how often a false positive of the previous binomial test is expected.

No prior information was available on when during the processing of the premises or the conclusion eye movements are to be expected. It is likely that participants spent some time understanding and verbally processing the presented premise or question before they started constructing the mental representation. Similarly, participants required some time preparing the action of clicking the mouse to trigger the next step after they finished the processing of the respective premise or question. We, therefore, used the obtained data during the first premises of the filler trials to gather information on when exactly participants started showing eye movements and whether we could find a temporal pattern. We only looked at eye movements during the first premise, because the filler trials are identical to the problem trials for the first premise. The difference in the order of the presented letters only became evident with the second premise. Therefore, we assumed the same behavior in the first premises of both the problem and the filler trials. We looked at the time interval between the first mention of the direction in the first premise and the time participants click to initiate the second premise. This interval was divided into ten equally long time slots. For each of these ten slots we summed up the eye movements of all participants for each experiment. We checked whether eye movements along the expected directions, i.e., those given in the respective premise (and its opposite), were significantly above chance in each of these intervals. We applied a binomial test using a probability of 1/4 for each of the four pairs of cardinal directions, e.g., north/south compared to east/west, north-east/south-west, and north-west/south-east. We applied this method independently for both experiments and used the identified time slots for the eye movement analysis of the problem trials.

3.4. Ethics Statement

The study was conducted within the Collaborative Research Center Spatial Cognition SFB/TR 8 funded by the German Research Foundation (DFG). The DFG’s board of ethics passed the research proposal that underlies the present study. DFG-funded projects do not require additional approval by other ethics committees. The studies are in full agreement with the ethical guidelines of the German Psychological Society (DGPs). Written informed consent was acquired from all participants.

4.1. Experiment 1: Spatial Mental Representation

4.1.1. participants.

Thirty undergraduate students of the University of Bremen, 12 male and 18 female, volunteered to take part in the experiment for monetary compensation.

Out of the 30 participants, one aborted the experiment and four were discarded due to an error rate of more than 30% incorrectly answered trials. The remaining 25 participants comprised 11 males and 14 females. The 0.05 level of significance was used for all statistical tests in both experiments.

4.1.2. Preferences

For the analysis of the preferences we discarded those trials for which the participants gave no or incorrect answers (12% of all trials). We compared the answers of all participants for all remaining trials to identify possible preferences. We differentiated between 90° and 45° problems and assumed that the given answers indicate the employment of the corresponding model. If no preferences existed, one would expect to observe distorted models and equal-distance models in 66% and 33% of all 90° problem trials, respectively. Likewise, distorted models, equal-distance models, and cardinal models should occur in 50%, 25%, and 25% of all 45° problem trials, respectively. To check for the existence of preferences, we compared the observed model percentages to these hypothetical ones. Figure 3 shows the resulting preferences for both problem types. There is a clear preference for the equal-distance model in the 90° problems. The answer corresponding to this model was given in 88.34% of all trials ( t (24) = 17.233; p < 0.001). The distorted models were employed significantly less than expected by chance with 11.66% ( t (24) = −17.233; p < 0.001). We found a significant preference for the equal-distance model in the 45° problems with 62.88% ( t (24) = 5.352; p < 0.001), whereas the 23.46% of the cardinal model did not differ significantly from the expected value ( t (24) = −0.215; p > 0.8). The distorted models were used significantly less than expected by chance with 13.66% ( t (24) = −9.995; p < 0.001).

Figure 3. Preferences in the first experiment . The vertical axis represents the frequency of the given answer. Top: 90° problems; bottom: 45° problems. Error bars show the standard error of the mean. EDM, equal-distance model; CM, cardinal model; DM, distorted models.

4.1.3. Eye movements

Table 2 shows the time slots identified by analyzing the eye movements during the filler trials. We used the last six out of ten time slots for our analysis of the eye movements during the actual problem trials. We decided to use all six slots despite the fact that two out of those did not show significant eye movements in the filler trials, because it is plausible that processing was not interrupted in between, but ran continuously after participants have understood the premise. Table 3 shows that the amount of participants showing eye movements along the given directions is not significant in neither the first nor the second premise (all p > 0.35), but significant during the conclusion phase ( p < 0.05).

Table 2 . Analysis of eye tracking data from the first premise of all filler trials .

Table 3 . The number of participants showing significant eye movements along the given directions .

The left parts of the Figures 4 and 5 show diagrams of the recorded eye movements during all first premises of the form A is west of B and A is north-west of B , respectively. It is evident that the percentage of saccades along the given direction and the opposing direction are not above chance, i.e., 12.5%, for both types of premises.

Figure 4. Distribution of eye movements during first premises of the form “A is west of B.” Amplitude represents the percentage of saccades mapped onto the respective cardinal direction.

Figure 5. Distribution of eye movements during first premises of the form “A is north-west of B.” Amplitude represents the percentage of saccades mapped onto the respective cardinal direction.

4.2. Experiment 2: Visual Mental Representation

4.2.1. participants.

Thirty one undergraduate students of the University of Bremen, 15 male and 16 female, participated in the study for monetary compensation.

Eight of the 31 participants were discarded due to an error rate of more than 30% incorrectly answered trials. The remaining 23 participants comprised 12 males and 11 females.

4.2.2. Preferences

Preferences were analyzed in the same way as in Experiment 1. We discarded those trials for which the participants gave no or an incorrect answer for the analysis of the preferences (9% of all trials). Figure 6 shows the preferences for both problem types. For the 90° problems, the equal-distance model was used in 93.2% of all trials, which shows a significant preference ( t (22) = 29.350; p < 0.001). Consequently, the distorted models are employed significantly below chance with 6.8% ( t (22) = −29.350; p < 0.001). For the 45° problems, we found a significant preference for the equal-distance model with 46.32% ( t (22) = 2.512; p < 0.05) as well as for the cardinal model with 47.9% ( t (22) = 2.683; p < 0.05). The distorted models were used significantly less compared to their expected value with 5.78% ( t (22) = −25.360; p < 0.001).

Figure 6. Preferences in the second experiment . The vertical axis represents the frequency of the given answer. Top: 90° problems; bottom: 45° problems. Error bars show the standard error of the mean. EDM, equal-distance model; CM, cardinal model; DM, distorted models.

4.2.3. Eye movements

Table 2 shows the time slots during which participants showed significant eye movements during the filler trials. Based on this, we used the last seven out of ten time slots for the eye movement analysis for the problem trials. We decided to use all seven slots despite the fact that one out of those did not contain significant eye movements, because we assumed, just as in the first experiment, that processing is not paused in between. Contrasting the first experiment, we found a significant amount of participants showing significant eye movements during all three phases (Prem. 1: p < 0.001; Prem. 2: p < 0.05; Concl.: p < 0.01) as shown in Table 3 .

The right parts of the Figures 4 and 5 show diagrams of the recorded eye movements during all first premises of the form A is west of B and A is north-west of B , respectively. The Figures show that saccades along the given direction as well as the opposing direction are above the frequency of chance (i.e., 12.5%) for both types of premises.

4.2.4. Comparison of eye-movers to non-eye-movers

Based on the literature, we expected to find spontaneous eye movements corresponding to the processed spatial relations when a visual mental representation is employed. In line with this assumption, a majority of participants exhibited systematic eye movements. We compared the participants that showed a significant amount of eye movements along the given directions in any of the phases (both premises or the conclusion) with those that did not show significant eye movements in any of the phases. Given this definition, 13 out of the 23 participants qualified as eye-movers; the 10 remaining participants will be referred to as non-eye-movers.

There was no significant difference between eye-movers and non-eye-movers regarding error rate, reaction times, and sex (all p > 0.19). The eye-movers and non-eye-movers, however, showed different preferences for the 45° problems as shown in Figure 7 . The eye-movers showed a significant preference for the cardinal direction model with 54.84% ( t (12) = 2.7884; p < 0.05). The equal-distance model was not significantly preferred with 37.85% ( t (12) = 1.2527; p > 0.23) and the distorted models were employed significantly less than expected with 7.31% ( t (12) = −16.1961; p < 0.001). The non-eye-movers showed no significant preference for the cardinal direction model with 38.88% ( t (9) = 0.994; p > 0.34) but for the equal-distance model with 57.32% ( t (9) = 2.2926; p < 0.05). The distorted models were significantly below expectation with 3.8% ( t (9) = −22.3421; p < 0.001).

Figure 7. Preferences of the 45° problems in the second experiment . The vertical axis represents the frequency of the given answer. Top: non-eye-movers; bottom: eye-movers. Error bars show the standard error of the mean. EDM, equal-distance model; CM, cardinal model; DM, distorted models.

4.3. Comparison of the Experiments

In the first experiment, it is only during the conclusion phase that the number of participants that showed systematic eye movements becomes significant. This finding seems unexpected given that the number of eye-movers of the second experiment is highest during the first premise whereas the number of eye-movers in the first experiment is not significant for neither premise. Furthermore, analysis of the eye movements should be most accurate for the first premise as participants are only aware of one spatial relation at that time and all saccades along the other directions can be assumed not to have any relation to the mental representation constructed. In contrast, during the second premise or the conclusion, all three spatial relations are (at least implicitly) available to the participant and could also result in eye movements, which would, however, not all be counted as “correct” eye movements, because we only checked for the spatial relations of the two premises during the second premise and we only checked for the relation that is given as the answer during the conclusion. Thus, the chance for finding significant eye movements during specifically the conclusion phase should be lower than for the first premise. It can, accordingly, be argued that eye movements during the conclusion phase did not necessarily result from the internal processing of spatial relations but that some participants moved their gaze in anticipation of the answer screen. The answer screen was projected on the wall just after participants clicked to indicate they found an answer. A saccade from the middle of their visual field toward the appropriate number on the answer screen, i.e., the number which represents their given answer, would have been mapped onto the cardinal direction that corresponds to their answer. Thus, there is reason to doubt that the significant number of eye-movers that we find for the conclusion phase in the first experiment is a result of the employed mental representation.

Given the lack of spontaneous eye movements along the processed relations for the non-eye-movers of the second experiment, we conclude that these participants did not employ a visual mental representation. This conclusion is based on the literature (see Section 3.2) which shows that employment of visual mental representations is related to the occurrence of such spontaneous eye movements and, furthermore, that these eye movements have a functional role in the employment of visual mental representations ( Johansson et al., 2012 ). It may be that the non-eye-movers likely used a spatial mental representation like the participants of the first experiment; this conclusion does, however, not follow from the observation or the literature. We, therefore, remain agnostic regarding the mental representation of the non-eye-movers of the second experiment.

4.3.1. Comparing reasoning with visual and spatial mental representations

As the two experiments consisted of the same task with only slightly different instructions, we compared participants across the experiments 1 . Reaction times that were outside a 2.5*SD range from the mean reaction time of the corresponding phase (first and second premise and the conclusion) were excluded from analysis (3%).

In order to compare the employment of visual mental representations with that of spatial mental representations, we defined two groups: the visual group and the spatial group. The spatial group comprises all participants of the first experiment. The eye-movers of the second experiment constitute the visual group. That is, the spatial group contains those participants which employed a spatial mental representation and the visual group contains those that employed a visual mental representation. There were no significant differences regarding error rate, reaction times, and sex (all p > 0.35) between the visual and the spatial group. However, the preferences of the two groups differed as indicated by a significant interaction between group (spatial or visual) and type of model (cardinal or equal-distance), F (1,36) = 5.644 ,p < 0.05. Figure 8 shows the preferences of the visual and the spatial group. The spatial group showed a preference for the equal-distance model (EDM) but not for the cardinal model (CM). In contrast, the visual group showed a preference for the cardinal model (CM) but not the equal-distance model (EDM). Table 4 shows an overview of the preferences for the different groups and experiments. Interestingly, the non-eye-movers of the second experiment showed the same preferences as the participants of the first experiment, i.e., a significant preference for the equal-distance model (EDM) and no significant preference for the cardinal model (CM). This may be taken to indicate that the non-eye-movers employed a spatial mental representation despite the fact that the instructions are formulated to induce a visual mental representation.

Figure 8. Preferences of the 45° problems for the spatial group (top) and the visual group (bottom) . The vertical axis represents the frequency of the given answer. Error bars show the standard error of the mean. EDM, equal-distance model; CM, cardinal model; DM, distorted models.

Table 4 . Comparison of preferences for the 45° problems between different groups; S+, frequency significantly above chance; S−, frequency significantly below chance; NS, frequency does not significantly differ from chance; CM, cardinal model; EDM, equal-distance model; DM, distorted models .

5. Discussion

The conducted experiments yielded two main results. First, the employed reasoning task led to no significant systematic eye movements when a spatial mental representation was employed, i.e., for the spatial group. In contrast, we found significant systematic eye movements for a majority of the participants in the second experiment, i.e., the visual group which employed a visual mental representation. Second, there are significant preferences in the answers for the under-specified problems in both the visual and the spatial group. The preferences did, however, differ between the employed mental representations.

These results relate to the two main open issues about the relationship between spatial and visual mental representation (identified in Section 3.3): (1) whether spatial mental representations are modality-specific, and (2) whether human visuo-spatial reasoning is realized on the level of spatial mental representations.

Regarding the first issue, we observed systematic eye movements in the second experiment but not for the first experiment. The eye movements observed in second experiment, i.e., the one in which the employment of a visual mental representation was induced, corroborate several studies reporting spontaneous eye movements during visual mental imagery. The fact that we did not find these eye movements for the essentially same reasoning task in the first experiment, i.e., the one in which the employment of a spatial mental representation was induced, suggests that other (attentional) processes are employed when reasoning with spatial mental representations. Since eye movements have been found to play a functional role in processing visual mental representations ( Johansson et al., 2012 ) and are therefore not epiphenomenal, we can conclude that reasoning with visual mental representations draws on overt attentional processes of visual perception and reasoning with spatial mental representations does not. This finding lends support to the assumption of mental model theory that spatial mental representations are amodal or multi-modal.

Regarding the second issue – whether reasoning is realized on the level of spatial mental representations – our results show different preferences depending on the employed mental representation. The visual group showed a significant preference for the cardinal model (CM) but not for the equal-distance model (EDM) for the 45° problems. In contrast, the spatial group showed a significant preference for the equal-distance model (EDM) but not for the cardinal model (CM) for the 45° problems. Mental model theory assumes that the hierarchical relationship between visual mental representations and spatial mental representations is such that reasoning happens on the level of the spatial mental representation ( Knauff and Johnson-Laird, 2002 ; Knauff et al., 2003 ) specifically when visual information is irrelevant to the task at hand (as it is the case in the presented experiments). This assumption seems in contradiction to the presented results. The fact that we observed different preferences for the two mental representations for essentially the same reasoning task challenges the claim that reasoning is based on spatial mental representations. This similarly affects the theory of mental imagery which also states that visual mental representations require underlying spatial mental representations. In order to construct, inspect and reason with a visual representation, spatial information is necessary to, for example, “know” the location, size, and spatial relations of the shapes that make up a visual mental representation. The results of the experiments are thus hard to reconcile with both the mental model theory and the theory of mental imagery. The assumed hierarchical relationship between spatial and visual mental representations has to be extended with additional explanations about how spatial information is transformed or processed differently in a visual mental representation. In the following, we interpret the results on the preferences with respect to this assumption of the two theories.

The preferred answer given by participants in the spatial group was such that the spatial configuration of the problem has equal distances between the entities. In contrast, the preferred answer of the participants in the visual group was such that the spatial configuration contains distances of different length. This is especially puzzling given the assumption that those spatial relations are supposed to be provided by the spatial mental representation to the visual mental representation. Sticking with the assumption that the spatial information is provided by the underlying spatial representation, one can think of two general explanations: (1) the spatial relations are somehow altered in the context of a visual representation, or (2) the spatial relations are the same but are processed differently in the two mental representations. Regarding the first option, spatial relations might become more specified when represented in a visual mental representation ( Schultheis et al., 2007 ). That is, additional properties such as distance are specified. On the level of the spatial mental representation, distance might only be represented with generic default values. This would fit with the preferred mental model of the spatial group, in which all distances are equal. This option is furthermore supported by an assumption of mental model theory: “[w]hen people understand spatial descriptions, they imagine symmetrical arrays in which adjacent objects have roughly equal distances between them […]” ( Johnson-Laird and Byrne, 1991 , p. 94). Regarding the second option, the way spatial relations are processed could differ between the two mental representations. This explanation would fit well with the fact that we found spontaneous eye movements that align with the currently processed spatial relations for the visual group, but we did not find such eye movements for the spatial group. An implementation of the second option is proposed by a new model of visuo-spatial mental imagery in which processing of spatial relations is affected by additional visual information and realized by attention shifts such as eye movements ( Sima, 2011 ; Sima and Freksa, 2012 ).

6. Conclusion

Our experiments provided two new insights on the so far little investigated relationship between visual and spatial mental representations: (1) visual and spatial mental representations differ in their employment of overt attentional processes of visual perception, (2) there are preferences when employing visual mental representations just as for spatial mental representations, but the preferences can differ for the same reasoning task. These findings are hard to reconcile with current theories on visuo-spatial processing and challenge some of their assumptions. Future work is necessary to shed more light on the exact relationship between visual and spatial mental representations. This will have to include the refinement of the existing theoretical frameworks on the one hand as well as further empirical research on the other hand. Regarding the theories, we have additionally presented a systematic comparison of mental model theory and the theory of mental imagery. This comparison showed that the two theories that are often investigated separately likely investigate the same visual and spatial mental representations. This comparison might serve as the basis of a new unified theory combining the results achieved within both mental model theory and the theory of mental imagery. Regarding the future empirical work, the presented experiments show one way of comparing visual and spatial mental representations while keeping the experimental task essentially the same.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The presented work was done in the project R1-[ImageSpace] of the Transregional Collaborative Research Center SFB/TR 8 Spatial Cognition. Funding by the German Research Foundation (DFG) is gratefully acknowledged. We thank Sven Bertel for fruitful discussions with respect to experimental design and Maren Lindner for her role in designing and conducting the experiments. We thank Thomas Lachmann, David Peebles, and Jelica Nejasmic for their comments and suggestions which helped improve this article.

• ^ There is evidence that the ( a priori ) differences between the two groups of the two experiments were not more substantial or qualitatively different than if the groups had resulted from random assignments within a single experiment. First, the participants of both experiments were recruited from the same population of students from the University of Bremen. The setup of the two experiments was identical apart from the variation in instructions. This includes specifically the equipment, the room, the experimenter, and the materials. The experiments were conducted within two consecutive semesters. Second, the two groups did not significantly differ with respect to sex ( χ ( 1 ) 2 = 1 . 42 , p > 0 . 2 ) or field of study ( χ ( 2 ) 2 = 1 . 18 , p > 0 . 5 ) . The two groups also did not differ significantly in their age ( t (46) = –1.084; p > 0.28, two-tailed) or performance in the paper-folding test ( t (46) = –0.455; p > 0.65, two-tailed). Third, using the method described in Masson (2011) , the participants age and performance in the paper-folding test, provided positive evidence for the null hypothesis that the two groups did not differ ( p ( H 0 | D ) = 0.79, p ( H 0 | D ) = 0.86, respectively).

Borst, G., Kosslyn, S., and Denis, M. (2006). Different cognitive processes in two image-scanning paradigms. Mem. Cognit. 34, 475–490.

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Brandt, S. A., and Stark, L. W. (1997). Spontaneous eye movements during visual imagery reflect the content of the visual scene. J. Cogn. Neurosci. 9, 27–38.

CrossRef Full Text

Byrne, R. M. J., and Johnson-Laird, P. (1989). Spatial reasoning. J. Mem. Lang. 28, 564–575.

Chambers, D., and Reisberg, D. (1985). Can mental images be ambiguous? J. Exp. Psychol. Hum. Percept. Perform. 11, 317–328.

Courtney, S. M., Ungerleider, L. G., Keil, K., and Haxby, J. V. (1996). Object and spatial visual working memory activate separate neural systems in human cortex. Cereb. Cortex 6, 39–49.

Demarais, A. M., and Cohen, B. H. (1998). Evidence for image-scanning eye movements during transitive inference. Biol. Psychol. 49, 229–247.

Farah, M. J., Levine, D. N., and Calvanio, R. (1988). A case study of mental imagery deficit. Brain Cogn. 8, 147–164.

Finke, R. A. (1989). Principles of Mental Imagery . Cambridge, MA: MIT Press.

Goel, V., and Dolan, R. J. (2001). Functional neuroanatomy of three-term relational reasoning. Neuropsychologia 39, 901–909.

Holsanova, J., Hedberg, B., and Nilsson, N. (1998). “Visual and verbal focus patterns when describing pictures,” in Current Oculomotor Research: Physiological and Psychological Aspects , eds W. Becker, H. Deubel, and T. Mergner (New York: Plenum) 303–304.

Jahn, G., Knauff, M., and Johnson-Laird, P. N. (2007). Preferred mental models in reasoning about spatial relations. Mem. Cognit. 35, 2075–2087.

Johansson, R., Holsanova, J., Dewhurst, R., and Holmqvist, K. (2012). Eye movements during scene recollection have a functional role, but they are not reinstatements of those produced during encoding. J. Exp. Psychol. Hum. Percept. Perform. 38, 1289–1314.

Johansson, R., Holsanova, J., and Holmqvist, K. (2006). Pictures and spoken descriptions elicit similar eye movements during mental imagery, both in light and in complete darkness. Cogn. Sci. 30, 1053–1079.

Johnson-Laird, P. N. (1972). The three-term series problem. Cognition 1, 57–82.

Johnson-Laird, P. N. (1989). “Mental models,” in Foundations of Cognitive Science , ed. M. I. Posner (Cambridge, MA: MIT Press), 469–499.

Johnson-Laird, P. N. (1998). “Imagery, visualization, and thinking,” in Perception and Cognition at Century’s End , ed. J. Hochberg (San Diego: Academic Press), 441–467.

Johnson-Laird, P. N. (2001). Mental models and deduction. Trends Cogn. Sci. (Regul. Ed.) 5, 434–442.

Johnson-Laird, P. N., and Byrne, R. M. J. (1991). Deduction . Hove: Erlbaum.

Klauer, K., and Zhao, Z. (2004). Double dissociations in visual and spatial short-term memory [Review]. J. Exp. Psychol. Gen. 133, 355–381.

Knauff, M., Fangmeier, T., Ruff, C. C., and Johnson-Laird, P. N. (2003). Reasoning, models, and images: behavioral measures and cortical activity. J. Cogn. Neurosci. 15, 559–573.

Knauff, M., and Johnson-Laird, P. (2002). Visual imagery can impede reasoning. Mem. Cognit. 30, 363–371.

Kosslyn, S. (1973). Scanning visual images – some structural implications. Percept. Psychophys. 14, 90–94.

Kosslyn, S. M. (1980). Image and Mind . Cambridge, MA: Harvard University Press.

Kosslyn, S. M. (1994). Image and Brain: The Resolution of the Imagery Debate . Cambridge, MA: The MIT Press.

Kosslyn, S. M., Reiser, B. J., Farah, M. J., and Fliegel, S. L. (1983). Generating visual images: units and relations. J. Exp. Psychol. Gen. 112, 278–303.

Kosslyn, S. M., and Thompson, W. L. (2003). When is early visual cortex activated during visual mental imagery? Psychol. Bull. 129, 723–746.

Kosslyn, S. M., Thompson, W. L., and Ganis, G. (2006). The Case for Mental Imagery . New York: Oxford University Press.

Levine, D. N., Warach, J., and Farah, M. (1985). Two visual systems in mental imagery: dissociation of “what” and “where” in imagery disorders due to bilateral posterior cerebral lesions. Neurology 35, 1010–1018.

Masson, M. E. J. (2011). A tutorial on a practical Bayesian alternative to null-hypothesis significance testing. Behav. Res. Methods 43, 679–690.

Mellet, E., Tzourio-Mazoyer, N., Bricogne, S., Mazoyer, B., Kosslyn, S. M., and Denis, M. (2000). Functional anatomy of high-resolution visual mental imagery. J. Cogn. Neurosci. 12, 98–109.

Newcombe, F., Ratcliff, G., and Damasio, H. (1987). Dissociable visual and spatial impairments following right posterior cerebral lesions: clinical, neuropsychological and anatomical evidence. Neuropsychologia 25, 149–161.

Pylyshyn, Z. W. (2002). Mental imagery: in search of a theory. Behav. Brain Sci. 25, 157–238.

Rauh, R., Hagen, C., Knauff, M., Kuss, T., Schlieder, C., and Strube, G. (2005). Preferred and alternative mental models in spatial reasoning. Spat. Cogn. Comput. 5, 239–269.

Rips, L. J. (1994). The Psychology of Proof: Deductive Reasoning in Human Thinking . Cambridge, MA: MIT Press.

Sack, A. T. (2009). Parietal cortex and spatial cognition. Behav. Brain Res. 202, 153–161.

Schultheis, H., and Barkowsky, T. (2011). Casimir: an architecture for mental spatial knowledge processing. Top. Cogn. Sci. 3, 778–795.

Schultheis, H., and Barkowsky, T. (2013). Variable stability of preferences in spatial reasoning. Cogn. Process. doi:10.1007/s10339-013-0554-4

Schultheis, H., Bertel, S., Barkowsky, T., and Seifert, I. (2007). “The spatial and the visual in mental spatial reasoning: an ill-posed distinction,” in Spatial Cognition V – Reasoning, Action, Interaction , eds T. Barkowsky, M. Knauff, G. Ligozat, and D. R. Montello (Berlin: Springer Verlag), 191–209.

Sereno, M. I., Pitzalis, S., and Martinez, A. (2001). Mapping of contralateral space in retinotopic coordinates by a parietal cortical area in humans. Science 294, 1350–1354.

Sima, J. F. (2011). “The nature of mental images – an integrative computational theory,” in Proceedings of the 33rd Annual Conference of the Cognitive Science Society , eds L. Carlson, C. Hoelscher, and T. Shipley (Austin, TX: Cognitive Science Society), 2878–2883.

Sima, J. F., and Freksa, C. (2012). Towards computational cognitive modeling of mental imagery. Künstliche Intell. 26, 1–7.

Smith, E. E., and Jonides, J. (1997). Working memory: a view from neuroimaging. Cogn. Psychol. 33, 5–42.

Spivey, M. J., and Geng, J. J. (2001). Oculomotor mechanisms activated by imagery and memory: eye movements to absent objects. Psychol. Res. 65, 235–241.

Tversky, B. (1993). “Cognitive maps, cognitive collages, and spatial mental models,” in Spatial Information Theory: A Theoretical Basis for GIS – Proceedings of COSIT’93 , eds A. U. Frank and I. Campari (Berlin: Springer), 14–24.

Ungerleider, L., and Mishkin, M. (1982). “Two cortical systems,” in Analysis of Visual Behavior , eds D. J. Ingle, M. A. Goodale, and R. J. W. Mansfield (Cambridge: MIT Press), 549–586.

Keywords: mental representation, mental imagery, mental models, preferred mental models, visual mental representation, spatial mental representation, eye tracking

Citation: Sima JF, Schultheis H and Barkowsky T (2013) Differences between spatial and visual mental representations. Front. Psychol. 4 :240. doi: 10.3389/fpsyg.2013.00240

Parts of this research have been presented at the Spatial Cognition conference 2010.

Received: 07 December 2012; Accepted: 12 April 2013; Published online: 08 May 2013.

Reviewed by:

Copyright: © 2013 Sima, Schultheis and Barkowsky. This is an open-access article distributed under the terms of the Creative Commons Attribution License , which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.

*Correspondence: Jan Frederik Sima, Department of Informatics, Cognitive Systems, Universitat Bremen, Enrique-Schmidt-Str. 5, 28359 Bremen, Germany. e-mail: sima@sfbtr8.uni-bremen.de

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.