32  Acting on the Seen World

Visual Guidance, Reach-to-Grasp Control, and Skilled Object Use

32.1 From force to a target in the world

The preceding chapter followed action through motor cortex, descending pathways, spinal circuits, motor neurons, and muscle. Those systems make force available and organize it into postural adjustments, withdrawal, locomotion, and flexible voluntary movement. They do not, by themselves, specify which cup to reach toward, where its handle lies relative to the hand, or how the fingers should change when the cup begins to slip.

Those are relational problems. A target is visible at one place on the retina, but the eyes can move within the head, the head can turn on the trunk, the trunk can lean, and the arm can begin from many configurations. The hand is not a point that merely has to arrive at a coordinate. It has a size, orientation, joint limits, possible contact surfaces, and a history of forces acting upon it. The target can move. The animal can move. The intended outcome can change while the movement is already underway.

Visually guided action therefore cannot be reduced to a one-time conversion of an image into a completed motor command. It is a recurrent control process in which visual, proprioceptive, tactile, vestibular, and motor-related signals continually update the relation among the target, the body, and the action in progress.

Acting on the seen world means continually keeping a changing body related to a changing target.

Vision is prominent in this chapter because it often identifies distant opportunities before the body contacts them. It is not sufficient. A person reaching in darkness can combine a remembered target location with proprioceptive evidence about the arm. A climber adjusts grip when tactile afferents signal incipient slip. Vestibular and postural systems stabilize the body from which the arm moves. What is commonly called a visuomotor transformation is therefore embedded within a multisensory and biomechanical control system.

32.2 Orienting comes before reaching

Before an animal can manipulate an object, it must usually orient toward something that matters. This problem is older than hands and older than neocortex.

Across vertebrates, the optic tectum—called the superior colliculus in mammals—contains spatially organized sensory signals and contributes to orienting the eyes, head, and body toward relevant events. Work in living lamprey makes the ancestral organization especially visible. Different regions of the tectal map can recruit brainstem pathways that turn the animal toward a possible target or away from a possible threat. These circuits do not produce a primate reach, but they solve a more basic version of the same control problem: relate a mapped event in the world to a coordinated change in the body [@suzuki2019tectum; @grillner2021evolution].

Evolution did not replace that midbrain system when forelimbs became available for manipulation. It added control above control. In primates, the superior colliculus is best known for its contribution to gaze shifts, but its activity is also related to coordinated eye–hand behavior. Collicular neurons can be active before and during arm movements toward visual targets [@werner1993arm]. Behaviorally, gaze commonly reaches an object before the hand and can remain anchored near it while the hand approaches [@neggersbekkering2000]. None of this means that every reach requires the eyes to remain fixed on the target. People can reach toward peripheral, remembered, or briefly viewed locations. The point is that orienting and reaching are normally coordinated components of one action rather than independent events.

Expanded posterior parietal and frontal systems added capacities that a tectal orienting map alone could not provide. A primate can select one object among several, delay a reach, use an arbitrary cue, choose either hand, route the arm around an obstacle, rotate the wrist, and shape individual digits for a precision grip. Those newer cortical contributions still act through the postural, brainstem, spinal, and muscular systems described in Chapter 31. They also remain coupled to the older orienting system.

The evolutionary progression is therefore not visual cortex replaces tectum, and motor cortex replaces brainstem. It is a hierarchy of interacting controllers. The tectum or superior colliculus helps align sensory surfaces and the body with events in space. Parietofrontal networks add flexible limb-specific and object-specific control. Descending pathways recruit older brainstem and spinal machinery. The changed body then generates new visual, proprioceptive, vestibular, and tactile evidence.

32.3 A target has no single coordinate

The cortical contribution to visually guided action is often summarized as the dorsal visual stream, extending from occipital visual cortex into posterior parietal and frontal motor-related regions. The name is useful if stream is not imagined as one river carrying a finished visual description forward. Several occipitoparietal routes interact recurrently with frontal cortex, the superior colliculus, somatosensory systems, and ventral visual networks. Posterior parietal cortex does not become purely motor when visual signals arrive there; it helps maintain changing relations among sensory events, the body, and possible actions [@goodalemilner1992; @goodalewestwood2004].

A spot of light falling five degrees to the right of the fovea specifies a location on the retina. It does not, by itself, specify where an object lies relative to the head, shoulder, hand, or room. The same retinal location can be produced by many combinations of eye position and object position. Turn the eyes to the left and an object straight ahead can fall on the same retinal region as an object to the right when gaze is centered.

The ambiguity multiplies as action approaches the body. The eyes rotate within the head. The head rotates relative to the trunk. The shoulder translates when the trunk leans. The elbow and wrist change the location and orientation of the hand. A reach also begins from a particular posture, and the nervous system must estimate that posture from proprioception, touch, vision of the limb, and motor-related signals. Each source carries noise and delay. The current hand location is therefore not simply read from one sensor.

Researchers describe these relations using reference frames. A neuron or population may be characterized as more eye-centered, head-centered, body-centered, hand-centered, or object-centered when its activity varies most systematically with one of those descriptions. These terms are useful, but they should not be mistaken for a row of anatomical boxes through which information passes in sequence. Posterior parietal neurons commonly combine variables. A cell may be strongly tuned to the retinal location of a target while its response is modulated by gaze direction, hand position, intended effector, or current task. Nearby cells can express different mixtures [@andersenmountcastle1983; @chang2010idiosyncratic].

Human imaging likewise reveals partial and overlapping reference frames rather than one final coordinate transformation. During reach preparation, some parietofrontal activity is better described relative to gaze, some relative to the hand or body, and some by mixtures that change with the task and stage of preparation [@beurze2010reference; @sobersabes2005]. A network can therefore preserve several descriptions at once. That may be advantageous. Eye-centered information remains useful when the eyes move again; hand-centered error is useful for directing the limb; object-centered geometry is useful for shaping a grasp; and body-centered information helps keep the action compatible with posture and balance.

A reference frame is consequently an analysis of how neural activity relates to variables in an experiment. It is not necessarily the format of an internal picture. Nor must every intermediate description be converted into one master map before movement can begin. Distributed populations can support action because their combined activity contains enough information to relate the current target to the current body.

NoteDeeper Dive: What a gain field does—and does not—compute

A classic posterior parietal neuron may respond when a visual target appears in a particular retinal region. Its visual receptive field can remain anchored to the retina while the magnitude of the response changes with eye position. The target is the same distance from the fovea, but the neuron fires more strongly when the eyes point left than when they point right. Eye position has altered the neuron’s gain [@andersenmountcastle1983].

A population containing many such combinations can distinguish situations that a retinal signal alone cannot. One neuron prefers a target to the right of gaze and is amplified when gaze is leftward; another has a different retinal preference and a different eye-position modulation. Taken together, the pattern can support an estimate of target location relative to the head or body.

This finding does not mean that a gain-modulated neuron has calculated a complete head-centered coordinate. Its receptive field may remain largely retinal, and other variables can modulate it as well. Nor does it show that posterior parietal cortex first performs one explicit equation and then hands a finished answer to premotor cortex. Gain fields are one population mechanism by which several signals can be combined. The resulting representations remain mixed, distributed, and dependent on the behavior being performed [@chang2010idiosyncratic].

32.4 Reaching and grasping are coupled controllers

A reach toward a cup can be divided descriptively into two components. Transport moves the hand toward the object. Grasp formation changes hand aperture, finger configuration, and wrist orientation so that appropriate surfaces can be contacted. The distinction is useful because these components impose partly different demands and are associated with relative biases within parietofrontal cortex [@jeannerod1984prehension].

A dorsomedial network that includes medial and superior posterior parietal regions and dorsal premotor cortex is strongly engaged by reaching. A more lateral network that includes the anterior intraparietal region and ventral premotor cortex is especially associated with hand shaping and grasp. Human imaging reveals corresponding reach- and grasp-related gradients, and macaque recordings show neurons sensitive to target location, gaze, wrist orientation, grip type, and combinations of these variables [@konen2013parietal; @murata2000aip; @fattori2009v6a].

The terms reach pathway and grasp pathway should nevertheless be used as shorthand for functional biases, not exclusive ownership. The hand begins to open and rotate before transport is complete. Maximum grip aperture is timed relative to the approach of the hand. A change in object orientation alters the wrist trajectory; an obstacle can alter both the route of the arm and the side from which the fingers approach. Reaching and grasping are prepared concurrently and continually constrain one another.

Neither component is specified by visual geometry alone. Imagine lifting a familiar mug. Its visible shape helps determine where the thumb and fingers can contact it, but successful lifting also depends on the initial posture of the arm, joint limits, expected weight, surface friction, and what the mug is being lifted for. A sip, a pass to another person, and placing the mug upside down can begin with similar transport and diverge in wrist orientation, contact points, and forces. Visual object information associated with ventral temporal cortex interacts with parietofrontal systems rather than remaining sealed in a separate stream.

The body also contributes constraints that are easy to omit from a diagram of cortical pathways. Shoulder rotation changes the feasible range of elbow and wrist positions. The trunk can extend the reachable workspace. Gravity alters the torques required along different trajectories. The fingers must contact the object without destabilizing it, and the arm must preserve the body’s balance while lifting the load. A neural controller works with these mechanics rather than calculating a movement for an abstract, weightless hand.

This is another expression of motor abundance. Many joint configurations can bring the hand to the same region, but not all are equally stable, economical, comfortable, or compatible with the intended grasp. Parietofrontal activity helps constrain the available solutions; spinal feedback, muscle properties, and the physical interaction with the object continue to shape which solution is realized.

32.5 The action is corrected while it unfolds

The older language of sensorimotor transformation can suggest a sequence: see the target, transform its coordinates, prepare the movement, and execute the completed plan. Real reaching exposes the limits of that sequence.

In a target-jump experiment, a person begins reaching toward a visible target. After the movement starts, the target is displaced. The hand often changes course rapidly, sometimes even when the displacement is difficult to report. The correction does not require vision of the hand itself; information about the new target can be combined with an estimate of the moving limb [@goodale1986adjustments; @pelisson1986doublestep]. The trajectory that finally appears was therefore not wholly specified before movement onset. It emerged from preparation plus continuing sensory control.

Posterior parietal cortex is important for these updates. Brief disruption by transcranial magnetic stimulation can impair correction to a displaced target, and parietal lesions can produce a related inability to redirect the hand efficiently [@desmurget1999updating]. Yet the correction should not be assigned to one cortical site. Visual signals pass through several cortical and subcortical routes. Premotor and motor cortex alter descending output. Proprioceptive feedback reports the changing limb. Spinal and brainstem circuits respond to the resulting forces. The cerebellum helps estimate state and calibrate correction. The limb’s own inertia, elasticity, and interaction with the environment determine how any neural change affects the trajectory.

Correction also continues after contact. When the fingertips first touch an object, cutaneous afferents signal contact location, pressure, vibration, and local slip. If a smooth object begins to move within the grasp, grip force increases rapidly to restore a safety margin. Experience with the object also shapes the forces applied before slip occurs. Precision grip therefore combines anticipatory scaling with tactile correction rather than choosing between feedforward and feedback control [@johanssonwestling1984; @johanssonwestling1987].

Proprioception supplies another route. A mechanical perturbation that displaces the arm changes muscle length, joint configuration, and skin deformation. Early spinal responses are followed by later responses that incorporate more of the task, as described in Chapter 31. Visual and proprioceptive evidence can be given different weights depending on their reliability and on which variable the movement must control. When the hand is hidden, proprioception becomes especially important; when limb vision is clear, the two sources can be combined. Neither is a perfect copy of the limb’s state [@sobersabes2005].

The useful unit is therefore not a visual signal followed by a motor output. It is a recurrent relation:

1. a target is estimated relative to gaze, body, and hand;
2. descending activity changes the limb;
3. the movement changes retinal, proprioceptive, and tactile input;
4. the estimated relation is updated;
5. output changes again where the remaining error matters.

These stages overlap. Sensory processing continues during movement, and motor-related activity changes how incoming sensory evidence is interpreted. The nervous system is not repeatedly starting the action over. It is preserving some task variables—hand near target, stable contact, upright body—while allowing many lower-level details to vary.

NoteDeeper Dive: Can the hand correct before the person notices?

Some target displacements produce rapid hand corrections even when participants do not give a reliable verbal report of the change. Findings of this kind helped motivate the idea of an automatic pilot for the hand [@pisella2000pilot]. The phrase captures an important fact: visual evidence can influence an unfolding action without waiting for a deliberate decision or a verbal description.

It should not be turned into a separate miniature agent. Correction speed depends on target visibility, attention, movement phase, displacement size, and the response being measured. A person may also become aware of the change after the correction has begun. Failure to report a brief displacement does not prove that the information traveled through a wholly unconscious pathway, and successful correction does not show that conscious perception played no role in the larger action.

The more defensible conclusion is that perceptual report and rapid visuomotor correction place different demands on a distributed visual system. Their timing and vulnerabilities can differ without making them independent.

32.6 Affordances depend on the current body

James Gibson used the term affordance for an opportunity for action available to an animal in an environment [@gibson1979ecological]. An affordance is not simply a visual feature stored inside an object. It is a relation.

A ledge affords sitting only for a body of an appropriate size and mobility. A gap affords passage for one animal but not another. A cup handle may afford a precision grip from one direction, a whole-hand grip from another, and no comfortable grasp when the wrist is already near its limit. A target just beyond reach can become reachable when the trunk is free to lean or when a tool extends the hand. Fatigue, injury, current posture, and what the animal is trying to accomplish all alter the available action.

This relational definition prevents a common error. The visual system does not first label an object graspable in isolation and then send that property to motor cortex. Object shape, location, body configuration, available effectors, and current goals jointly constrain what can be done. The same object can invite several actions, and the same action can often be achieved with several effectors.

32.6.1 Peripersonal space is a multisensory control interface

Some neurons in parietal and premotor cortex respond both to touch on a particular body region and to visual events near that region. A neuron responsive to touch on the face may also respond when an object approaches the face; another may combine touch on the hand with vision near the hand. In macaque ventral premotor area F4, such visual receptive fields can move with the associated body part rather than remaining fixed at one location in the room [@fogassi1996peripersonal; @grazianocooke2006].

These findings are often described as representations of peripersonal space, the immediately reachable or body-relevant region around the animal. The term does not refer to one sharply bounded cortical bubble. The represented region changes with posture, task, threat, and potential contact. Visual–tactile convergence can support reaching and grasping, but also avoidance, defense, and protection of the body [@fogassi1996peripersonal; @grazianocooke2006]. It is better understood as a flexible multisensory interface between events near the body and the actions those events may require.

A visual event near the hand can therefore be represented differently from an identical event far from the body, not because near space is a separate sense, but because the event has different consequences. It may be contacted, grasped, deflected, or avoided with little delay. The geometry of the environment is being evaluated relative to a body capable of acting within it.

32.6.2 Several actions can be available at once

A cup can be grasped by the handle, lifted around its body, pushed aside, passed to another person, or ignored. One influential proposal—the affordance competition hypothesis—is that parietal and frontal populations can specify several potential actions in parallel, while goals, rules, expected outcomes, and selection systems bias the competition [@cisek2007affordance; @cisekkalaska2005]. On this view, perception does not have to finish constructing a neutral description before action possibilities appear.

The proposal fits many observations, but it is a framework rather than a settled map of one competition circuit. Neural activity related to more than one possible reach does not by itself establish that the neurons are literally competing. Experimental choices are usually few, highly trained, and defined by the investigator. The important insight is more general: the world normally permits several actions, and action selection can alter sensorimotor processing before one movement has been released.

This provides a natural link to the basal ganglia. Parietofrontal systems help specify where and how the body could act; basal-ganglia and frontal systems contribute to whether one candidate is selected, withheld, or replaced. The division is not absolute. Selection signals influence parietal and premotor activity, and the sensory consequences of an emerging action can change what remains available.

32.6.3 Actions are nested inside larger actions

A grasp is rarely an end in itself. It can be the first component of eating, placing, pouring, passing, or throwing. Recordings from macaque inferior parietal cortex showed that some neurons responding during a grasp did so differently when the grasp was followed by bringing food to the mouth versus placing the object in a container [@fogassi2005actions].

The result is often described as coding the intention of the action. That wording risks claiming more than the experiment establishes. The neurons were sensitive to the larger sequence in which the grasp occurred, including the expected next act and the cues that distinguished the conditions. This is already consequential. It shows that a local movement can be configured in relation to what comes next rather than represented only by its immediate kinematics.

Action is hierarchical in time as well as anatomy. Finger closure belongs to a grasp; the grasp belongs to lifting the cup; lifting belongs to drinking or passing; the larger action belongs to the animal’s current needs and social context. Later levels of control do not merely begin after earlier ones finish. They alter how the component action is prepared and performed.

32.7 When visual guidance fails: optic ataxia

The term optic ataxia describes a disproportionate impairment in using vision to guide the hand toward a target. A patient may see and identify an object yet misreach, misorient the wrist, or form an inappropriate grasp, despite having enough strength and elementary movement to perform the action under other conditions. The syndrome is most strongly associated with damage to posterior parietal and occipitoparietal systems [@pereninvighetto1988].

The deficit is not uniform. Errors are often greater for targets presented away from fixation, in particular regions of the visual field, or with one hand. Some patients show a strong contralesional field effect; others show hand-related or fixation-related patterns. Reaching to the remembered location of a target, reaching with central vision, or using proprioceptive and tactile information can produce a different level of impairment. Individual lesions also differ in their cortical and white-matter involvement [@pereninvighetto1988; @pisella2000pilot].

Optic ataxia should not be confused with weakness. The arm can generate force and follow a trajectory. Nor is it identical to cerebellar ataxia, in which prediction, timing, coordination, and error correction can be disturbed across visual and nonvisual tasks. The historical word ataxia identifies disordered coordination in both names, but the disorders arise from different failures within the control hierarchy.

Bilateral parietal injury can produce optic ataxia as part of Bálint syndrome, together with disturbed visual attention and difficulty directing gaze. Unilateral lesions can produce more restricted forms. It is therefore inaccurate to define optic ataxia only as one component of the full bilateral syndrome.

As discussed in the third vision chapter, patient D.F. became an important contrast case because some immediate, visually guided actions were less impaired than her explicit judgments of object form. Optic ataxia appears to reverse that emphasis: perception can be relatively successful while online guidance of the hand fails. The contrast helped establish that visual judgments and visually guided actions can be differentially vulnerable [@goodale1991; @pereninvighetto1988].

The contrast should remain conditional. D.F.’s action performance is not normal under every condition and can depend on immediate viewing, fixation, task design, and haptic feedback. Patients with optic ataxia do not lose all visually guided action while retaining an otherwise untouched visual world. Their deficits vary with field, hand, delay, available feedback, and lesion anatomy [@schenk2012]. The cases support differently weighted contributions within interacting networks, not two sealed pipes called perception and action.

Optic ataxia is especially informative because it reveals the difference between locating an object in a visual scene and continually relating that object to a moving limb. Naming the cup, describing its orientation, and reaching accurately toward its handle are not interchangeable tests. Each places different demands on vision, attention, body-state estimation, and online correction.

32.8 A different failure: limb apraxia

A person with optic ataxia has particular difficulty guiding the limb with current visual information. A person with limb apraxia can have a different problem: organizing familiar, learned actions even though elementary strength, range of movement, sensation, and comprehension are not sufficient to explain the failure.

The definition is necessarily clinical rather than absolute. Weakness, sensory loss, aphasia, neglect, memory impairment, and executive dysfunction can all make a skilled action fail. Limb apraxia is diagnosed when the pattern is disproportionate to those accompanying deficits and is expressed across appropriately chosen tasks.

Those tasks do not all measure the same thing. An examiner may ask a patient to:

• imitate an unfamiliar hand or finger posture;
• produce a familiar communicative gesture, such as waving goodbye;
• pantomime using a named tool when no tool is present;
• demonstrate how to use an actual object;
• select the appropriate tool for an object;
• organize several actions into a practical sequence.

A patient can fail one and succeed at another. Imitating a meaningless posture depends strongly on converting a seen body configuration into one’s own joint configuration. Pantomime requires generating an action without the object’s shape, weight, contact surfaces, or mechanical consequences. Actual tool use supplies those constraints, but it can still fail when knowledge of tool–object relations or the organization of the action is disturbed. Pantomime and actual use are therefore not simply voluntary and automatic versions of one stored motor program [@hoeren2014praxis; @goldenbergspatt2009tool].

The traditional labels ideomotor apraxia and ideational apraxia remain common. Ideomotor has generally referred to spatial and temporal errors in producing a familiar gesture, especially to command or imitation. Ideational has been used for impaired tool concepts, misuse of objects, or disorganization of multistep action. The categories are historically useful but do not divide patients into two natural, nonoverlapping diseases. Modern assessment increasingly describes the particular tasks, errors, and lesion connections rather than assuming that one broken box explains them all [@hoeren2014praxis; @buxbaum2014tool].

Limb apraxia has a strong left-hemisphere association, including left inferior parietal, lateral temporal, premotor, and frontal regions and the white-matter pathways connecting them. The anatomy is distributed, and different patterns of disconnection can yield different deficits. Lesion studies support separable profiles involving gesture production and tool-related action rather than one unitary store of skilled movements [@hoeren2014praxis; @buxbaum2014tool; @goldenbergspatt2009tool].

Context can help some patients. A real object supplies visual, tactile, and mechanical information that pantomime removes. That does not make contextual action universally preserved. Patients can misuse real tools, choose the wrong action, or fail to organize a multistep task. The often-repeated opposition between defective action “to command” and intact automatic action in daily life captures some cases but cannot define apraxia as a whole.

The syndrome is best understood as a disturbance at a different level from paralysis or optic ataxia. The problem is not merely getting force to the limb, and it is not limited to updating a target relative to the hand. It concerns learned relations among gestures, objects, body configurations, and larger actions.

Clinical problem	Principal failure revealed by the task	What may remain possible	Important qualification
Weakness or paralysis	Generating force or transmitting effective descending and peripheral output	The target and appropriate action may still be understood	Profiles depend on lesion level, pathway, and time since injury
Optic ataxia	Using current visual information to guide the hand and update the reach	Object recognition, elementary movement, and some centrally viewed or nonvisual actions	Effects depend on visual field, hand, fixation, delay, and feedback
Limb apraxia	Organizing learned gestures or tool-related actions beyond elementary weakness	Some movements, gestures, or contextual actions	Imitation, pantomime, actual tool use, and sequence production can dissociate
Cerebellar ataxia	Calibrating timing, prediction, multijoint coordination, and correction	Strength and the intended outcome may be relatively preserved	The exact profile depends on cerebellar territory and task

32.9 How visually guided action is studied

A reach appears simple enough to measure, but conclusions depend on what is recorded and how the task is perturbed.

Kinematic recording tracks the position, velocity, acceleration, and orientation of the eyes, head, trunk, arm, hand, and digits. It can reveal when gaze reaches the target, when the wrist begins to rotate, when grip aperture peaks, and when a correction changes the trajectory. Kinematics describe the movement that occurred. They do not by themselves reveal which neural pathway caused it or which variables the nervous system controlled.

Force sensors and electromyography add information about grip force, load force, muscle recruitment, and responses to slip or perturbation. Similar hand trajectories can be produced by different muscle patterns, and similar muscle activity can have different mechanical effects from different postures. Force, movement, and neural activity must therefore be related rather than treated as interchangeable measures.

Target jumps, obstacle changes, and mechanical perturbations test online control. Moving the target after reach onset asks whether the system can update the action. Applying a force to the arm asks how proprioceptive and tactile evidence is used. Altering visual feedback can create disagreement among the senses. The timing of the perturbation matters: a change before movement, early in transport, near contact, or after grasp tests different portions of the loop.

Single-neuron recording in nonhuman primates can relate activity to retinal target location, gaze, hand position, intended grip, movement, and feedback. The variables are often correlated. A neuron that fires during rightward reaches may covary with target position, shoulder torque, gaze, or a learned choice. Carefully factorial tasks can separate some relationships, but no experiment removes every alternative description.

Reversible inactivation and stimulation add causal leverage. Inactivating a parietal region can reveal whether it is necessary for a particular correction or grasp under the tested conditions. Stimulation can show that a network has access to an output. As in the preceding chapter, the result depends on location, intensity, timing, behavioral state, and the fibers affected.

Functional imaging identifies distributed human networks and can compare reach, grasp, eye movement, tool use, and observation. Multivoxel patterns can contain information about target or action. Decoding that information does not prove that the brain uses the decoded variable in the same form, and an activated region need not be necessary. Imaging is strongest when its spatial patterns converge with perturbation, anatomy, physiology, and lesion evidence.

Transcranial magnetic stimulation can disrupt processing within a limited time window. A pulse over posterior parietal cortex shortly after a target jump asks a different question from a pulse delivered during initial preparation. The spatial reach of stimulation and network propagation prevent it from becoming a perfect virtual lesion, but temporal specificity is a major advantage.

Clinical lesions reveal what an intact system cannot easily show. Optic ataxia separates seeing an object from using vision to guide the limb; apraxia separates elementary movement from learned action organization. Natural lesions, however, cross cortical fields and white matter, and recovery changes the network. A memorable patient is not automatically a pure lesion experiment.

The most persuasive account comes from convergence. A coordinate-frame hypothesis is stronger when neural tuning, behavior under controlled gaze and hand positions, causal perturbation, and lesion patterns point in the same direction. A single activation, decoding result, or dramatic dissociation rarely settles the mechanism by itself.

32.10 From correction to prediction

Online feedback solves much of the problem, but it creates another. Visual, proprioceptive, and tactile signals arrive after delays. By the time visual cortex receives evidence that the hand has drifted, the hand has moved farther. By the time a corrective command reaches muscle and changes force, the body and target may have changed again.

Slow actions can tolerate more delayed correction. Fast actions cannot simply wait. The controller needs an estimate of the body’s current state, not only a record of its recent state. Motor-related signals can help anticipate the sensory consequences of an action; recent sensory evidence can correct the estimate; repeated errors can recalibrate the relation between command, body, and outcome [@flanaganwing1997internal].

Prediction does not replace feedback. A prediction is useful because later sensory evidence can test it. Feedback is useful because it updates a system that is already acting on an estimate. The distinction between feedforward and feedback control therefore describes different contributions within one recurrent process, not two independent modes that take turns.

This chapter has shown why such estimation is necessary. Orienting must remain coordinated with reaching. Several reference frames must be related without waiting for one perfect map. Reach and grasp evolve together. A target can move after the hand begins. Contact can reveal an unexpected load or an unstable grip. A controller that used only delayed sensory error would always be correcting a body that no longer exists in quite the same state.

The next chapter turns to the predicting machine that is especially important for calibrating these relations: the cerebellum. Its contribution will not be a finished movement plan added on top of the systems described here. It will be the continual improvement of prediction and correction within the same embodied control hierarchy.

TipA note on what this chapter is sure of, and what it isn’t

We are confident that:

• The optic tectum or superior colliculus is part of an evolutionarily conserved system for relating mapped sensory events to orienting of the eyes, head, and body.
• Primate visually guided reaching depends on distributed, recurrent interactions among occipital, posterior parietal, premotor, motor, subcortical, brainstem, spinal, and cerebellar systems.
• Retinal target location is insufficient to specify a reach. Eye position, head and trunk orientation, hand position, posture, and the intended effector also matter.
• Posterior parietal and frontal populations use mixed and partially transformed reference frames rather than one uniform final coordinate system.
• Dorsomedial parietofrontal territories have a relative bias toward reach transport, and more lateral anterior-intraparietal and ventral-premotor territories have a relative bias toward grasp formation. The networks overlap and interact.
• Visual target displacements can alter an ongoing reach rapidly. Tactile and proprioceptive evidence continue to modify the action before and after contact.
• Optic ataxia is a genuine disorder of visually guided action that cannot be reduced to weakness or poor object recognition, although its expression depends strongly on task and lesion anatomy.
• Limb apraxia is a genuine disorder of learned skilled action beyond elementary weakness, sensory loss, or incomprehension. Its task profiles are heterogeneous.

We have good reason to think that:

• Evolution expanded flexible cortical control of the limb and hand while preserving older tectal, brainstem, and spinal controllers.
• Gain fields and other mixed population codes help relate retinal events to gaze, body, and hand without requiring one explicit master map.
• Peripersonal visual–tactile representations provide a flexible interface for grasping, avoidance, defense, and anticipated contact near the body.
• Several possible actions can be specified before one is selected, with current goals and selection systems biasing parietofrontal activity.
• Neural activity during a component movement can depend on the larger action sequence in which that movement is embedded.
• The classic contrast between D.F. and optic ataxia reveals different vulnerabilities of visual judgment and online action, but not fully independent visual systems.

We remain genuinely unsure about:

• How many reference frames are maintained during ordinary behavior and how their weighting changes from initial orientation through contact.
• How labor is divided among posterior parietal territories during visual, proprioceptive, and tactile correction.
• When rapid correction is accompanied by conscious awareness and what failure to report a perturbation reveals about the underlying pathway.
• How literally the brain implements competition among simultaneously available actions, and how that competition interacts with basal ganglia, frontal cortex, and value.
• Which current taxonomy best captures the different disorders grouped under limb apraxia.
• How sharply perception-for-report and vision-for-action can be separated once fixation, delay, haptic feedback, lesion extent, and task demands are controlled.