Abstract
The general setting for our work is to locate language perception and production within the broader context of brain mechanisms for action and perception in general, modeling brain function in terms of the competition and cooperation of schemas. Particular emphasis is placed on mirror neurons – neurons active both for execution of a certain class of actions and for recognition of a (possibly broader) class of similar actions. We build on the early VISIONS model of schema-based computer analysis of static scenes to present SemRep, a graphical representation of dynamic visual scenes designed to support the generation of varied descriptions of episodes. Mechanisms for parsing and production of sentences are currently being implemented within Template Construction Grammar (TCG), a new form of construction grammar distinguished by its use of SemRep to express semantics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Arbib, M.A.: Rana computatrix to human language: towards a computational neuroethology of language evolution. Philos. Transact. A Math. Phys. Eng. Sci. 361, 2345–2379 (2003)
Arbib, M.A., Liaw, J.-S.: Sensorimotor Transformations in the Worlds of Frogs and Robots. Artificial Intelligence 72, 53–79 (1995)
Arbib, M.A.: Perceptual structures and distributed motor control. In: Brooks, V.B. (ed.) Handbook of Physiology — The Nervous System II. Motor Control, pp. 1449–1480. American Physiological Society, Bethesda, MD (1981)
Draper, B.A., Collins, R.T., Brolio, J., Hanson, A.R., Riseman, E.M.: The schema system. International Journal of Computer Vision 2, 209–250 (1989)
Erman, L.D., Hayes-Roth, F., Lesser, V.R., Reddy, D.R.: The HEARSAY-II speech understanding system: Integrating knowledge to resolve uncertainty. Computing Surveys 12, 213–253 (1980)
Itti, L., Arbib, M.A.: Attention and the minimal subscene. In: Arbib, M.A. (ed.) Action to language via the mirror neuron system, pp. 289–346. Cambridge University Press, Cambridge (2006)
Fagg, A.H., Arbib, M.A.: Modeling parietal-premotor interactions in primate control of grasping. Neural Netw. 11, 1277–1303 (1998)
Gibson, J.J.: The ecological approach to visual perception. Houghton Mifflin, Boston (1979)
Bonaiuto, J., Rosta, E., Arbib, M.: Extending the mirror neuron system model, I: Audible actions and invisible grasps. Biol. Cybern. 96, 9–38 (2007)
Oztop, E., Arbib, M.A.: Schema design and implementation of the grasp-related mirror neuron system. Biol. Cybern. 87, 116–140 (2002)
Gershkoff-Stowe, L., Goldin-Meadow, S.: Is there a natural order for expressing semantic relations? Cognitive Psychology 45, 375–412 (2002)
Arbib, M.A., Rizzolatti, G.: Neural expectations: a possible evolutionary path from manual skills to language. Communication and Cognition 29, 393–424 (1997)
Arbib, M.A.: From Monkey-like Action Recognition to Human Language: An Evolutionary Framework for Neurolinguistics (with commentaries and author’s response). Behavioral and Brain Sciences 28, 105–167 (2005)
Arbib, M.A.: Broca’s Area in System Perspective: Language in the Context of Action-Oriented Perception. In: Grodzinsky, Y., Amunts, K. (eds.) Broca’s Region, pp. 153–168. Oxford University Press, Oxford (2006)
Saussure, F.: Cours de linguistique générale. In: Bally, C., Sechehaye, A., Riedlinger, A. (eds.) Payot Lausanne and Paris (1916) (English translation by W. Baskin, Course in General Linguistics, Fontana/Collins, Glasgow (1977))
Gallese, V., Goldman, A.: Mirror neurons and the simulation theory of mind-reading. Trends Cognit. Sci. 2, 493–501 (1998)
Chomsky, N.: Lectures on Government and Binding. Foris, Dordrecht (1981)
Croft, W., Cruse, D.A.: Cognitive Linguistics. Cambridge University Press, Cambridge (2005)
Goldberg, A.E.: Constructions: A new theoretical approach to language. Trends in Cognitive Science 7, 219–224 (2003)
Fillmore, C.J., Kay, P., O’Connor, M.K.: Regularity and idiomaticity in grammatical constructions: the case of let alone. Language & Cognitive Processes 64, 501–538 (1988)
De Beule, J., Steels, L.: Hierarchy in Fluid Construction Grammar. In: Furbach, U. (ed.) KI 2005. LNCS (LNAI), vol. 3698, pp. 1–15. Springer, Heidelberg (2005)
Roy, D.: Semiotic schemas: A framework for grounding language in action and perception. Artificial Intelligence 167, 170–205 (2005)
Steels, L.: Evolving grounded communication for robots. Trends in Cognitive Sciences 7, 308–312 (2003)
Cangelosi, A., Riga, T.: An Embodied Model for Sensorimotor Grounding and Grounding Transfer: Experiments With Epigenetic Robots. Cognitive Science 30, 673–689 (2006)
Dominey, P.F., Hoen, M.: Structure mapping and semantic integration in a construction-based neurolinguistic model of sentence processing. Cortex 42, 476–479 (2006)
Feldman, J., Narayanan, S.: Embodied meaning in a neural theory of language. Brain Lang. 89, 385–392 (2004)
Steels, L., Loetzsch, M.: Perspective Alignment in Spatial Language. In: Coventry, K.R., Tenbrink, T., Bateman, J.A. (eds.) Spatial Language and Dialogue, Oxford University Press, Oxford (2007)
Kosko, B.: Bidirectional associative memories. IEEE Transactions on Systems, Man and Cybernetics 18, 49–60 (1988)
Navalpakkam, V., Itti, I.: Modeling the influence of task on attention. Vision Research 45, 205–231 (2005)
Guazzelli, A., Corbacho, F.J., Bota, M., Arbib, M.A.: Affordances, Motivation, and the World Graph Theory. Adaptive Behavior 6, 435–471 (1998)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Arbib, M.A., Lee, J. (2007). Vision and Action in the Language-Ready Brain: From Mirror Neurons to SemRep. In: Mele, F., Ramella, G., Santillo, S., Ventriglia, F. (eds) Advances in Brain, Vision, and Artificial Intelligence. BVAI 2007. Lecture Notes in Computer Science, vol 4729. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-75555-5_11
Download citation
DOI: https://doi.org/10.1007/978-3-540-75555-5_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-75554-8
Online ISBN: 978-3-540-75555-5
eBook Packages: Computer ScienceComputer Science (R0)