Synesthesia: from cross-modal to modality-free learning and knowledge
- Submitting institution
-
University of Portsmouth
- Unit of assessment
- 32 - Art and Design: History, Practice and Theory
- Output identifier
- 7149504
- Type
- D - Journal article
- DOI
-
10.1162/LEON_a_00937
- Title of journal
- Leonardo
- Article number
- 0
- First page
- 48
- Volume
- 48
- Issue
- 1
- ISSN
- 0024-094X
- Open access status
- Out of scope for open access requirements
- Month of publication
- October
- Year of publication
- 2014
- URL
-
-
- Supplementary information
-
-
- Request cross-referral to
- -
- Output has been delayed by COVID-19
- No
- COVID-19 affected output statement
- -
- Forensic science
- No
- Criminology
- No
- Interdisciplinary
- No
- Number of additional authors
-
2
- Research group(s)
-
-
- Proposed double-weighted
- No
- Reserve for an output with double weighting
- No
- Additional information
- This article draws together various research projects the authors had been involved in over five years about how people learn, interact, and make sense of the world and the new
possibilities that emerge in digital and networked media. The thread running through all of this work is the idea that learning is holistic and integrated across cognition, perception, embodiment, context, epistemology, affect and ontology. The article explores how synaesthesia, embodiment, transcription, abstraction and engagement can inform our view of learning.
Gumtau’s contribution to the article is on discussions of the underlying theory, in particular the two case studies on Montessori learning and the MEDIATE environment (Multisensory Environment Design for and Interface between Autistic and Typical Expressiveness). The latter is a multisensory environment for so-called low-functioning children on the autistic spectrum with little or no verbal skills. It is a collaboration of five teams across Europe, involving designers, programmers, and psychologists. Gumtau contributed design research (interaction models and tactile interfaces).
The MEDIATE environment contains complex adaptive, intelligent systems, meaning the child is an actor in a responsive network (which resonates with Montessori’s concept of the ‘prepared environment’). The child is free to interact with the automated system and experience their own agency through creative play. This overall ecology of learning benefits from automation to some degree, which children with autism seem to relate to better than other humans, but also from a focus on allowing perception, synaesthesia and embodiment be at the centre of the interaction – by avoiding symbol based communication, for example, and focusing on a semantic, intuitive mapping between the senses. One of the user evaluations showed that the child was able to communicate with their parents nonverbally by selecting their favourite colour in MEDIATE.
- Author contribution statement
- -
- Non-English
- No
- English abstract
- -