Last month I attended the Visual Learning Conference at the Visual Learning Lab, Budapest. The slides of my presentation are already up here, but I thought it would be worth sharing some of the notes I took while I was there.
The Budapest University of Technology & Economics was originally founded in 1782. It has 24,000 students and has produced three Nobel laureates, including the inventor of holography. Erno Rubik is a graduate of the university. It is a research university with a focus on innovation of the infrastructure supporting scientific research. Recent projects include MASAT-1, a satellite built by graduate and PhD students and still in orbit after 300 days; Odoo, a solar powered pre-fabricated house; and Hand-in-scan, a device which measures the hand-washing quality of surgeons with the aim of preventing nosocomial infection.
The theme this year is Communication, Cognition & Curriculum. The Visual Learning Conference is now in its third year, and continues to attract an interesting range of academics who appear to interpret the novelty of visual learning in different ways. In the spirit of ‘openness’ I’m publishing my notes from the event here. (These are my interpretations of what was presented, so please contact me if there are inaccuracies or you’d prefer not to have a summary of work in progress blogged.)
KATZ, James E. – HALPERN, Daniel:
Is a Tour Worth a Thousand Clicks? Visual Information Processing as Affected by Spatial Abilities and Individual Differences in a Museum Environment
Almost all museums now have a web presence, but they are typically visited for operational information rather than to make collections available online. There are many ideas about how best to promote audience engagement.
Widening participation: virtual visits to museums may take the form of panoramic 3D environments and virtual galleries. There are lots of different views about how best to structure these experiences, which it should be noted require additional technological, spatial and navigating skills from users. Scilia, Ruiz & Mumuera (2005) argue that more processing resources are required of virtual visitors, who have additional information to manage. Studies (Styliani et al. 2009) indicate that virtual visitors are more likely to become physical visitors.
Spatial reasoning ability has been cited as a good indicator of HCI skills. Google Art publishes 30,000 artworks online and, in recognition of this, offers both 2D and 3D navigation. Katz & Halpern put the thesis to the test in order to explore whether those with higher spatial reasoning skills could more easily navigate these exhibitions. 337 US students aged 18 to 33 took part, split into 2D and 3D groups with data gathered through online surveys. The Santa Barbara scale was used to assess spatial and navigational skills and preferences (Hegarty et al, 2002). They also took assessments in spatial orientation skills (Hegarty & Waller, 2004). Information was also gathered about their background and ICT skills.
Those in the 3D tour felt more like they were taking part in a virtual tour but also felt that their experience was more like a real visit when they had higher spatial reasoning skills. However, there were some unexpected effects. Those with low spatial reasoning skills enjoyed the virtual tour much more than those with higher skills of this kind. Hierarchical regression models were used to isolate variables. The only significant factor influencing involvement was sense of direction: the stronger the spatial reasoning, the less the sense of involvement. Those with high spatial reasoning seem to like virtual tours less – a paradoxical finding. (This may be explained through expectancy-value theory (1982)). But it seems though museums and other institutions need to rethink their approach if they want to attract spatially or visually literate audiences. There is a great deal of potential for considering visual cognition within content and curriculum.
The Role & Possibilities of Visual Skills in a Technology-Driven Architecture
Architects are 50% engineer and 50% artist; they learn how to draw and how to use CAD. They must mediate on forms which are both contemporary and traditional. Where do visual skills fit? A fundamental aim of architecture is to establish artificial environments which can support human activities. It has always utilised new technologies, but in the last century there has been massive technological innovation while human natures remains unchanged, resulting in mass-production of poor quality environments. Efficient information is essential to progress in this respect. We need to understand the expectations of all relevant persons in order to find physical solutions to real needs.
Recent changes to the fundamentals of architectural design have thrown into question the assumption that we should always be making the biggest spaces with the fewest materials using the most recent technologies. While society still expects positive results of this kind from architecture, the over-riding factor should be the function of the building. Successful examples of recent architecture are characterised by effective function which is often not obvious; they correlate with the place and culture around them.
Epistemic Functions of Pictures. Some Conceptual Preliminaries
Picture fulfill crucial social functions, but are not typically thought of as fulfilling a scientific function. Pictures perform an obvious epistemic function within a scientific context – such as empirical evidence of phenomena which are not accessible to our natural senses. The interpretation of such images remains dependent on certain philosophical and epistemic assumptions. As imaging processes become increasingly complex and technical, the more interpretation is required, and the more the possibility of error increases (e.g. ultrasound). Pictures are epistemologically relevant when they help us to justify a particular claim by providing support for it – even though arguments are usually thought of as expressions of technical language. They can illustrate arguments, or act as arguments themselves. Moreover, they can be more accessible than arguments in verbal form (consider, for example, geometrical proofs). The influential works of Kuhn show that frameworks for scientific justification deeply influence the development of scientific theory. But the basis for all scientific terms is metaphorical, and can be thought of as including a visual dimension. But such models contain only structural resemblances, and not justifications themselves. They can nonetheless provide useful orientation for scientific work.
Since the Platonic criticism of the image they have often been represented negatively within the philosophical tradition. Many philosophical functions – such as negation – are difficult to visualize.
We often use pictures to explain or define concepts (e.g. drawing a triangle). Since antiquity, knowledge has been understood as true and justified opinion: this usually implies community and verbal communication. Do pictures have propositional content? If so, the conditions for justification may be different for pictorial and verbal content. Perhaps pictures are closer to predicates… but a truth value can only be applied when a predicator is connected to a nominator. The nominator cannot be arrived at through a picture alone; we need to associate or refer the picture to that which it applies to. Context is thus particularly important for the images while for verbal language we can use proper nouns to add specificity. It can also be difficult to separate out the content of the picture from the communicative or rhetorical aim.
Pictures have many epistemic functions, but the quality is highly dependent on contextualization. But can pictures correspond to sentences, or to words? After all, words are not truth-apt. [Nyiri contends that the moving image is equivalent to the sentence and thus is truth-apt where the still image is not.]
The Riddle of Images Revisited
Mitchell (1987), Belting (2005) – images are not there. They require us to see something that is specifically not there… it conveys this through a triad of the medium, the pictorial subject and the pictorial image. Cartesian representationalism sees such images as the medium through which the physical world is conveyed to the mental world. He makes extensive use of visual metaphor; vision is arguably a paradigmatic Cartesian motif. Bergson, by contrast, argues that we must understand the nature of dualism in terms of time rather than space. The spatial starting point is a mistake because the real problem is that of causation. ‘Representation’ is a key category for Bergson, who defines image as less than that which the realist calls a ‘thing’; it does not have full ontological status: “representation is always virtual, neutralized at the point at which it might become actual”.
Visual Learning through Games: The Case of GPS-Based Treasure Hunting
The context for this research is the moral panic about people being lost in social media while they lose meaningful connections with their peers. Is this really the case, or do digital media augment our surroundings? Hargittai argues for the latter and uses geo-caching and GPS based games to argue this.
The goal of these games is to physically go outside and do something. In the case of geocaching, it is to find something. There have been something like 2 million geocaches since 2000, there are 5 million people who have taken part. They often find pill bottles, bison tubes or plastic containers. They often contain paper logs which are signed and returned, or QR codes to a website where you can register your discovery. It is primarily done for fun, but there are other benefits, including: map-reading, navigation, visiting new places, and looking at surroundings more closely and carefully. These behaviours are often carried across outside of the game.
KIMBLE, James J. – GOODNOW, Trischa:
Metaphor, Narrative, and the Visual:
On the Role of Cognitive Possibility in Propaganda Appeals
A General Motors propaganda poster from 1942 was displayed. The image and caption is an appeal that would have made sense among its blue-collar audience, who may not have had strong literary skills. It refers to previous events (Pearl Harbour) and implies a future course of action. Perhaps less obvious are the ways in which the viewer’s own presence is implied (“Let’s Go Everybody!”).
Based on the study of several hundred similar posters, it is contended that many image-texts operate in a similar fashion: combining metaphor and narrative.
‘Rhetoric of Possibility’: Pulokis [?] (1982) suggests that the Sophists saw communication as rebellious and retaliatory. The world seen this was is incomplete, and requires action. Rhetoric invokes a sort of telos: that which can – or might – be. Narratives are the most obvious locus for the rhetoric of possibility (e.g. the American Dream). Metaphor is less obvious, but also powerful (e.g. ‘a civilian is a soldier’). We are encouraged to see ourselves and/or the world differently. This possibility can be seen as ontological or ‘actuational’ when it leads to action.
‘Visual Moment’: cognitive possibility can be engendered by images (n.b. magazine/poster advertising). War posters are a particularly important locus for this effect: they are cheap and easy to make; easily consumed (even for the illiterate); and ubiquitous. They could nonetheless be extremely moving. They used both metaphoric and narrative impact to present a new possibility.
We have overarching or ‘master’ narratives that run throughout our culture. They provide a global meaning, whereas actions are localised. But as new metaphors are introduced they and perpetuate these narratives.
Visionary Rhetoric: Teaching Imagistic Communication
The contention here is that we can teach rhetoric through imagistic communication. Aczel defines the ‘rhetoric of possibility’ as ‘visionary rhetoric’ since it involves envisioning, looking into the future. There is a kind of religious connotation: one must have a kind of faith in a certain vision of the future. Rhetoric involves a certain kind of competence in terms of promoting a particular vision of the world.
- Vision – (above)
- Space – we live within space which is rhetorically constructed
- Procedure – procedural intelligence
- Sound – patterns in the aural world
- Rhetoric as ingenium (Graszi ?) – the primal way through which we come to know the world, a precursor to rational experience. Two languages: one is rational, dialectical, demonstrative and without emotion; the other is rhetorical, immediate, illuminating, sensual, imaginative and empathic.
- Symbolic convergence theory (Borman; Burke): fantasy themes (not metaphorical, nor metaphors themselves), cues and rhetorical vision. The meaning of a message is not in the speaker, nor the medium, but in the message itself and the way that this relates to a symbolic reality. (But what if images = metaphors?!)
- Imagism (a recent movement in British poetry): every word should have an intended effect; truth is less important than intended effect; the image is the message itself.
- Metaphor as effect (Sopbury & Dillard ?): metaphors are more effective than literal speech in terms of engendering a change in attitudes. When it comes to figurative comparisons, less is more: metaphors disturb our imagination. Novel metaphors are associated with greater impact than those which are already familiar. They also tend to be more effective in the aural than the written form.
- Rhetoric is not simply a technique for influence; it has an originary connection with the visual.
- We have to teach that the meaning is in the message when teaching imagistic communication and visual rhetoric.
- In the digital age we communicate with images and artifacts rather than other individuals. (What could be a metaphor for Facebook?)
- We should raise consciousness of our experiences and how we gather experiences (vision/non-vision).
- We should teach ourselves to retain a kind of distance from texts, even those which we have written ourselves.
The Tacit Image: Michael Polányi Revisited
Polanyi was a medical doctor who became a chemist, and later a social scientist and philosopher. He produced a paper in 1970 (at the end of his career) about painting. He identified three meanings:
- Canvas coated by paint with the brush of the artist
- Picture depicts the surroundings of the painter
- Cultural, moral, aesthetic, financial value
Polanyi argued that “all knowledge is either tacit or rooted in tacit knowledge”. Scientific knowledge integrates compatible elements to create new (empirical) knowledge. Art integrates incompatible elements.
Pozzo’s church frescos employ a 3D perspective from a certain point and there is a representational perspective from various locations. Distance is significant; it has an influence on the way that the painting is viewed. Velasquez’s Maids of Honour (1656) can also be broken down into constituent paintings of the various figures it portrays.
Gombrich (1960) held that paintings were either whole or broken down into individual marks of paint. Polanyi argued against this dichotomy on the grounds that explanation is based on tacit knowledge: “we know more than we can tell”. Consider facial recognition: we recognize faces instantly even though they may share features with countless others. This may be considered to be a feature of tacit knowledge. The core of Polanyi’s argument is thus the Gestalt psychology of the 1920s. According to this theory, humans recognize entire shapes and not the sum of constituent parts. Recognition is holistic. The Figure/Ground principle states that when we look at a picture everything that is not the focus of the picture is the ‘ground’. (Comparison drawn with a blind man sensing the curb (focal awareness) through his cane (subsidiary awareness).) Gestalt refers to wholes but there is still a focal/subsidiary aspect to our attention.
Although Kuhn includes few references in his Structure of Scientific Revolutions, many of them are to Polanyi!
Images in Conservative Education
Conservatism is an indispensible but seemingly paradoxical mission. Its paradoxes can be dissolved by allotting a proper (philosophical or educational) role to images. Nyiri’s central claim is that thought is primordially pictorial, but that even below this there is a motor-dimension.
The conservative conception of knowledge: post-Hayek, this is often taken to be tacit, local and practical. It is embodied in concrete human beings; and not even in their minds as representation. Conservatism treats knowledge as decentralized and local (Hayek). Correspondingly, conservatives believe education should be decentralized and local. A postmodern conservative tries to embrace the digital world, but is reluctant to leave the world of hard copy behind.
Images and conservatism: political regimes have used images & visual thinking to support themselves. Mechanical images (photos) preserve details which were not necessarily intended to be included by the creator, but which may be of interest to future generations. Images are thus conservative in the sense that conservatism is an attitude of common-sense realism, which suggests that what images depict is ‘real’. Conservatism is haunted by paradoxes: it is concerned with traditions, but traditions alone have no binding force. In the sense that it is based in tradition, conservatism has no basis in the (post-) modern world. The image comes to take the place of traditions. We now have the means to produce and convey argument; images convey the world as it is structured into the visual. It is the realism of images that can take the role of ‘convincer’ that once belonged to tradition. Postmodern conservatism is cursed by the fact that we live in an age of basic uncertainty, and it relies on images to replace tradition.
What Does the Cinematic Background Demonstrate? Depth of Field Thinking in Bazin
Bergson: “The objects which surround my body reflect its possible action upon them”.
Mersch (Visual Epistemology): “there is a genuine ‘logic of showing’ which differs from the logical form of language…”
Bazin’s neo-realism: the aesthetic qualities of photography are in its power to lay bare reality… the impassive lens strips its object of preconceptions. The idea of education here is not about changing one’s view, but about restoring integrity to things in general, to experience the real anew and recreated.
- Neither the observer nor the film are representations of reality: they are in reality
- The picture frame polarizes space inwards, but the cinema screen is centrifugal, outward-facing
- Depth of cinematic field brings uncertainty, openness, democracy
- Neo-realism is the open and honest rendering of a fictional world; the viewer is educated into a filmic world: “les yeux dans les yeux”
- Bazin argues that we need a kind of naivety or learned idiocy
A reversal of Plato: Plato (Republic 523-4) argues that we should not trust the senses because they are fallible. Delueze argues the opposite: that we should return to the immanence of sensation.
An Education of Attention: The Perception of Change in Bergson and Performance
An education of attention: learning as fundamentally embodied activities, less to do with improving information-processing and more to do with the capacity to encounter ‘change’, qualitative difference, or the genuinely new.
A contemporary “crisis of attention” in modernity? Steinbock (2004:1) Crary (2001:35) Terranova (2012:1). For Crary, ADD acts as a reminder of the “durability of attention as a normative category of institutional power”. Modern capitalism must distract as part of a strategy of control but also manage distraction.
Bergson: “Time is invention or it is nothing at all”. Bergson is interested in a notion of perception that is broader and more inclusive, opening us up to the perception of change and movement. Bergson almost goes to the point of pathologizing our everyday forms of attention. Philosophy can lead us to an improved perception by means of a displacement of our perception (1992:137). Bergson’s sense of completeness is not directed towards a transcendent experience of the world, but new attunement to change.
We become habitually inattentive, routinized. We are given to distracted doing or detached spectatorship. Kaprow’s ‘Activities’ (1972) is a series of exercises which are intended to direct attention towards modes of attention. Many of these relate to processes of material change (e.g. ice melting) and push our attention away from clock time and toward other forms of temporal experience. Visual art can be understood as a way of encouraging us to pay extra attention to the visual, music to the aural. Cull’s general thesis is that exercises like those proposed by Kaprow can direct us towards Bergson’s state of attention; thus they have a pedagogical purpose. (We can best learn to play by example.)
Viewers as Performers: Dynamic Approaches to Gesture Observation and Gestural Meaning-making
Lapaire began with a video showing a headless figure performing a number of gestures with their hands. This was inspired by Hands (Burrows, 1996). We perform visible bodily actions occur in close temporal synchrony with speech (McNeill 1992, 2005). For the listener, the experience is mental and physical, concrete and abstract. Gesture space is physical, conceptual, narrative, textual, grammatical, and interactional.
Visualizing Grammar: didactic anchoring (here and now), approximation, concession, totalization, futurity, duration, conceptualization, centrality, etc.
These are meaning-making or ‘enactive’ gestures (Streek, 2009) or fabrication activities (Kendon, 2004). We can thus show the centrality of the body for language use. Creative and dynamic approaches to gesture observation can be developed which are based in visual stimulation. The forms and dimensions of movement may be explored from a combined artistic and academic perspective (e.g. choreography) which conflate the roles of performer, audience, analyst, and communicator.
Lapaire engages his linguistics students with gesture projects in order to develop their understanding of motor gestures for linguistic communication. They are encouraged to focus on form and manner of motion, constraints and variables, and exploring the physical dimensions of gesture which typically remain unconscious. Word-gesture correspondence is established; kinetic activity is associated with verbal activity.
Visual Literacy: the Stepchild in Reading Comprehension Teaching and Testing
Reading comprehension is not typically approached in terms of visual literacy. In the knowledge economy there are ever-more fields of knowledge, skills, literacies and competencies. There is a significant literacy issue in the developing world. In the developed world, there is considerable functional illiteracy (BBC News, 2000 – 1 in 5 adults are ‘illiterate’). There are many immigrants who lack a working knowledge of their target language. For new literacies, we need to promote the skills and strategies necessary to successfully adapt to the changing world of ICT.
The separation of textual and visual literacies contributes to the problematic status quo. Research (Burmark, “Getting the picture” suggests that people are able to process visual information 60,000 times more quickly than textual information.
Visual literacy was first proposed as a concept in 1968. It is the ability to recognize and understand ideas conveyed through visible actions or images. The use of visual images to communicate is probably as old as mankind (n.b. cave paintings). We now live in a multimedia world, and tend to use visuals when proper decoding is of key importance: signs, manuals, health products, warnings, maps, etc. Literacies tend to be taught and assessed independently; this may have consequences for future generations as the need for visual and other literacies increases in importance.
Reading comprehension testing is a recurring part of education, tested at different levels and different ages. There have been many studies which aimed to identify the factors that influence reading performance (background music, gender, etc.) but the affective role of the visual has been overlooked. Images organize information into different levels of relative importance. Information is made easier to understand when it is in a visual hierarchy (Bradley, 2009).
Rakes, Rakes & Smith (1995) found that visual tests provide excellent forms of evaluation “that are closely matched to instructional objectives”. It has also been found that visual images help to decode and schematize information. Viegas & Wattenberg (2011) argue that digital natives are more visually literate. [It’s worth noting that there is a body of new research which challenges the ‘digital natives’ hypothesis.]
JÓZSA, Eszter – HERCEGFI, Károly – KOMLÓDI, Anita – KÖLES, Máté:
Eye Tracking Analysis of Learnability of Various User Interfaces
INTegrated Evaluation and Research Facilities for Assessing Computer-users Efficiency (INTERFACE): an attempt to go beyond Neilsen’s heuristics. It involved a workstation which tracks eye movement, screen content, mouse & keystrokes, user pulse, skin galvanization, pupil dilation; and video records user behavior. Users were given the task of buying some tickets online and using an eBay style auction site (Aphrod.hu).
- Buttons were not always easily identified, or were thought to be advertisements and thus not clicked on.
- Category buttons were often eschewed in favour of drop-down menus
GOETHE, Norma B.:
Reasoning about Form, Shape & Structure
Philosophy of mathematics: it has been argued that there are notions which structure mathematics: equality, ordering, duality, combination, induction, contradiction, axiomaticization, etc. These can be expressed as propositions (n.b. the influence of Frege) but in doing so they are often considerably distorted.
In foundational mathematics the notion of truth is both an idealization of mathematical proofs and of logical proofs. Predicate logic is often thought to ‘underlie’ mathematical proof. Axiomatic proofs and proofs by natural deduction are held to provide the essence or final justification of mathematical proof. Formal proofs attempt to recreate as accurately as possible the process of mathematical reasoning, but axiomatic proofs themselves are almost never found in practice.
If we observe the practice of mathematicians, proofs serve the function of correcting errors and providing more detail or better scholarship. They do not necessarily improve results; their effect is to encourage mathematicians to think more closely about their work.
‘Child Art’ of the Z Generation – A Multimedia Model of Visual Skills Development
The digital and the real: complementary or opposing? They are equally valid methods of image production. Digital modes of production need not result in loss of traditional skills.
- Technical drawing is affected by psychomotor skills, previous learning & medium
- Motive choice is affected by perception of ability
- The role of creativity is unquestionable
- Affected by tools, not ability
- Motive choice less affected by ability
- Still relies on creativity
Psychogalaxy.hu provides a range of tests for digital painting with a mouse, including collaborative drawing. It uses standard psychometric evaluations to relate the images that are created to personality type and other profiling.
Visual culture teaches a range of skills:
- Visual perception; observation, appreciation, evaluation
- Creation in 2D & 3D; manipulation, abstraction, symbolisation, modality change
- Spatial abilities
- Design & Construction
The Potential Role of Visualization in the Vocational Training and Continuous Education of Adults
The role of perception in the learning process: most people tend to think we operate in the ‘real world’ when in fact we operate within a heavily filtered map of the world (Michael Caroll). From an NLP perspective, this filtering is a matter of sensory inputs being emphasized or de-emphasized at the level of experience. We all have different sensory maps (visual, auditory and kinesthetic) because our senses are all unique and change over time. Our values and beliefs also function as filters in this way. But linguistic filters are held to be especially important because they govern the way we share experience with others. When we communicate with another, in a manner of speaking we are communicating with their own internal reality (Boyley, NLP Loops internal/external experience).
People typically have an (unconscious) preference for visual, auditory or kinesthetic communication. People with the same preference often find it easier to communicate with each other.
Visual – supported through visual props & cues, ‘photographic memory’
Auditory – learn through repetition, reading out loud
Kinesthetic – learn by doing
The NLP communication model emphasizes the importance of internal representation. Learning through visualization is a matter of matching the sensory maps of teacher and learner without verbal reference, thus eliminating the possibility of audio filtering (comment on RF presentation). Technology can be used to support different pathways through learning materials which are suited to learners with different preferences for content delivery, resulting in more flexible teaching aids. It should be noted, however, that mechanical solutions can result in the loss of contact and human relationships. [presence research / embodiment?]
Paths and Traps in the Forest of the Digitization of Education
The title refers to a complex situation; the ‘paths’ are our discussions and the ‘traps’ are barriers and obstacles to innovation. The VLL is now three years old a journey into the ‘forest’. It is difficult to know which paths to follow: software, theory, practice, policy, research, hardware…
Science & Higher Education: a new professional community is developing around the VLL. The (Nottingham) VLL was a place to showcase innovation across the five faculties of the university. It officially came to an end in 2010 when funding was exhausted.
New ICT applications are making access to learning easier. The VLL has conducted experiments to prove that new tools can be effective for learning. Films can be used to teach by making ‘sentences’ of images; providing an extra layer of ‘grammar’ (e.g. Waltz with Bashir).
The extant volumes of conference proceedings comprise 22 papers of research and professional message.
Nyiri’s work in emphasizing the role (and ontological primacy) of the visual has been of central importance.