Prof. Judy Fan: Cognitive Tools for Making the Invisible Visible

Summary

Key Points

• —Judy Fan's research explores how humans create and leverage cognitive tools, like the number line and rectangular coordinates, to facilitate thought, discovery, and innovation. 4:25
• —Human learning and discovery are deeply intertwined with technologies for making the invisible visible, exemplified by historical uses of visual abstraction in scientific illustrations and diagrams. 6:55
• —Visual abstraction allows humans to communicate what they see and know by highlighting relevant information, ranging from faithful depictions to schematic representations. 8:51
• —Studies on freehand drawing demonstrate that people adapt their level of visual abstraction based on communicative context, producing more detailed drawings when specificity is needed and sparser ones for category-level communication. 19:14
• —Research on visual explanations reveals that people prioritize mechanistic information over visual appearance when illustrating how something works, suggesting a dissociable rather than cumulative approach to visual communication. 22:42
• —Current artificial intelligence systems, particularly vision-language models, still exhibit a significant human-model gap in both understanding and generating human-like visual abstractions, especially under tight production budgets. 34:50
• —Data visualization is a powerful cognitive tool that enables statistical reasoning, helping people resolve patterns and phenomena too large, noisy, or slow to perceive directly. 38:53
• —Benchmarking studies show that while advanced multi-modal AI systems can perform graph-based reasoning, they consistently fall short of human performance and do not produce human-like error patterns. 42:44
• —People, including non-experts, are sensitive to the features of plots that make them suitable for answering specific questions, indicating an intuitive understanding of effective data visualization. 51:21
• —The overarching goal of this research is to develop psychological theories explaining how humans use and innovate cognitive technologies to improve education and continually reimagine and better the world. 56:22