Understanding complex concepts is necessary for gaining knowledge in academic fields and for making informed decisions, e.g., getting vaccinated or taking out a mortgage. 

Recent news about misinformation on diverse and high-stakes topics has underscored the importance of studying factors influencing concept formation. Much of concept learning occurs in classrooms in a second language (L2) for the world’s bilingual population, so the challenge in understanding new concepts is compounded by the language barrier in L2 reading comprehension. Although language is a dominant interface for accessing and storing concepts, pictorial and structural representations of knowledge moderate successful concept learning. Previous research using people’s first language (L1) has shown that knowledge presented with images result in better comprehension, while concept maps make knowledge structure more explicit and promote learning retention. The results are mixed in whether and how multimodal presentations support concept learning in non-native learners, perhaps because processing of L2 multimodal materials are susceptible to cross-cultural differences and affective variables. 

The proposed study will address these issues by comprehensively examining how concept learning is influenced by language proficiency and multimodal processing, and potentially mediated by affective factors. 

Experiment 1 will be an online study involving three sites (n = 200 each at Hong Kong, the Netherlands, and the US). Adult participants who are first-year university students will learn new concepts in English from a series of sentence definitions (Vasopressin is a hormone that alters kidney function.). Effects of language groups and multimodal presentations on concept learning will be compared while taking into account the corresponding changes in learning interest and perceived ease. The learning outcomes will be two levels of conceptual knowledge, i.e., meaning recognition and generation, immediately after learning and at one week delay. The international comparisons across native English speakers and L2 English participants with different L1 backgrounds will allow us to systematically compare language and cultural distances. 

In Experiment 2, we will recruit Chinese-English participants in Hong Kong (n = 60) to examine event-related potentials (ERP) responses to L2 multimodal concept learning. By comparing the neural changes within learners, the ERP data can extend behavioral findings to pinpoint the mechanisms of concept learning under different multimodal and language conditions. The results from this study will inform theories of bilingual word learning and guide practical instructional design in concept learning and knowledge dissemination. The study will have important implications for pedagogical development across educational sectors.