EXTENDING STORY READING FOR DEAF AND HARD-OF-HEARING CHILDREN: ARTIFICIAL INTELLIGENCE AS A MEDIATOR OF BILINGUAL LITERACY IN DONNA JO NAPOLI’S HANDS & HEARTS

Ioanna Kaliakatsou

Abstract


Shared story reading represents a fundamental context for children's literacy development; however, deaf and hard-of-hearing children may experience barriers when literacy practices are predominantly organized around spoken language. Contemporary perspectives in Deaf education emphasize bilingual approaches that recognize sign language as a linguistic foundation and written language as an additional pathway for participation and learning. This chapter explores the potential of artificial intelligence (AI) to mediate bilingual literacy through Donna Jo Napoli’s multimodal picture book Hands & Hearts. Drawing on theories of multimodality, Deaf bilingual education, and inclusive literacy, the chapter examines how AI-supported features can extend connections among American Sign Language (ASL), written language, visual representation, and shared story reading. It argues that AI should be understood not as a replacement for human communication or picture books, but as a mediating tool that expands opportunities for accessible, meaningful, and inclusive bilingual literacy experiences for deaf and hard-of-hearing children.


Keywords


deaf and hard-of-hearing children; multimodal picture books; artificial intelligence; inclusive literacy; shared story reading

Full Text:

PDF

References


Berke, M. (2013). Reading books with young deaf children: Strategies for mediating between American Sign Language and English. The Journal of Deaf Studies and Deaf Education, 18(3), 299–311. https://doi.org/10.1093/deafed/ent001

Bragg, D., Koller, O., Bellard, M., Berke, L., Boudreault, P., Braffort, A., Kearns, J., Palmer, M., Ploetz, T., & Vogler, C. (2019). Sign language recognition, generation, and translation: An interdisciplinary perspective. In Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) (pp. 651–653). Association for Computing Machinery. https://doi.org/10.1145/3308561.3353774

Bus, A. G., van IJzendoorn, M. H., & Pellegrini, A. D. (1995). Joint book reading makes for success in learning to read: A meta-analysis on intergenerational transmission of literacy. Review of Educational Research, 65(1), 1–21. https://doi.org/10.3102/00346543065001001

CAST. (2018). Universal Design for Learning Guidelines version 2.2. CAST. Retrieved from https://lincs.ed.gov/professional-development/resource-collections/profile-1050

Chamberlain, C., & Mayberry, R. I. (2008). American Sign Language syntactic and narrative comprehension in skilled and less skilled deaf readers: A test of the linguistic interdependence hypothesis. Applied Psycholinguistics, 29(3), 367–388. https://doi.org/10.1017/S014271640808017X

Dickinson, D. K., & Smith, M. W. (1994). Long-term effects of preschool teachers’ book readings on low-income children’s vocabulary and story comprehension. Reading Research Quarterly, 29(2), 104–122. https://doi.org/10.2307/747807

Grosjean, F. (2010). Bilingual: Life and reality. Harvard University Press. Retrieved from https://books.google.ro/books/about/Bilingual.html?id=XgRum7AWOoUC&redir_esc=y

Hall, W. C. (2017). What you don’t know can hurt you: The risk of language deprivation by impairing sign language development in deaf children. Maternal and Child Health Journal, 21, 961–965. https://doi.org/10.1007/s10995-017-2287-y

Hoffmeister, R. J., & Caldwell-Harris, C. L. (2014). Acquiring English as a second language via print: The task for deaf children. Cognition, 132(2), 229–242. https://doi.org/10.1016/j.cognition.2014.03.014

Humphries, T., Kushalnagar, P., Mathur, G., Napoli, D. J., Padden, C., Rathmann, C., & Smith, S. (2012). Language acquisition for deaf children: Reducing the harms of zero tolerance to the use of alternative approaches. Harm Reduction Journal, 9, Article 16. https://doi.org/10.1186/1477-7517-9-16

Jewitt, C., Bezemer, J., & O’Halloran, K. (2016). Introducing multimodality. Routledge. https://doi.org/10.4324/9781315638027

Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. Routledge. Retrieved from https://books.google.ro/books/about/Multimodality.html?id=ihTm_cI58JQC&redir_esc=y

Kusters, A., De Meulder, M., & O’Brien, D. (2017). Innovations in Deaf studies: Critically mapping the field. American Annals of the Deaf, 162(4), 285–297. https://doi.org/10.1353/aad.2017.0026

Ladd, P. (2003). Understanding Deaf culture: In search of Deafhood. Multilingual Matters. https://doi.org/10.21832/9781853595479

Lane, H. (1992). The mask of benevolence: Disabling the Deaf community. Alfred A. Knopf. Retrieved from https://books.google.ro/books/about/The_Mask_of_Benevolence.html?id=11FHAAAAMAAJ&redir_esc=y

Marschark, M., & Hauser, P. C. (2012). How Deaf children learn: What parents and teachers need to know. Oxford University Press. Retrieved from https://psycnet.apa.org/record/2011-25233-000

Mayberry, R. I. (2007). When timing is everything: Age of acquisition effects on second-language learning. In A. M. B. de Groot & J. F. Kroll (Eds.), Tutorials in bilingualism: Psycholinguistic perspectives (pp. 89–117). Lawrence Erlbaum Associates. Retrieved from https://doi.org/10.1017/S0142716407070294

Mayberry, R. I., & Kluender, R. (2018). Rethinking the critical period for language: New insights into an old question from American Sign Language. Bilingualism: Language and Cognition, 21(5), 938–944. https://doi.org/10.1017/S1366728917000724

Mayer, C. (2007). What really matters in the early literacy development of deaf children. Journal of Deaf Studies and Deaf Education, 12(4), 411–431. https://doi.org/10.1093/deafed/enm020

Mol, S. E., Bus, A. G., de Jong, M. T., & Smeets, D. J. H. (2008). Added value of dialogic parent–child book readings: A meta-analysis. Early Education and Development, 19*(1), 7–26. https://doi.org/10.1080/10409280701838603

Napoli, D. J., & Bates, A. J. (2014). Hands & Hearts: With 15 words in American Sign Language. Abrams Books for Young Readers. Retrieved from https://books.google.ro/books/about/Hands_Hearts.html?id=TXRmAgAAQBAJ&redir_esc=y

New London Group. (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66(1), 60–92. Retrieved from https://doi.org/10.17763/haer.66.1.17370n67v22j160u

Nikolajeva, M. (2013). Picturebooks and emotional literacy. The Reading Teacher, 67*(4), 249–254. https://doi.org/10.1002/TRTR.1229

Nikolajeva, M., & Scott, C. (2006). How picturebooks work. Routledge. Retrieved from https://books.google.ro/books/about/How_Picturebooks_Work.html?id=EbCfCLUOZD0C&redir_esc=y

Nikolaraizi, M., Vekiri, I., & Easterbrooks, S. R. (2013). Investigating deaf students’ use of visual multimedia resources in reading comprehension. American Annals of the Deaf, 157(5), 458–473. https://doi.org/10.1353/aad.2013.0007

Oatley, K. (2016). Fiction: Simulation of social worlds. Trends in Cognitive Sciences, 20(8), 618–628. https://doi.org/10.1016/j.tics.2016.06.002

Padden, C., & Humphries, T. (2005). Inside Deaf culture. Harvard University Press. Retrieved from https://books.google.ro/books/about/Inside_Deaf_Culture.html?id=2B4XWIFPgowC&redir_esc=y

Sandler, W., & Lillo-Martin, D. (2006). Sign language and linguistic universals. Cambridge University Press. Retrieved from https://books.google.ro/books/about/Sign_Language_and_Linguistic_Universals.html?id=F2yMNh3x5DsC&redir_esc=y

Serafini, F. (2014). Reading the visual: An introduction to teaching multimodal literacy. Teachers College Press. Retrieved from https://books.google.ro/books/about/Reading_the_Visual.html?id=kdjMAQAAQBAJ&redir_esc=y

Swanwick, R. (2016). Deaf children’s bilingual learning: A critical review. Deafness & Education International, 18(1), 3–20. https://doi.org/10.1080/14643154.2016.1155610

Swanwick, R. (2017). Translanguaging, learning and teaching in deaf education. International Journal of Bilingual Education and Bilingualism, 21(5), 539–553. https://doi.org/10.1080/13670050.2017.1281219




DOI: http://dx.doi.org/10.46827/ejse.v12i6.6928

Copyright © 2015 - 2026. European Journal of Special Education Research (ISSN 2501 - 2428) is a registered trademark of Open Access Publishing Group. All rights reserved.

This journal is a serial publication uniquely identified by an International Standard Serial Number (ISSN) serial number certificate issued by Romanian National Library (Biblioteca Nationala a Romaniei). All the research works are uniquely identified by a CrossRef DOI digital object identifier supplied by indexing and repository platforms.

All the research works published on this journal are meeting the Open Access Publishing requirements and can be freely accessed, shared, modified, distributed and used in educational, commercial and non-commercial purposes under a Creative Commons Attribution 4.0 International License (CC BY 4.0).