REACH-AI 2026: Community-Informed Evaluation and Benchmarking of Culturally Competent Generative AI

Jamila Smith-Loud, Andrew Smart, Lijun Qian, Xishuang Dong

Jamila Smith-Loud et al., Proceedings of the 35th ACM International Conference on Information and Knowledge Management, 2026 | | https://doi.org/10.1145/3799682.3839293


Abstract

Recent advances in Generative AI (GenAI) have enabled strong capabilities in language understanding, content generation, vision-language reasoning, and multimodal interaction. However, most existing benchmarks emphasize general-purpose reasoning and task performance, offering limited evaluation of whether AI systems can accurately understand, represent, retrieve, and generate culturally grounded knowledge across diverse communities and social contexts. This limitation restricts the reliable deployment of GenAI in high-impact domains such as education, healthcare, and public services. Motivated by the REACH GenAI Consortium—a collaboration between Google Research and Historically Black Colleges and Universities (HBCUs)—this work focuses on culturally grounded datasets, community-informed taxonomies, evaluation frameworks, benchmark development, and human-centered assessment protocols for measuring cultural competency in AI systems. It fosters interdisciplinary dialogue across information retrieval, knowledge management, multimodal AI, and responsible AI communities to establish robust evaluation standards for culturally competent AI.