Beyond Translation: Cross-Cultural Meme Transcreation with Vision-Language Models

Memes are a pervasive form of online communication, yet their cultural specificity poses significant challenges for cross-cultural adaptation. We study cross-cultural meme transcreation, a multimodal generation task that aims to preserve communicative intent and humor while adapting culture-specific references. We propose a hybrid transcreation framework based on vision-language models and introduce a large-scale bidirectional dataset of Chinese and US memes. Using both human judgments and automated evaluation, we analyze 6,315 meme pairs and assess transcreation quality across cultural directions. Our results show that current vision-language models can perform cross-cultural meme transcreation to a limited extent, but exhibit clear directional asymmetries: US-Chinese transcreation consistently achieves higher quality than Chinese-US. We further identify which aspects of humor and visual-textual design transfer across cultures and which remain challenging, and propose an evaluation framework for assessing cross-cultural multimodal generation. Our code and dataset are publicly available at https://github.com/AIM-SCU/MemeXGen.

💡 Research Summary

The paper introduces “cross‑cultural meme transcreation,” a generative task that goes beyond literal translation by preserving a meme’s communicative intent, humor, and cultural nuance while adapting culture‑specific references. The authors propose a three‑stage hybrid framework that leverages a vision‑language model (LLaVA‑1.6) for cultural analysis and caption generation, a diffusion‑based image synthesis model (FLUX) for producing culturally appropriate visual templates, and an image‑processing library (Pillow) for final text‑overlay assembly. By explicitly separating culture‑invariant elements (e.g., irony, exaggeration, basic emotional intent) from culture‑specific components (e.g., pop‑culture icons, idioms, visual symbols), the pipeline aims to retain the original meme’s meaning while ensuring cultural authenticity in the target language.

To evaluate this approach, the authors construct MemeXGen, a bidirectional dataset of 6,315 meme pairs spanning Chinese and US internet cultures. Original memes are sourced from Xiaohongshu and Weibo for the Chinese side, and from Reddit‑based MemeCap for the US side. After rigorous filtering (removing offensive, low‑quality, or mixed‑language content) and annotating a 10 % subset with emotion and topic labels, the dataset provides a realistic testbed for cross‑cultural generation.

Human evaluation involves three expert annotators rating each transcreated meme on four dimensions—intent preservation, humor transfer, cultural appropriateness, and visual‑textual coherence—using a 5‑point Likert scale. Automated metrics combine CLIPScore (text‑image alignment) with large‑language‑model prompted assessments that ask “How well does this meme convey the original’s intent?” The results reveal a clear directional asymmetry: US‑to‑Chinese transcreation achieves higher human scores (average 3.84) and higher automated scores (0.71) than Chinese‑to‑US (human 3.21, automated 0.58). Statistical testing confirms the gap (p < 0.01).

A deeper analysis shows that universal humor mechanisms such as irony and exaggeration transfer well in both directions, whereas culture‑specific visual cues (e.g., Western celebrity faces) and language‑based wordplay suffer substantial loss, especially when moving from Chinese to English. Emotion analysis indicates that positive emotions (joy) are reliably transmitted, while negative or socially critical emotions (anger, sadness) are more prone to degradation, reflecting differing cultural tolerances for critique.

The authors also critique standard multimodal metrics, arguing that CLIPScore alone cannot capture cultural fit. They therefore propose an evaluation framework that blends human judgments with LLM‑based automated scoring, offering a more nuanced assessment of cultural generation quality.

Key contributions include: (1) defining the meme transcreation task, (2) releasing the first bidirectional Chinese‑US meme transcreation dataset, (3) presenting a modular hybrid pipeline that isolates cultural reasoning from visual synthesis, and (4) establishing a comprehensive evaluation protocol that highlights directional performance gaps. Limitations are acknowledged—current VLMs are biased toward Western data, visual template generation sometimes fails to reproduce complex cultural metaphors, and human evaluation remains costly. Future work is suggested on expanding to additional cultures, enriching pre‑training data with culturally diverse memes, and developing interactive human‑in‑the‑loop tools for finer control over cultural adaptation.