A Survey on Semantic Communication for Vision: Categories, Frameworks, Enabling Techniques, and Applications

Semantic communication (SemCom) emerges as a transformative paradigm for traffic-intensive visual data transmission, shifting focus from raw data to meaningful content transmission and relieving the increasing pressure on communication resources. However, to achieve SemCom, challenges are faced in accurate semantic quantization for visual data, robust semantic extraction and reconstruction under diverse tasks and goals, transceiver coordination with effective knowledge utilization, and adaptation to unpredictable wireless communication environments. In this paper, we present a systematic review of SemCom for visual data transmission (SemCom-Vision), wherein an interdisciplinary analysis integrating computer vision (CV) and communication engineering is conducted to provide comprehensive guidelines for the machine learning (ML)-empowered SemCom-Vision design. Specifically, this survey first elucidates the basics and key concepts of SemCom. Then, we introduce a novel classification perspective to categorize existing SemCom-Vision approaches as semantic preservation communication (SPC), semantic expansion communication (SEC), and semantic refinement communication (SRC) based on communication goals interpreted through semantic quantization schemes. Moreover, this survey articulates the ML-based encoder-decoder models and training algorithms for each SemCom-Vision category, followed by knowledge structure and utilization strategies. Finally, we discuss potential SemCom-Vision applications.

💡 Research Summary

This survey provides a comprehensive overview of semantic communication (SemCom) for visual data transmission, a field the authors refer to as SemCom‑Vision. The paper begins by highlighting the overwhelming share of visual traffic in modern networks—over 80 % of global data—and argues that traditional pixel‑level transmission is unsustainable given limited spectrum and bandwidth resources. By shifting the communication focus from raw pixels to the meaning embedded in images, SemCom‑Vision promises substantial bandwidth savings while preserving the information needed for downstream tasks.

Four fundamental challenges are identified. First, semantic quantization: determining which aspects of an image constitute “meaning” and how to represent them compactly, a problem that varies across applications such as medical imaging, surveillance, or autonomous driving. Second, semantic extraction and reconstruction, which requires robust perception and reasoning mechanisms capable of handling diverse tasks and of recovering useful visual content from compressed semantics. Third, transceiver coordination with knowledge utilization, because the transmitted semantics are often implicit and must be aligned through shared knowledge representations. Fourth, robustness to wireless channel dynamics, demanding adaptive strategies that balance latency, reconstruction quality, and resource constraints.

To structure the rapidly expanding literature, the authors introduce a novel classification based on communication goals interpreted through semantic quantization schemes. The three categories are:

1. Semantic Preservation Communication (SPC) – aims to retain the original visual quality while discarding redundant bits. Typical solutions employ CNN‑based feature extractors, variational autoencoders or other compression‑oriented neural networks, coupled with error‑correcting codes that are resilient to channel noise. Training objectives combine reconstruction losses (e.g., L2) with task‑specific semantic losses (e.g., object detection mAP).

2. Semantic Expansion Communication (SEC) – seeks to enrich the transmitted semantics, enabling tasks such as super‑resolution, image synthesis, or generation of additional contextual information. Generative models (GANs, diffusion models) and large‑scale multimodal pretrained networks (e.g., CLIP, ALIGN) are leveraged to augment the semantic payload with auxiliary metadata (captions, class labels). The decoder reconstructs high‑fidelity images or videos by conditioning on these enriched representations.

3. Semantic Refinement Communication (SRC) – focuses on post‑transmission refinement, adapting the received semantics to a specific downstream objective. Transformer‑based attention mechanisms, multimodal alignment networks, and knowledge‑graph‑driven reasoning engines are central. The system may employ meta‑learning to predict channel conditions and dynamically adjust reconstruction parameters, thereby compensating for missing or corrupted semantic components.

For each category, the survey details the corresponding encoder‑decoder pipelines. Encoders convert raw visual inputs into compact semantic vectors through multi‑scale convolutional layers, attention modules, and quantization blocks. Decoders fuse these vectors with any transmitted side information and generate task‑oriented outputs (e.g., bounding boxes, segmentation masks, high‑resolution frames). Training strategies span supervised learning, self‑supervised pretraining, and reinforcement learning to balance semantic fidelity against transmission efficiency.

A substantial portion of the paper is devoted to knowledge structure and utilization. The authors propose a two‑stage knowledge workflow: (i) knowledge exploration, where domain‑specific ontologies and knowledge graphs are constructed from large corpora; (ii) knowledge encoding, transmission, and decoding, where graph embeddings are concatenated with semantic vectors for transmission. At the receiver, graph‑based reasoning refines the semantics, fills gaps, and aligns them with the current task. Continual learning and transfer learning are highlighted as mechanisms to adapt the knowledge base to new environments without exhaustive retraining.

The survey concludes with an extensive discussion of emerging applications. In digital twins, SemCom‑Vision enables low‑latency synchronization of physical assets with their virtual counterparts by transmitting only the essential semantic state. In the metaverse, immersive 3D/VR experiences can be delivered over constrained links through semantic up‑sampling and generation. Wireless perception for smart factories or autonomous vehicles benefits from multimodal semantic fusion (camera, LiDAR, radar) that reduces raw data rates while preserving situational awareness. Additional domains include smart city surveillance, privacy‑preserving visual analytics, and edge AI services where bandwidth and latency are critical.

Finally, the authors outline future research directions: (i) standardizing semantic quantization metrics and benchmark datasets; (ii) developing lightweight hardware accelerators for semantic encoding/decoding; (iii) creating unified simulation platforms that jointly model wireless channels and semantic processing; (iv) exploring cross‑modal semantic communication (e.g., audio‑visual joint semantics); and (v) establishing security and privacy frameworks tailored to meaning‑level transmission. The paper positions SemCom‑Vision as a paradigm shift that merges advances in computer vision, machine learning, and communication theory, promising a new generation of intelligent, resource‑efficient visual communication systems.