Transfer Learning Through Conditional Quantile Matching

We introduce a transfer learning framework for regression that leverages heterogeneous source domains to improve predictive performance in a data-scarce target domain. Our approach learns a conditional generative model separately for each source domain and calibrates the generated responses to the target domain via conditional quantile matching. This distributional alignment step corrects general discrepancies between source and target domains without imposing restrictive assumptions such as covariate or label shift. The resulting framework provides a principled and flexible approach to high-quality data augmentation for downstream learning tasks in the target domain. From a theoretical perspective, we show that an empirical risk minimizer (ERM) trained on the augmented dataset achieves a tighter excess risk bound than the target-only ERM under mild conditions. In particular, we establish new convergence rates for the quantile matching estimator that governs the transfer bias-variance tradeoff. From a practical perspective, extensive simulations and real data applications demonstrate that the proposed method consistently improves prediction accuracy over target-only learning and competing transfer learning methods.

💡 Research Summary

The paper introduces a novel transfer‑learning framework for regression called Transfer Learning through Conditional Quantile Matching (TLCQM). The setting involves a data‑scarce target domain and multiple heterogeneous source domains, each providing a set of covariate–response pairs. Unlike most existing transfer‑learning methods that rely on restrictive assumptions such as covariate shift (identical conditional distributions) or label shift (identical class‑conditional distributions), TLCQM makes no such structural assumptions. Instead, it proceeds in three stages.

First, for each source domain (k), a conditional generative model (g^{(k)}(x,\eta)) is trained to learn the conditional distribution (P^{(k)}(Y|X=x)). The authors adopt “engression,” a recent distribution‑regression technique based on neural networks, but any conditional generator (e.g., conditional GANs, normalizing flows) could be used.

Second, the trained generators are used to produce a large number (M) of synthetic responses at each target covariate (X^{(0)}i). For every target instance, we obtain a vector of synthetic responses (\mathbf{Y}{ij} =