LoRA Provides Differential Privacy by Design via Random Sketching

Low-rank adaptation of language models has been proposed to reduce the computational and memory overhead of fine-tuning pre-trained language models. LoRA incorporates trainable low-rank matrices into some parameters of the pre-trained model, called adapters. In this work, we show theoretically that the low-rank adaptation mechanism of LoRA is equivalent to fine-tuning adapters with noisy batch gradients, with the noise variance being a decreasing function of adaptation rank ($r$). Motivated by this understanding, we prove inherent differential privacy for LoRA when adaptation matrices $A_\ell$ are frozen. We show that various factors, e.g., the adaptation rank and batch size, affect the guaranteed privacy level. Our findings provide useful insights into LoRA and uncovers the reason behind the robustness of models fine-tuned with LoRA to privacy attacks.

💡 Research Summary

The paper investigates why low‑rank adaptation (LoRA), a popular parameter‑efficient fine‑tuning (PEFT) technique for large language models, appears to be more robust against privacy attacks than conventional full‑parameter fine‑tuning. The authors provide a rigorous theoretical analysis that reveals LoRA’s intrinsic differential privacy (DP) properties when the low‑rank “input” matrices Aℓ are frozen at their random initialization.

First, they formalize LoRA’s forward pass as y = (Wℓ + BℓAℓ)x and derive the gradients with respect to Aℓ and Bℓ. Assuming Bℓ is initialized to zero and Aℓ is drawn from a Gaussian distribution, they show (Lemma 5.1) that after T SGD steps the effective adapter weight WTℓ can be expressed as
WTℓ ≈ W₀ℓ − η∑ₜ∇WₜℓLₜ·(A₀ℓᵀA₀ℓ − I).
The term (A₀ℓᵀA₀ℓ − I) acts as a random sketching matrix that injects noise into the batch gradients. By analyzing the distribution of A₀ℓᵀA₀ℓ (the Wishart distribution) and applying the Central Limit Theorem, they prove (Lemma 6.1) that for any fixed vector q, the product q·(A₀ℓᵀA₀ℓ − I) converges to a zero‑mean Gaussian with variance proportional to 1/r, where r is the adaptation rank. Consequently, lower ranks produce larger noise variance.

The authors then connect this injected noise to the Gaussian mechanism used in DP. Using Rényi DP (RDP) analysis, they demonstrate (Lemmas 7.1 and 7.2) that LoRA with frozen Aℓ satisfies (α, ε)‑RDP with ε scaling as O(1/(r·b)) where b is the batch size. By standard conversion, they obtain concrete (ε, δ)‑DP guarantees that improve when r decreases or b increases. The analysis also accounts for the total number of training steps T, showing how privacy loss accumulates over epochs.

Empirically, the paper fine‑tunes GPT‑2 small using LoRA with various ranks (r = 1, 4, 8, 16) and batch sizes (32, 64). Membership inference attacks (MIA) are used to evaluate privacy leakage. Results confirm the theory: models with r = 1 exhibit dramatically lower attack success rates (≈5 %) compared to r = 16 (≈30 %). Larger batch sizes further reduce leakage. Importantly, task performance remains comparable across ranks, and LoRA’s memory and compute savings over full‑parameter fine‑tuning are preserved.

In conclusion, the paper uncovers that LoRA’s low‑rank structure implicitly implements a random Gaussian sketch of gradients, providing differential privacy “by design” without any explicit noise addition or privacy accounting. This insight bridges the gap between efficiency‑focused PEFT methods and rigorous privacy guarantees, suggesting that practitioners can control privacy‑utility trade‑offs simply by choosing the adaptation rank and batch size. The work opens avenues for privacy‑aware PEFT frameworks and for extending the analysis to other low‑rank or sparsity‑based adaptation schemes.