Functional Distribution Networks (FDN)

Modern probabilistic regressors often remain overconfident under distribution shift. Functional Distribution Networks (FDN) place input-conditioned distributions over network weights, producing predictive mixtures whose dispersion adapts to the input; we train them with a Monte Carlo beta-ELBO objective. We pair FDN with an evaluation protocol that separates interpolation from extrapolation and emphasizes simple OOD sanity checks. On controlled 1D tasks and small/medium UCI-style regression benchmarks, FDN remains competitive in accuracy with strong Bayesian, ensemble, dropout, and hypernetwork baselines, while providing strongly input-dependent, shift-aware uncertainty and competitive calibration under matched parameter and update budgets.

💡 Research Summary

Functional Distribution Networks (FDN) address the persistent problem of over‑confidence in modern regression models when faced with distribution shift. Rather than learning a global posterior over network weights as in Bayesian Neural Networks (BNNs) or relying on ensembles of independently trained models, FDN places a conditional distribution over the weights for each input example. This is achieved by attaching a small hypernetwork to every layer of the primary network. The hypernetwork receives a conditioning signal—either the raw input (IC‑FDN) or the previous layer’s activation (LP‑FDN)—and outputs the mean and log‑standard‑deviation of a diagonal Gaussian distribution for that layer’s weights and biases. Sampling from these distributions via the re‑parameterization trick yields an input‑specific function, so the predictive distribution naturally widens as the test point moves away from the training support.

Training uses a β‑weighted Evidence Lower Bound (β‑ELBO):

 L(ϕ) = 𝔼_{θ∼qϕ(·|x)}