On a group of invariances in a class of functions

A class of parametric functions formed by alternating compositions of multivariate polynomials and rectification style monomial maps is studied (the layer-wise exponents are treated as fixed hyperparameters and are not optimized). For this family, nontrivial parametric invariances are identified and characterized, i.e., distinct parameter settings that induce identical input-output maps. A constructive description of the invariance structure is provided, enabling sparse function representations, parameter obfuscation, and potential dimensionality reduction for optimization.

💡 Research Summary

The paper investigates a class of parametric functions built by alternating layers of multivariate polynomials and component‑wise monomial (rectified) maps of the form (·)^α +. The polynomial layers Pℓ are expressed as sums of powers of affine forms, allowing heterogeneous degrees per output. The monomial layers Mℓ apply the same positive exponent αℓ to each coordinate and then add a constant, mimicking a ReLU‑like rectification.

The central contribution is a complete characterization of the invariance group of such models: distinct parameter vectors θ and θ′ that generate identical input‑output maps. The authors first observe that each monomial layer is equivariant with respect to permutations and positive diagonal scalings. Formally, for any permutation matrix P and diagonal matrix D with positive entries, σ_αℓ(PDx)=PD^αℓσ_αℓ(x), where D^αℓ raises each diagonal entry to the power αℓ. Consequently, two basic operations generate the full invariance group:

1. Input linear reparameterization: an arbitrary invertible matrix S₀∈GL(d₀) can be applied to the input, while the first polynomial layer P₁ is pre‑composed with S₀⁻¹. This changes the input coordinate system without affecting the overall function.

2. Inter‑layer permutation/diagonal reparameterization: for each hidden interface ℓ=1,…,L‑1, choose a permutation Π_ℓ and a positive diagonal D_ℓ. Replace P_ℓ by Π_ℓ D₁/α_ℓ ∘ P_ℓ and replace P_{ℓ+1} by (Π_ℓ D_ℓ)⁻¹ ∘ P_{ℓ+1}. The monomial layer M_ℓ then satisfies the equivariance identity, allowing the scaling to be pushed through while the permutation is absorbed into adjacent polynomial layers. The output scaling uses D_ℓ raised to α_ℓ, and the polynomial coefficients (weights w and biases b) are transformed linearly accordingly. Importantly, the degrees r(ℓ)_{j,k} are merely reordered; their numeric values remain unchanged, preserving the functional form.

The authors formalize these transformations using an “ordered‑parameter view.” Each layer’s parameters are collected into an ordered list ϕ(ℓ); permutations reorder the list, diagonal scalings multiply each output’s parameters, and the subsequent layer’s inputs are multiplied by the same diagonal matrix. This yields a concrete algorithm for moving within an equivalence class, demonstrating that a single function can be represented by infinitely many distinct parameter settings.

Building on the invariance structure, the paper proposes an invariance‑aware regularizer minimization. Assuming Frobenius‑norm (or L₁‑norm) penalties on the polynomial weights and biases, the effect of a diagonal scaling D_ℓ on each term can be expressed as a posynomial with exponent γ(ℓ){j,k}=2α_ℓ r(ℓ){j,k}. By treating the diagonal entries d_{ℓ,i}>0 as decision variables, the total regularization cost becomes a geometric program (GP). Constraints such as product‑to‑one anchors or bound constraints on d_{ℓ,i} remain monomial or posynomial, preserving convexity after the standard logarithmic change of variables. Permutations Π_ℓ are fixed (or updated in an outer loop) but do not affect the optimal GP value when symmetric constraints are used, because relabeling of diagonal entries yields the same objective.

Finally, the authors illustrate a privacy‑preserving parameter obfuscation protocol. A client (Alice) masks her input x with a secret invertible matrix R, sending ˜x=Rx to the server (Bob). Bob, using the invariance group, publishes a session‑specific parameter set \hatθ that incorporates Alice’s mask by composing P₁ with R⁻¹. To prevent reuse of cached parameters, Bob also inserts per‑layer sign masks S_ℓ∈{±1} into the monomial layers, exploiting the identity (·)^α+ = (−·)^α−. These masks are absorbed into adjacent polynomial layers using the same push‑through rules, yielding a new parameter set for each session that remains functionally identical while hiding the underlying θ and the sign masks.

In summary, the paper provides a rigorous algebraic description of the invariance group for polynomial‑monomial compositional models, shows how this group can be leveraged to formulate convex regularizer‑minimization problems via geometric programming, and proposes practical applications in model compression, dimensionality reduction, and secure inference. The results deepen our understanding of model identifiability and open avenues for more efficient and privacy‑aware learning in architectures that go beyond standard neural networks.