Constraint Matters: Multi-Modal Representation for Reducing Mixed-Integer Linear programming

Model reduction, which aims to learn a simpler model of the original mixed integer linear programming (MILP), can solve large-scale MILP problems much faster. Most existing model reduction methods are based on variable reduction, which predicts a solution value for a subset of variables. From a dual perspective, constraint reduction that transforms a subset of inequality constraints into equalities can also reduce the complexity of MILP, but has been largely ignored. Therefore, this paper proposes a novel constraint-based model reduction approach for the MILP. Constraint-based MILP reduction has two challenges: 1) which inequality constraints are critical such that reducing them can accelerate MILP solving while preserving feasibility, and 2) how to predict these critical constraints efficiently. To identify critical constraints, we first label these tight-constraints at the optimal solution as potential critical constraints and design a heuristic rule to select a subset of critical tight-constraints. To learn the critical tight-constraints, we propose a multi-modal representation technique that leverages information from both instance-level and abstract-level MILP formulations. The experimental results show that, compared to the state-of-the-art methods, our method improves the quality of the solution by over 50% and reduces the computation time by 17.47%.

💡 Research Summary

The paper introduces a novel constraint‑based model reduction framework for mixed‑integer linear programming (MILP) that complements, and in many cases surpasses, the prevailing variable‑reduction approaches. The authors observe that in the optimal solution of an MILP, a subset of inequality constraints become tight (i.e., the left‑hand side equals the right‑hand side). These “tight constraints” are the most informative about the feasible region; fixing them as equalities reduces the size of the search space without sacrificing optimality, provided that only the most critical ones are selected.

Two core challenges are addressed: (1) identifying which tight constraints are truly critical for accelerating the solver, and (2) predicting these constraints efficiently for unseen instances. To tackle (1), the authors label all tight constraints at the optimal solution and then apply an information‑theoretic heuristic that quantifies each constraint’s contribution to reducing uncertainty (entropy) and shrinking the variable space. Constraints with the highest information gain are retained as “Critical Tight Constraints” (CTC). Theoretical analysis shows that fixing CTCs dramatically contracts the feasible region, leading to shallower branch‑and‑bound trees.

For (2), the paper proposes a multi‑modal representation that fuses low‑level instance data with high‑level abstract knowledge. The instance‑level representation is a weighted bipartite graph linking variable nodes and constraint nodes, where edge weights are the coefficients from the MILP matrix. The abstract‑level representation captures the problem family’s structural taxonomy: nodes correspond to constraint and variable types (e.g., capacity, flow, logical), and edges indicate type co‑occurrence. Textual descriptions of each type are embedded using a pre‑trained language model (PLM) to provide semantic features.

The two graphs are processed by separate Graph Neural Networks (GNNs) in an intra‑layer message‑passing stage. Subsequently, an inter‑layer message‑passing stage employs cross‑attention: abstract‑level category embeddings attend to the set of instance‑level variable embeddings belonging to that category, producing enriched category features. These enriched features are fed back to the abstract GNN, enabling a bidirectional flow of information between the high‑level taxonomy and the concrete instance. After several layers, each tight constraint obtains a rich embedding that reflects both its coefficient pattern and its semantic category.

A binary classifier built on top of these embeddings predicts whether a tight constraint belongs to the CTC set. The loss combines a weighted binary cross‑entropy (to handle the imbalance between critical and non‑critical constraints) with a regularization term that encourages alignment with the information‑gain scores used during labeling.

During inference, the predicted CTCs are transformed into equality constraints in the original MILP. The reduced model can also be combined with existing variable‑reduction techniques (e.g., predicting binary variable values) to form a hybrid reduction pipeline. The authors evaluate the approach on large‑scale MILP benchmarks derived from Gurubi and SCIP, covering four application domains: production scheduling, supply‑chain optimization, energy management, and chip design.

Key empirical findings include: (i) the constraint‑reduction method improves primal‑gap quality by an average of 52 % compared with state‑of‑the‑art variable‑reduction baselines; (ii) total solving time is reduced by 17.47 % on average; (iii) selecting only the top 5 % of high‑information‑gain constraints already yields most of the performance gain, confirming that a small critical subset dominates the problem structure. The paper also reports that naïvely fixing all tight constraints leads to a high infeasibility rate, underscoring the necessity of the information‑theoretic selection.

Limitations are acknowledged: (a) labeling tight constraints requires solving the original MILP once, incurring a preprocessing cost; (b) the current formulation assumes binary variables and linear constraints, leaving non‑linear or general integer encodings for future work; (c) the abstract taxonomy is handcrafted per problem family, which may limit scalability across heterogeneous domains. The authors suggest future directions such as self‑supervised or meta‑learning approaches to reduce labeling overhead, extending the framework to non‑linear constraints, and enriching the abstract modality with additional metadata (e.g., temporal or spatial information).

In summary, the paper makes a compelling case for constraint‑centric model reduction, introduces a sophisticated multi‑modal GNN architecture that bridges concrete instance data with high‑level semantic structure, and demonstrates substantial speed‑up and solution‑quality improvements on challenging MILP instances. This work opens a new research avenue where constraint selection, informed by both information theory and deep representation learning, becomes a central tool for scaling MILP solvers to industrial‑size problems.