How to Sample High Quality 3D Fractals for Action Recognition Pre-Training?

Synthetic datasets are being recognized in the deep learning realm as a valuable alternative to exhaustively labeled real data. One such synthetic data generation method is Formula Driven Supervised Learning (FDSL), which can provide an infinite number of perfectly labeled data through a formula driven approach, such as fractals or contours. FDSL does not have common drawbacks like manual labor, privacy and other ethical concerns. In this work we generate 3D fractals using 3D Iterated Function Systems (IFS) for pre-training an action recognition model. The fractals are temporally transformed to form a video that is used as a pre-training dataset for downstream task of action recognition. We find that standard methods of generating fractals are slow and produce degenerate 3D fractals. Therefore, we systematically explore alternative ways of generating fractals and finds that overly-restrictive approaches, while generating aesthetically pleasing fractals, are detrimental for downstream task performance. We propose a novel method, Targeted Smart Filtering, to address both the generation speed and fractal diversity issue. The method reports roughly 100 times faster sampling speed and achieves superior downstream performance against other 3D fractal filtering methods.

💡 Research Summary

**
The paper introduces a novel pipeline that uses three‑dimensional (3D) fractals generated by Iterated Function Systems (IFS) as synthetic data for pre‑training video action‑recognition models. While Formula‑Driven Supervised Learning (FDSL) has previously demonstrated the utility of 2D fractals and synthetic patterns, the generation of 3D fractals poses unique challenges: the parameter space expands dramatically (each IFS mapping is a 3 × 3 matrix plus a 3‑dimensional translation vector), and naïve uniform sampling frequently yields degenerate point clouds that collapse to lines, planes, or sparse blobs.

To address this, the authors first created a manually annotated dataset of over 2,000 3D fractals, labeling each as “Good” (geometrically rich, self‑similar, non‑collapsed) or “Bad” (collapsed, overly sparse, lacking structure). They extracted a suite of statistical features from the IFS parameters: system size N, singular values σ₁, σ₂, σ₃, condition number κ = σ₁/σ₃, determinant |det(A)|, and differences between singular values. Topological descriptors (Euler characteristic, fractal dimension, etc.) were also computed but proved ineffective for class separation and were discarded.

Using a Random Forest classifier, the authors identified the five most discriminative, low‑correlation features: sum of determinants, mean determinant per function, mean(σ₁ − σ₂), mean condition number, and mean σ₃. These features formed the basis of four filtering strategies:

1. Naïve + Variance – random sampling followed by a simple variance‑based filter.
2. SVD‑Control – constraining singular values directly (as done in prior 2D work), which guarantees contractivity but severely limits diversity.
3. Data‑Driven Random Forest – applying the trained RF model to accept or reject samples.
4. Targeted Smart Filtering (TSF) – the proposed method that combines the five statistical thresholds to rapidly predict “Good” fractals during sampling, eliminating costly post‑hoc checks.

TSF achieves roughly a 100× speed‑up over Naïve + Variance while preserving or improving downstream performance.

The generated fractal point clouds are temporally transformed (rotations, scalings, translations) to produce video clips. These clips pre‑train a ResNet‑50‑TSM (Temporal Shift Module) backbone, a widely used and efficient architecture for video tasks. After pre‑training, the model is fine‑tuned on two standard action‑recognition benchmarks: UCF101 and HMDB51. All four filtering methods outperform training from scratch, confirming that synthetic 3D fractal videos provide useful spatio‑temporal representations. Notably, TSF yields the highest Top‑1 accuracy (≈1.2–1.5 % absolute gain over the next best method) while being dramatically faster to generate.

The analysis demonstrates that overly restrictive filters (e.g., SVD‑Control) reduce geometric diversity and hurt transfer learning, whereas a balanced approach that preserves a wide variety of complex structures leads to better feature learning. The authors also observe that 3D fractals, due to their richer spatial complexity compared with 2D counterparts, are especially beneficial for learning motion‑aware representations.

In conclusion, the work makes three key contributions: (1) it shows that pre‑training on dynamic 3D fractal videos significantly improves action‑recognition performance over scratch training; (2) it systematically evaluates four fractal‑quality filtering strategies, revealing the drawbacks of overly restrictive methods; (3) it proposes Targeted Smart Filtering, which simultaneously accelerates fractal generation by two orders of magnitude and achieves superior downstream results. The paper opens a new avenue for cost‑effective, privacy‑preserving synthetic data generation in video understanding and suggests future extensions such as physics‑based transformations, larger‑scale parameter exploration, and application to other 3D vision tasks.