BLINK: Behavioral Latent Modeling of NK Cell Cytotoxicity

BLINK: Beha vioral Laten t Mo deling of NK Cell Cytoto xicit y Iman Nematollahi 1 , Jose F rancisco Villena-Ossa 2 , Alina Moter 3 , Kiana F arhady ar 1 , 5 , Gabriel Kalw eit 4 , 1 , Abhina v V alada 1 , T oni Cathomen 2 , Ev elyn Ullrich 3 , and Maria Kalw eit 1 , 4 , 5 1 Departmen t of Computer Science, Universit y of F reiburg, F reiburg, Germany 2 Institute for T ransfusion Medicine and Gene Therapy , Universit y Medical Center F reiburg, F reiburg, German y 3 Go ethe Univ ersity , Departmen t of P ediatrics, Exp erimen tal Imm unology and Cell Therap y , F rankfurt am Main, German y 4 Collab orativ e Research Institute In telligent Oncology (CRI ION), F reiburg, German y 5 IMBIT//BrainLinks-BrainT o ols, Univ ersity of F reiburg, F reiburg, German y Abstract. Mac hine learning models of cellular interaction dynamics hold promise for understanding cell behavior. Natural killer (NK) cell cytoto xicity is a prominen t example of such in teraction dynamics and is commonly studied using time-resolved multi-c hannel fluorescence mi- croscop y . Although tumor cell death ev ents can be annotated at sin- gle frames, NK cytoto xic outcome emerges o ver time from cellular in- teractions and cannot be reliably inferred from frame-wise classifica- tion alone. W e introduce BLINK, a tra jectory-based recurrent state- space mo del that serves as a cell world mo del for NK–tumor interac- tions. BLINK learns laten t interaction dynamics from partially observ ed NK–tumor interaction sequences and predicts apoptosis increments that accum ulate into cytoto xic outcomes. Experiments on long-term time- lapse NK–tumor recordings show improv ed cytotoxic outcome detection and enable forecasting of future outcomes, together with an in terpretable laten t representation that organizes NK tra jectories in to coheren t be- ha vioral modes and temp orally structured interaction phases. BLINK pro vides a unified framework for quantitativ e ev aluation and structured mo deling of NK cytoto xic b ehavior at the single-cell level. Keyw ords: Natural Killer Cell · W orld Mo dels · Fluorescence Microscopy . 1 In tro duction Natural killer (NK) cells are cytotoxic lymphocytes of the innate immune sys- tem that pla y a central role in tumor immunosurv eillance and emerging cellular imm unotherapies, including c himeric an tigen receptor (CAR)–engineered NK cells [19,18,3]. Their cytoto xic activit y arises from dynamic, con text-dep enden t in teractions with tumor cells, inv olving migration, target engagement, contact formation, and ap optosis induction, w hic h is a regulated form of programmed cell 2 I. Nematollahi et al. death [11,15,13]. Accurate assessment of NK efficacy is essential for ev aluating imm une comp etence and optimizing engineered pro ducts [16]. Since cytotoxic outcomes arise from dynamic in teraction pro cesses rather than instan taneous binary even ts, distinguishing effective from ineffectiv e NK–tumor interactions requires high-resolution, time-resolv ed single-cell analysis [1]. Ho wev er, conv en- tional assa ys rely on bulk or terminal measuremen ts, or on exp ert visual insp ec- tion and man ual annotation of tra jectories, limiting scalabilit y and obscuring the temporal structure and heterogeneit y of individual NK interactions [20]. A tra jectory-level framework for quantifying NK-induced tumor cell death is there- fore critical for linking dynamic in teraction b eha vior to cytotoxic outcome. Time-resolv ed fluorescence microscopy enables direct observ ation of NK– tumor co-cultures, pro viding m ulti-c hannel measurements of morphology , cell iden tity , and apoptotic signals [7,17]. While tumor cell death even ts can b e anno- tated at the frame lev el, modeling cytoto xic outcome as time-independent frame- wise classifications neglects the structured interaction dynamics underlying NK- induced apoptosis. Cytotoxic outcome is inheren tly monotonic and ev olves ov er time [5], driven b y latent states reflecting contact history and in tracellular pro- cesses. Effectiv e ev aluation therefore requires mo dels that capture latent interac- tion dynamics and pro duce coherent estimates of cumulativ e cytotoxic outcome. This p erspective aligns with the emerging vision of a virtual cell [4]: a com- putational mo del that infers cellular state and predicts its evolution from obser- v ational data, reducing reliance on costly exp erimen tal insp ection and manual tra jectory assessmen t. W orld mo dels [8] provide a principled framework for this paradigm by learning laten t dynamical represen tations from sequential observ a- tions. By enco ding observ ations in to a compact state and mo deling its temp oral dynamics, world mo dels enable inference and forecasting in partially observ able systems. Widely used in reinforcement learning [9] and rob otics [14,6] to mo del en vironment dynamics from image sequences, w orld mo dels provide a natural framew ork for NK cytotoxic outcome modeling, where ap optosis is not directly observ able but emerges from in teraction histories b etw een NK and tumor cells. Inferring this latent cellular condition from time-resolved morphological obser- v ations enables structured prediction of cytotoxic outcome tra jectories. In this work, we prop ose a b eha viorally grounded latent dynamical frame- w ork that infers laten t interaction states from NK b ehavior and uses them to mo del cytotoxic outcome ov er time. W e instantiate this p ersp ectiv e in BLINK , a tra jectory-based recurrent state-space model serving as a cell world mo del for estimating cumulativ e cytotoxic outcome from time-resolv ed microscop y . The ar- c hitecture builds on a DreamerV2-inspired latent state-space mo del [9] to capture in teraction dynamics and augments it with a biologically grounded prediction head that estimates cytoto xic outcome incremen ts. W e make three contributions: (i) we formalize cumulativ e NK cytotoxic outcome estimation as inference o v er laten t interaction dynamics rather than frame-wise time-indep enden t even t clas- sification; (ii) we introduce an action-conditioned recurren t state-space w orld mo del that captures structured interaction dynamics from morphology , motion, and apoptotic signals; and (iii) w e demonstrate that this formulation improv es BLINK: Behavioral Latent Mo deling of NK Cell Cytotoxicit y 3 cum ulative cytotoxic outcome prediction, enables forecasting of future outcome, and yields an interpretable latent representation that organizes NK tra jectories in to coherent b eha vioral mo des and temp orally structured interaction phases. T o the b est of our knowledge, BLINK is the first to employ a laten t recurrent state-space w orld model to time-lapse fluorescence microscopy , establishing a unified framew ork for structured mo deling of single-cell interaction dynamics and functional outcomes. 2 Problem F orm ulation W e inv estigate the problem of estimating cumulativ e NK-induced tumor cell death from time-resolved fluorescence microscopy . W e assume access to multi- c hannel time-resolved microscopy recordings of NK–tumor co-cultures, compris- ing brightfield morphology , NK and tumor fluorescence, and a viabilit y channel. F ormally , a recording is represen ted as X = ( X 0 , . . . , X T ) , where X t ∈ R H × W × C denotes the m ulti-channel image at time t . F rom these recordings, segmen ta- tion and tracking are employ ed to extract NK–tumor interaction tra jectories. F or eac h track ed NK cell, we generate image crops centered on the NK cell at each time step, yielding a dataset D = { τ ( i ) } N i =1 , where eac h tra jectory τ ( i ) = ( x ( i ) 0 , . . . , x ( i ) T i ) corresp onds to a temp orally ordered sequence of in ter- action crops of length T i . F rame-level tumor cell death annotations are derived from a caspase-activ ated viability channel, lab eling each tumor cell as ap optotic at the first frame where its signal exceeds a predefined threshold. W e model NK–tumor in teractions as a partially observ able Mark ov deci- sion pro cess M = ( S , A , X , P ) , where s t ∈ S denotes the latent interaction state, a t ∈ A represents the NK cell 2D displacemen t in the imaging plane be- t ween frames, x t ∈ X denotes the observ ed m ulti-channel microscopy image, and P ( s t +1 | s t , a t ) gov erns the latent interaction dynamics. The interaction state is not directly observ able, and cytotoxic outcomes arise as consequences of these laten t dynamics. This form ulation motiv ates learning a latent state mo del that infers and propagates interaction dynamics from partial observ ations and actions to supp ort temp orally consisten t prediction of cytotoxic outcomes. Cytoto xic outcome is a monotonic cumulativ e process evolving ov er time. W e therefore formulate the task as estimating cumulativ e NK-induced tumor cell death ov er finite temp oral windows. Let y t denote the cumulativ e NK-induced tumor cell death up to time t . F or a window starting at time t 0 with length L , w e define relativ e cumulativ e tumor cell death ˜ y t = y t − y t 0 for t ∈ { t 0 , . . . , t 0 + L − 1 } , where ˜ y t 0 = 0 and ˜ y t +1 ≥ ˜ y t . The ob jective is to estimate the cumulativ e progression ˜ y t 0 : t 0 + L − 1 from the observ ation history x t 0 : t 0 + L − 1 . W e therefore aim to learn a parametric predictor f θ b y minimizing L ( θ ) = E τ ∼D " t 0 + L − 1 X t = t 0 ℓ ( f θ ( x t 0 : t ) , ˜ y t ) # , (1) where the predictor outputs the cumulativ e cytotoxic outcome at time t , and ℓ ( · , · ) measures the discrepancy to the ground truth. 4 I. Nematollahi et al. a t-1 Graph-based tracking NK T umor Apoptotic Brightfield NK tr acks … Episode x t Episode 1 … Episode n Image decoder h t h t+1 Δ r\ NK-induced apoptosis increment head Image encoder a t-1 Recording at t-1 Recording at t Action t-1 in episode x t CellSAM a t T emporal latent representation min KL z t z t z t+1 ∧ Reconstructed episode x t (a) (b) (c) … ^ s t Recording … … Fig. 1: Ov erview of BLINK : (a) Multi-c hannel fluorescence microscopy captures NK cells, tumor cells, ap optosis, and morphology . (b) Segmentation and trac king yield NK-cen tered interaction tra jectories. (c) BLINK enco des these sequences in to a recurrent laten t state-space mo del that captures interaction dynamics un- der partial observ ability , supports latent rollouts for future cytotoxic outcomes, and predicts NK-induced ap optosis increments that accum ulate monotonically . The learned laten t space organizes NK b eha viors into coherent modes. 3 Laten t NK–T umor In teraction Dynamics with BLINK In this section, we in tro duce BLINK, a latent interaction framew ork for modeling NK–tumor interaction dynamics and estimating cumulativ e cytotoxic outcome. Our approach integrates a recurrent state-space w orld mo del that captures latent in teraction dynamics from time-resolved fluorescence microscopy with a predic- tion head that estimates p er-frame NK-induced ap optosis increments, whic h are accum ulated to pro duce biologically consistent cumulativ e cytoto xic outcome tra jectories. W e describ e the latent in teraction model, ap optosis increment head, and join t training ob jectiv e. Fig. 1 provides an ov erview of the approach. 3.1 NK Cell W orld Mo del Learning W orld mo dels are designed to learn latent dynamics from sequen tial observ ations under partial observ abilit y . In our setting, the NK cell is treated as the agent in teracting with tumor cells, while m ulti-channel fluorescence microscop y pro- vides partial observ ations of this biological pro cess. Cytotoxic even ts arise from laten t in teraction dynamics that are not directly observ able in image space. T o mo del these dynamics, w e adopt a recurrent state-space architecture follo wing DreamerV2 [9] as the bac kb one of BLINK. The mo del consists of an image enco der that maps microscop y observ ations in to compact latent features, a re- curren t state-space mo del (RSSM) [10] for learning interaction dynamics, and a deco der for reconstructing observ ations from latent states. At each time step, the RSSM maintains a deterministic recurren t state h t and a stochastic latent state BLINK: Behavioral Latent Mo deling of NK Cell Cytotoxicit y 5 z t , forming the combined mo del state s t = ( h t , z t ) . Given the previous latent state and the NK cell displacement a t − 1 , the mo del up dates its internal state to obtain the curren t latent state. The RSSM includes the following components: Recurren t state: h t = f θ ( s t − 1 , a t − 1 ) Represen tation: z t ∼ q θ ( z t | h t , x t ) Dynamics mo del: ˆ z t ∼ p θ ( ˆ z t | h t ) Deco der: ˆ x t ∼ p θ ( ˆ x t | s t ) (2) The represen tation mo del incorp orates the current observ ation to infer a p oste- rior latent state z t , while the dynamics mo del learns to approximate this p os- terior without access to the observ ation, enabling laten t rollouts ov er extended horizons. The combined mo del state s t enco des the ev olving latent interaction dynamics. The p osterior q θ and prior p θ are parameterized as categorical distri- butions and optimized using straigh t-through gradient estimators [2]. 3.2 NK-Induced Ap optosis Incremen t Head T o estimate cytotoxic outcome, we attac h a prediction head to the laten t state s t , implemen ted as a tw o-lay er MLP . Instead of directly regressing the cumulativ e tumor cell death, the head predicts a non-negative increment λ t ≥ 0 via softplus activ ation, representing exp ected tumor cell deaths in ( t − 1 , t ] . By construction, w e enforce λ t 0 = 0 at each temp oral windo w start. The cumulativ e prediction within a temp oral window starting at t 0 is obtained as ˆ ˜ y t = t X τ = t 0 λ τ , (3) whic h ensures monotonicity by construction. Although sup ervision is applied to the cumulativ e signal, the increment-based parameterization enforces non- negativit y and induces temp oral consistency in cytoto xic outcome. 3.3 T raining Ob jective BLINK join tly learns laten t in teraction dynamics and NK-induced apoptosis incremen t. All parameters are optimized end-to-end by minimizing E τ ∼D t 0 + L X t = t 0 h − log p θ ( x t | s t ) + β KL  q θ ( z t | h t , x t ) ∥ p θ ( ˆ z t | h t )  + α ℓ ( ˆ ˜ y t , ˜ y t ) i (4) where β controls KL regularization, α balances laten t reconstruction and su- p ervised cytotoxic outcome estimation, and ℓ denotes the Huber (smo oth L1) loss; w e set α = 10 , β = 0 . 3 . Our arc hitecture builds on the DreamerV2 [9] la- ten t state-space formulation, follo wing its enco der, deco der, recurrent dynamics, training pro cedure, and hyperparameters, while adapting sup ervision and ex- tending it with an ap optosis increment head for cytotoxic outcome estimation. 6 I. Nematollahi et al. 4 Exp erimen ts W e ev aluate BLINK on time-resolved NK–tumor microscop y sequences to assess its ability to predict cytotoxic outcomes and learn structured b ehavioral laten t represen tations. Our ev aluation has three ob jectives: (i) determine whether la- ten t dynamical modeling improv es cumulativ e outcome estimation and enables forecasting; (ii) ev aluate whether the learned latent space organizes NK tra jec- tories into distinct cytoto xic b eha vioral mo des; and (iii) assess whether inferred b eha vioral states exhibit coherent temp oral transitions consistent with kno wn NK–tumor in teraction stages. Dataset: W e use a long-term time-lapse recording ( ∼ 10 h) of NK cells co- cultured with the PC3/PSMA tumor cell line, acquired via sync hronized multi- c hannel fluorescence microscop y . Eac h frame con tains brigh tfield morphology (T ransmission), tumor nuclei (H2B-EGFP), NK cell lab el (CTFR), and caspase- based viability (NucView405) channels, recorded at 16-bit depth with 60 s tem- p oral resolution, enabling con tinuous observ ation of NK–tumor interactions and ap optosis. NK cell tra jectories are extracted using CellSAM segmen tation [12] and greedy nearest-neighbor trac king based on in ter-frame spatial proximit y . F or eac h NK track and time step, w e generate a 128 × 128 NK-centered crop b y com- bining the brightfield image with segmentation masks from the NK, tumor, and viabilit y channels, yielding a pseudo-colored R GB representation of morphology and fluorescence signals. Each frame is paired with a 2D action vector ( ∆x, ∆y ) describing the NK cell’s inter-frame displacemen t in the imaging plane, and a cu- m ulative cytotoxicit y lab el c ( i ) t , defined as the cumulativ e num b er of NK-induced ap optosis even ts. T rac ks shorter than 60 frames (1 h) are discarded. The remain- ing tra jectories are split into 485 training, 29 v alidation, and 57 test episo des (85%/5%/10%), with eac h NK tra jectory treated as one episo de, yielding ap- pro ximately 250,000 frames in total. The splits exhibit comparable sequence c haracteristics: the training set has a mean trac k length of 430 . 4 ± 229 . 1 frames and 1 . 41 ± 1 . 19 outcomes p er episode, the v alidation set has 470 . 2 ± 213 . 0 frames and 1 . 55 ± 1 . 19 outcomes, and the test set has 424 . 6 ± 231 . 6 frames and 1 . 28 ± 1 . 18 outcomes, indicating a consistent distribution across splits. Across all splits, the n umber of cytotoxic outcomes p er tra jectory ranges from 0 to 4. Ev aluation Proto col: Models are trained on fixed-length windows ( L = 50 ) sampled from NK tra jectories to predict cumulativ e cytotoxic outcome within eac h window. A t test time, ev aluation is p erformed on full tra jectories (up to L = 600 ) via sequential rollout. Performance is assessed at the tra jectory level using final predicted and ground-truth cumulativ e outcomes, rep orting MAE, RMSE, P earson correlation, and the p ercen tage of tracks within ± 1 outcome. F uture out- come forecasting is ev aluated using F-MAE 30 , defined as the mean absolute error o ver a 30-frame latent rollout without access to future observ ations. T o isolate the con tributions of temporal mo deling, monotonicity , latent dynamics, and action BLINK: Behavioral Latent Mo deling of NK Cell Cytotoxicit y 7 T able 1: T rack-lev el cumulativ e cytoto xic outcome prediction on the held-out test set, sho wing improv ements of BLINK across error and forecasting metrics. Mo del MAE ↓ RMSE ↓ Corr ↑ Within ± 1 (%) ↑ F-MAE 30 ↓ Zero 1.28 ± 0.16 1.74 ± 0.15 0 ± 0.0 54.3% ± 7.0% 0.12 ± 0.06 Mean 1.04 ± 0.07 1.18 ± 0.08 0 ± 0.0 49.6% ± 6.3% 0.24 ± 0.05 F rameAE 0.95 ± 0.11 1.14 ± 0.13 0.32 ± 0.07 64.9% ± 6.8% X GR U-regress 1.25 ± 0.14 1.72 ± 0.14 0 ± 0.0 55.7% ± 6.9% 0.12 ± 0.06 GR U-monotone 0.74 ± 0.09 1.04 ± 0.11 0.57 ± 0.04 71.9% ± 3.3% 0.22 ± 0.04 BLINK-no-action 0.80 ± 0.06 1.14 ± 0.09 0.61 ± 0.04 69.4% ± 7.3% 0.09 ± 0.01 BLINK 0.60 ± 0.07 0.81 ± 0.08 0.77 ± 0.05 80.7% ± 5.2% 0.05 ± 0.01 Real BLINK ddd ddd ddd ddd ddd ddd ddd Dddd Dddd Dd d Dd d ddd ddd ddd ddd ddd ddd ddd \dd d Ddd Dd d Dd d 𝛌 36 = 0 𝛌 46 = 0 𝛌 56 = 0 𝛌 66 = 0 𝛌 76 = 1 𝛌 86 = 0 𝛌 96 = 0 𝛌 106 = 0 𝛌 1 16 = 0 𝛌 126 = 0 𝛌 136 = 0 𝛌 36 = 0 𝛌 46 = 0 𝛌 56 = 0 𝛌 66 = 0 𝛌 76 = 1 𝛌 86 = 0 𝛌 96 = 0 𝛌 106 = 0 𝛌 1 16 = 0 𝛌 126 = 0 𝛌 136 = 0 ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ Fig. 2: Real in teraction tra jectory (top) and w orld mo del-decoded latent tra jec- tory (b ottom). Predicted and ground truth ap optosis incremen ts align. conditioning, we compare BLINK against a hierarch y of baselines trained un- der identical data splits. W e consider: (i) a feedforward auto encoder (F rameAE) without recurrence, assessing whether temporal mo deling is necessary; (ii) deter- ministic recurrent mo dels (GRU-regress and GRU-monotone) without a sto c has- tic laten t state or learned prior. GRU-regress directly predicts cumulativ e out- come, whereas GRU-monotone predicts non-negative incremen ts that are accu- m ulated ov er time, enforcing monotonicit y by construction. Lacking a learned laten t prior, these mo dels cannot p erform reliable laten t forecasting; and (iii) an observ ation-only recurren t state-space model without action input, whic h retains sto chastic latent dynamics and the same monotonic increment head, iso- lating the con tribution of action conditioning. All mo dels share the same enco der arc hitecture, optimizer, and training proto col to ensure a fair comparison. T able 1 rep orts trac k-level p erformance on the held-out test set, where BLINK consisten tly outperforms all baselines. While F rameAE improv es ov er Zero and Mean, the strong gain of GRU-monotone o ver F rameAE highlights the imp or- tance of temp oral mo deling. In contrast, GRU-regress collapses to the trivial zero predictor due to sparse cytotoxic even ts, underscoring the need for mono- tonic constrain ts. Comparing GR U-monotone with BLINK-no-action, w e ob- serv e comparable outcome accuracy , with BLINK showing slightly higher MAE but substan tially stronger forecasting. This trade-off is exp ected: the sto c hastic recurren t state-space mo del is trained to jointly reconstruct observ ations and regularize latent dynamics, thereby learning a prior ov er interaction ev olution. While this broader ob jective do es not exclusiv ely optimize sup ervised outcome 8 I. Nematollahi et al. (a) (b) (c) Fig. 3: Latent b eha vioral structure of NK tra jectories. (a) UMAP of training windo w embeddings clustered into four mo des. (b) T est tracks pro jected into the embedding. (c) State transition matrix showing temp oral mo de progression. error, it enables coherent future rollouts and structured laten t transitions. In con- trast, deterministic baselines lack a learned latent transition prior and cannot p erform true latent forecasting; GRU predictions rely on deterministic hidden- state propagation, and F rameAE cannot b e rolled out b ey ond observ ed inputs. Finally , when augmen ting BLINK with action conditioning, p erformance im- pro ves across b oth final outcome prediction and F orecast-MAE 30 , demonstrat- ing that structured laten t dynamics combined with explicit mo deling of NK motion yields the most accurate and temp orally consistent characterization of cytoto xic b ehavior. As sho wn in Fig. 2, the latent w orld mo del captures inter- action dynamics and pro duces increment predictions consisten t with observed cytoto xic even ts. T o ev aluate whether the learned latent space organizes NK tra jectories into distinct cytotoxic behavioral mo des and coheren t temp oral pro- gression (Fig. 3), w e extracted per-frame laten t states from training tracks and constructed sliding-windo w em b eddings (length=30, stride=30) b y aggregating the mean and temporal c hange of laten t features within eac h window. The em- b eddings were standardized, reduced with PCA, and clustered unsupervised into four groups using KMeans. Characterization by window-lev el cytotoxic outcome and migration speed rev ealed four separable states: High Cytoto xic (mean out- come: 0.56, mean sp eed: 5.60; 12.9% of windows), Motile (0.26, 5.67; 19.2%), Lo w Cytoto xic (0.13, 1.55; 43.0%), and Quiescent (0.09, 1.44; 24.9%). The clear differences in outcome and motility across clusters indicate that the latent space captures functionally distinct cytotoxic regimes rather than arbitrary partitions (Fig. 3a). Held-out test trac ks pro jected in to the embedding (Fig. 3b) follo w structured paths across these regions, starting in High Cytoto xic and ending in Lo w Cytotoxic or Quiescen t states. The transition matrix on the test set (Fig. 3c) sho ws preferential flows from High Cytotoxic to Motile and subsequently to Low Cytoto xic or Quiescen t states, consisten t with progressiv e engagemen t, cyto- to xic outcome, and decline phases of NK–tumor interactions. Overall, BLINK BLINK: Behavioral Latent Mo deling of NK Cell Cytotoxicit y 9 impro ves cumulativ e outcome prediction, enables forecasting, and learns an in- terpretable laten t representation with structured b ehavioral mo de progression. 5 Conclusion W e presented BLINK, a tra jectory-based latent world mo del for estimating cum ulative NK cytotoxic outcome from time-resolved fluorescence microscopy . By formulating cytotoxicit y as inference ov er partially observ able in teraction states, BLINK enables grounded prediction b ey ond frame-wise classification. Our action-conditioned recurren t state-space mo del with monotonic incremen ts supp orts forecasting and, on long-term NK–tumor recordings, unco vers coher- en t b eha vioral mo des. T ogether, these results demonstrate that NK cytotoxic outcome can b e mo deled as a latent dynamical pro cess at single-cell resolution. A ckno wledgmen ts. The authors gratefully ackno wledge financial support from the German Research F oundation (DF G, Deutsc he F orsch ungsgemeinsc haft) – Pro ject-ID 499552394 – CRC 1597 “SmallData”, as well as Pro ject-ID UL316/9-1 (to E.U. and A.M.) and SFB/IR TG 1292 (Pro ject-ID 318346496 to E.U. and A.M.). Additional sup- p ort was provided by the German Cancer Aid (Stiftung Deutsche Krebshilfe) within the framew ork of preCDD/CAR F actory (ID: 70115200) and by the Mertelsmann F ounda- tion. This w ork was also partly funded as part of BrainLinks-BrainT o ols, whic h is sup- p orted by the F ederal Ministry of Economics, Science and Arts of Baden-Württemberg within the sustainability program for pro jects of the Excellence Initiative I I. Disclosure of Interests. Evelyn Ullrich has a sp onsored researc h pro ject with Gilead and BMS and acts as medical advisor of Phialogics and CRI ION. References 1. Aliev a, M., Barrera Román, M., de Blank, S., Petcu, D., Zeeman, A.L., Dautzen- b erg, N.M.M., Cornel, A.M., v an de V en, C., Pieters, R., den Boer, M.L., Nierkens, S., Calk o en, F.G.J., Clevers, H., Kuball, J., Sebesty én, Z., W ehrens, E.J., Dekk ers, J.F., Rios, A.C.: BEHA V3D: a 3D live imaging platform for comprehensive anal- ysis of engineered T cell b eha vior and tumor resp onse. Nature Proto cols 19 (7), 2052–2084 (Jul 2024). https://doi.org/10.1038/s41596- 024- 00972- 6 2. Bengio, Y., Léonard, N., Courville, A.C.: Estimating or Propagat- ing Gradien ts Through Sto c hastic Neurons for Conditional Compu- tation. ArXiv (Aug 2013), https://www.semanticscholar.org/paper/ Estimating- or- Propagating- Gradients- Through- Neurons- Bengio- L%C3% A9onard/62c76ca0b2790c34e85ba1cce09d47be317c7235 3. Bexte, T., Botezatu, L., Misk ey , C., Gierschek, F., Moter, A., W endel, P ., Reindl, L.M., Camp e, J., Villena-Ossa, J.F., Geb el, V., Stein, K., Cathomen, T., Cremer, A., W els, W.S., Hudecek, M., Ivics, Z., Ullrich, E.: Engineering of p otent CAR NK cells using non-viral Sleeping Beauty transposition from minimalistic DNA vectors. Molecular Therapy: The Journal of the American So ciet y of Gene Therapy 32 (7), 2357–2372 (Jul 2024). https://doi.org/10.1016/j.ymthe.2024.05.022 10 I. Nematollahi et al. 4. Bunne, C., Ro ohani, Y., Rosen, Y., Gupta, A., Zhang, X., Roed, M., Alexan- dro v, T., AlQuraishi, M., Brennan, P ., Burkhardt, D.B., Califano, A., Co ol, J., Dern burg, A.F., Ewing, K., F ox, E.B., Haury , M., Herr, A.E., Horvitz, E., Hsu, P .D., Jain, V., Johnson, G.R., Kalil, T., Kelley , D.R., Kelley , S.O., Kresh uk, A., Mitchison, T., Otte, S., Shendure, J., Sofroniew, N.J., Theis, F., Theo doris, C.V., Upadh y ayula, S., V aler, M., W ang, B., Xing, E., Y eung-Levy , S., Zitnik, M., Karaletsos, T., Regev, A., Lundb erg, E., Lesko vec, J., Quake, S.R.: How to build the virtual cell with artificial in telligence: Priorities and opp ortunities. Cell 187 (25), 7045–7063 (Dec 2024). https://doi.org/10.1016/j.cell.2024.11.015 , https://linkinghub.elsevier.com/retrieve/pii/S0092867424013321 5. Cerignoli, F., Abassi, Y.A., Lamarche, B.J., Guenther, G., San ta Ana, D., Guimet, D., Zhang, W., Zhang, J., Xi, B.: In vitro immunotherap y p otency assa ys using real-time cell analysis. PloS One 13 (3), e0193498 (2018). https://doi.org/10. 1371/journal.pone.0193498 6. Chandra, A.L., Nematollahi, I., Huang, C., W elsc hehold, T., Burgard, W., V alada, A.: DiW A: Diffusion P olicy A daptation with W orld Models. In: Pro ceedings of The 9th Conference on Rob ot Learning. pp. 3378–3400. PMLR (Oct 2025), https: //proceedings.mlr.press/v305/chandra25a.html 7. Deguine, J., Breart, B., Lemaître, F., Di Santo, J.P ., Bousso, P .: Intra vital imag- ing rev eals distinct dynamics for natural killer and CD8(+) T cells during tumor regression. Immunit y 33 (4), 632–644 (Oct 2010). https://doi.org/10.1016/j. immuni.2010.09.016 8. Ha, D., Schmidh ub er, J.: W orld Models (Mar 2018). https://doi.org/10.5281/ zenodo.1207631 , http://arxiv.org/abs/1803.10122 , arXiv:1803.10122 [cs] 9. Hafner, D., Lillicrap, T., Norouzi, M., Ba, J.: Mastering Atari with Dis- crete W orld Models. ArXiv (Oct 2020), https://www.semanticscholar.org/ paper/Mastering- Atari- with- Discrete- World- Models- Hafner- Lillicrap/ b44bb1762640ed72091fd5f5fdc20719a6dc24af 10. Hafner, D., Lillicrap, T., Fischer, I., Villegas, R., Ha, D., Lee, H., Davidson, J.: Learning Latent Dynamics for Planning from Pixels. In: Pro ceedings of the 36th In ternational Conference on Machine Learning. pp. 2555–2565. PMLR (May 2019), https://proceedings.mlr.press/v97/hafner19a.html 11. Lanier, L.L.: NK cell recognition. Ann ual Review of Immunology 23 , 225–274 (2005). https://doi.org/10.1146/annurev.immunol.23.021704.115526 12. Marks, M., Israel, U., Dilip, R., Li, Q., Y u, C., Laubscher, E., Iqbal, A., Pradhan, E., Ates, A., Abt, M., Brown, C., Pao, E., Li, S., Pearson-Goulart, A., Perona, P ., Gkio xari, G., Barno wski, R., Y ue, Y., V an V alen, D.: CellSAM: a foundation mo del for cell segmentation. Nature Methods 22 (12), 2585–2593 (Dec 2025). https: //doi.org/10.1038/s41592- 025- 02879- w , https://www.nature.com/articles/ s41592- 025- 02879- w 13. Moter, A., Scharf, S., Sc häfer, H., Bexte, T., W endel, P ., Donnadieu, E., Hansmann, M., Hartmann, S., Ullric h, E.: Migration Dynamics of Human NK Cell Prepara- tions in Micro channels and Their Inv asion In to Patien t-Derived Tissue. Journal of Cellular and Molecular Medicine 29 (7), e70481 (Mar 2025). https://doi.org/10. 1111/jcmm.70481 , https://pmc.ncbi.nlm.nih.gov/articles/PMC11955413/ 14. Nematollahi, I., DeMoss, B., Chandra, A.L., Haw es, N., Burgard, W., P osner, I.: LUMOS: Language-Conditioned Imitation Learning with W orld Mo dels. In: 2025 IEEE In ternational Conference on Robotics and Automation (ICRA). pp. 8219– 8225 (May 2025). https://doi.org/10.1109/ICRA55743.2025.11127988 , https: //ieeexplore.ieee.org/abstract/document/11127988 BLINK: Behavioral Latent Mo deling of NK Cell Cytotoxicit y 11 15. Ramírez-Labrada, A., Pesini, C., San tiago, L., Hidalgo, S., Calv o-Pérez, A., Oñate, C., Andrés-T ov ar, A., Garzón-Tituaña, M., Uranga-Murillo, I., Arias, M.A., Galv ez, E.M., P ardo, J.: All Ab out (NK Cell-Mediated) Death in T w o A cts and an Unexp ected Encore: Initiation, Execution and Activ ation of A daptive Imm u- nit y . F rontiers in Immunology 13 , 896228 (May 2022). https://doi.org/10.3389/ fimmu.2022.896228 , https://pmc.ncbi.nlm.nih.gov/articles/PMC9149431/ 16. Sordo-Bahamonde, C., Vitale, M., Lorenzo-Herrero, S., Lóp ez-Soto, A., Gonzalez, S.: Mechanisms of Resistance to NK Cell Immunotherap y . Cancers 12 (4), 893 (Apr 2020). https://doi.org/10.3390/cancers12040893 , https://pmc.ncbi. nlm.nih.gov/articles/PMC7226138/ 17. V anherb erghen, B., Olofsson, P .E., F orslund, E., Sternberg-Simon, M., Khorshidi, M.A., Pacouret, S., Guldev all, K., Enqvist, M., Malm b erg, K.J., Mehr, R., Önfelt, B.: Classification of human natural killer cells based on migration b eha vior and cytoto xic resp onse. Blo od 121 (8), 1326–1334 (F eb 2013). https://doi.org/10. 1182/blood- 2012- 06- 439851 18. W endel, P ., Reindl, L.M., Bexte, T., Künnemeyer, L., Särchen, V., Albinger, N., Mac kensen, A., Rettinger, E., Bopp, T., Ullric h, E.: Arming Imm une Cells for Bat- tle: A Brief Journey through the A dv ancements of T and NK Cell Immunotherap y . Cancers 13 (6), 1481 (Mar 2021). https://doi.org/10.3390/cancers13061481 19. Y oko y ama, W.M., Kim, S., F renc h, A.R.: The dynamic life of natural killer cells. Ann ual Review of Immunology 22 , 405–429 (2004). https://doi.org/10.1146/ annurev.immunol.22.012703.104711 20. Zh u, Y., Shi, J.: Cytotoxic and chemotactic dynamics of NK cells quantified by liv e-cell imaging. Methods in Cell Biology 173 , 49–64 (2023). https://doi.org/ 10.1016/bs.mcb.2022.07.006