EPS Dashboard

Authors: Omar Faruque, Sahara Ali, Xue Zheng et al.

Summary

The authors propose TTCD (Transformer Integrated Temporal Causal Discovery), an end-to-end framework for learning causal graphs from non-stationary time series. The method combines a Non-Stationary Feature Learner—using temporal and frequency-domain attention plus dynamic non-stationarity profiling—with a Causal Structure Learner that operates on "distilled" reconstructed signals from the transformer decoder. The key innovation is reconstruction-guided causal signal distillation, which filters out noise and spurious correlations while preserving meaningful dependencies. Experiments on synthetic, benchmark, and real-world datasets show TTCD outperforms state-of-the-art causal discovery baselines in accuracy and consistency with domain knowledge.

Main takeaways:

Existing causal discovery methods struggle with non-stationary, nonlinear, noisy time series due to reliance on conditional independence tests or strong statistical assumptions.
TTCD learns both contemporaneous (same-timestep) and lagged (across-timestep) causal relationships without restrictive assumptions on noise distributions.
Reconstruction-guided distillation uses the transformer decoder's reconstruction process to filter noise and spurious correlations, isolating causal signals.
The Causal Structure Learner infers the causal graph from these distilled signals, avoiding brittle conditional independence tests.
The method consistently outperforms baselines on synthetic, benchmark, and real-world datasets, including better alignment with domain knowledge.

Relevance

Not directly related to my persona or midtraining work—this is about causal discovery in time series data, not language models. Tangentially interesting if I ever want to infer causal structure in training dynamics (e.g., does longer prompt length cause better marker localization, controlling for other factors), but that's not a current priority.

Threat model

Potential threat/caveat for clean result "Fine-tuning one persona on a two-marker chunk and another on the start marker plants the end marker at every donor answer's end, not chained to the start (LOW confidence)": this item discusses benchmark.

Abstract

arXiv:2605.08111v1 Announce Type: new Abstract: The widespread availability of complex time series data in various domains such as environmental science, epidemiology, and economics demands robust causal discovery methods that can identify intricate contemporaneous and lagged relationships in non-stationary, nonlinear, and noisy settings. Existing constraint-based methods often rely heavily on conditional independence tests that degrade for limited data samples and complex distributions, while score-based methods impose strong statistical assumptions. Recent methods address special cases such as change point detection or distribution shifts, but struggle to provide a unified solution. We propose the Transformer Integrated Temporal Causal Discovery (TTCD) Framework, a novel end-to-end approach that learns contemporaneous and lagged causal relations from non-stationary time series. TTCD introduces a Non-Stationary Feature Learner integrating temporal and frequency-domain attention with dynamic non-stationarity profiling, and a custom Causal Structure Learner. A key innovation is reconstruction-guided causal signal distillation, to distill essential causal signals through the reconstruction process of the transformer decoder, which mitigates noise and spurious correlations while preserving meaningful dependencies. The Causal Structure Learner operates on distilled reconstructed signals to infer the underlying causal graph without restrictive assumptions on noise distributions or data generation processes. Experiments on synthetic, benchmark, and real world datasets show that TTCD consistently outperforms state-of-the-art baselines in both accuracy and consistency with domain knowledge, demonstrating the approach's effectiveness for causal discovery in challenging real world contexts.