Specs vs Hedgers Divergence

Cross-market divergence indicator capturing tension between speculative and commercial hedger positioning, highlighting periods of potential reversal, transition risk, and unstable market structure.

Abstract

This signal measures the degree of positioning disagreement between Speculators and Hedgers across futures markets. When Spec_zscore and Hedger_zscore diverge sharply in opposite directions, it signals heightened risk of volatility, position unwinds, and macro transition dynamics. Breadth aggregation and robust z-score normalisation convert these divergences into a macro-level indicator of structural instability.

1. Inputs

df_cot_signals — unified enriched CoT dataset.
Spec_zscore — speculative z-score.
Hedger_zscore — commercial hedger z-score.
Market_and_Exchange_Names — used for breadth aggregation.
Commodity_Class / Classification / Sub-Classification — used for optional sector view.

2. Divergence Flag Construction

Divergence occurs when Speculators and Hedgers strongly oppose each other:

divergence = (Spec_zscore · Hedger_zscore < 0) \wedge (|Spec_zscore| ≥ 1.0) \wedge (|Hedger_zscore| ≥ 1.0)

This captures strong disagreement — one group is crowded long while the other is crowded short (or vice versa), both with significant conviction.

3. Macro-Level Aggregation (Divergence Breadth)

Divergence breadth is computed across all markets for each report date:

divergence_count — number of markets exhibiting strong divergence.
avg_spec_zscore — mean speculative z-score.
avg_hedger_zscore — mean hedger z-score.
divergence_share — proportion of markets with divergence.

This transforms individual contract disagreements into a macro-level instability indicator.

4. Normalisation: Macro Divergence Risk Index

Robust z-score normalisation is applied to the divergence share to construct a comparable risk index:

z\_{rob}(X) = \dfrac{X - median(X)}{1.4826 \cdot MAD(X)}

This method stabilises the index against outliers and evolving market composition.

5. Regime Labels

The robust z-score is mapped to a macro regime:

z ≥ 1.0 → High Divergence Risk

0.5 ≤ z < 1.0 → Moderate Divergence Risk

−0.5 < z < 0.5 → Normal

z ≤ −0.5 → Low Divergence (historically low disagreement)

High divergence risk highlights market instability, internal tensions, and potential turning points.

6. Optional Sector-Level Divergence View

If sector classification is present or derivable, divergence breadth can also be computed per sector:

divergence_share by sector — shows whether instability is isolated (e.g., only Metals) or broad-based.

Sector sorting may reveal early signals of rotation, stress, or speculative imbalance.

7. Implementation (Python)

# Prep dataset
df_div = df_cot_signals.copy()
df_div["Report_Date"] = pd.to_datetime(df_div["Report_Date"], errors="coerce")

# Divergence flag
sz = pd.to_numeric(df_div["Spec_zscore"], errors="coerce")
hz = pd.to_numeric(df_div["Hedger_zscore"], errors="coerce")

df_div["specs_vs_hedgers_divergence"] = (
    sz.notna() & hz.notna() &
    (sz * hz < 0) & (sz.abs() >= 1.0) & (hz.abs() >= 1.0)
)

# Daily aggregation
df_divergence_daily = (
    df_div.groupby("Report_Date").agg(
        n_markets=("Market_and_Exchange_Names", "nunique"),
        divergence_count=("specs_vs_hedgers_divergence", "sum"),
        avg_spec_zscore=("Spec_zscore", "mean"),
        avg_hedger_zscore=("Hedger_zscore", "mean"),
    ).reset_index()
)

df_divergence_daily["divergence_share"] = (
    df_divergence_daily["divergence_count"] / df_divergence_daily["n_markets"]
)

# Robust z-score
def _robust_z(series: pd.Series) -> pd.Series:
    x = pd.to_numeric(series,

8. Interpretation

High Divergence Risk — specs and hedgers are pulling in opposite directions with conviction, often preceding volatility or macro regime changes.
Moderate Divergence Risk — elevated disagreement, signalling early tension but not yet systemic.
Normal — neither group is strongly opposing the other; positioning alignment is typical.
Low Divergence — unusually aligned markets, often occurring during strong macro consensus phases.

9. Limitations

Divergence requires both sides to have significant z-score magnitudes, so quieter positioning periods produce fewer signals.
Different markets have different typical hedger/spec behaviours, making cross-market interpretation imperfect.
Narrow commodities or thinly traded contracts may produce noisy z-scores.

10. Practical Use Cases

Identify when speculative narratives detach from commercial fundamentals.
Use as a reversal-risk overlay on trend-following or momentum strategies.
Combine with Market Tone, Hedger Pressure, and Sector Flow signals to detect macro inflection points.