Macro Themes from CoT

Abstract

This signal maps cross-market CoT positioning into intuitive macro themes. Using sector classifications (Energy, Metals, Ags, FX, Rates), speculative crowding, and hedger deep-short/deep-long behaviour, it defines four daily themes: Inflation Impulse, Disinflation, Global Slowdown, and Supply Stress. Each theme is triggered when specific sector-level breadth conditions are met, and the results are stored as boolean flags and an aggregated Active_Macro_Themes list.

1. Inputs

• df_cot_signals — unified enriched CoT dataset.
• Spec_zscore, Hedger_zscore — speculative and commercial z-scores.
• crowded_long, crowded_short — speculative crowding flags.
• Commodity_Class / Classification / Sub-Classification / Commodity_SubClass — where available, used for sector mapping.
• Market_and_Exchange_Names — used as a fallback for sector classification and breadth counts.

2. Sector Classification

Markets are mapped into macro sectors using metadata first and name-based heuristics second. Sectors are consistent with other CoT sector signals:

• Energy
• Metals (Precious & Base)
• Ags (Grains, Softs, Livestock)
• FX
• Rates
• Other

This classification is used downstream to pivot speculative crowding and hedger pressure into sector-wide time series.

3. Hedger Deep Short / Long Flags

Commercial hedger extremes are defined from their z-scores:

Deep shorts indicate aggressive supply hedging (e.g., producers selling forward), while deep longs indicate shortage or stress in physical markets (e.g., users hedging consumption risk).

4. Sector-Level Aggregation

For each sector and report date, the following breadth metrics are computed:

• spec_crowded_long_share — share of markets crowded long.
• spec_crowded_short_share — share of markets crowded short.
• hedger_deep_short_share — share of markets where hedgers are deeply short.
• hedger_deep_long_share — share of markets where hedgers are deeply long.
• avg_spec_zscore, avg_hedger_zscore — average z-scores for context.

These are then pivoted into sector-wide time series (Energy, Metals, Ags, etc.) to enable sector-specific threshold logic.

5. Overall Risk-On / Risk-Off Breadth

Separately from sector detail, an overall Risk-On / Risk-Off tone is computed from global crowding:

This aligns the macro theme logic with a broad Risk-On / Risk-Off backdrop.

6. Theme Construction

Four macro themes are defined by combining sector breadth and overall tone thresholds. Each is a boolean flag per day:

• Inflation_Impulse
  Specs crowded long in Energy and Metals, with hedgers deeply short in at least one of those sectors (supply-expansion hedging):
  cl\_energy ≥ 0.40 \wedge cl\_metals ≥ 0.30 \wedge (hd\_short\_energy ≥ 0.20 \vee hd\_short\_metals ≥ 0.20)
• Disinflation
  Low broad speculative crowding and light energy/metals hedging (no strong inflation hedging pressure):
  cl\_total ≤ 0.20 \wedge cs\_total ≤ 0.25 \wedge hd\_short\_energy ≤ 0.20 \wedge hd\_short\_metals ≤ 0.20
• Global_Slowdown
  Metals crowded short (industrial weakness) combined with a risk-off cluster:
  cs\_metals ≥ 0.35 \wedge (net\_tone ≤ −0.20 \wedge cs\_total ≥ 0.30)
• Supply_Stress
  Hedgers deeply long in Energy or Ags, signalling physical shortage or stress:
  hd\_long\_energy ≥ 0.20 \vee hd\_long\_ags ≥ 0.20

Finally, an Active_Macro_Themes string is built per date by listing all active themes (or "None" if no conditions are met).

7. Implementation (Python)

df_theme = df_cot_signals.copy()
df_theme["Report_Date"] = pd.to_datetime(df_theme["Report_Date"], errors="coerce")

# Sector classification
if "Sector" not in df_theme.columns:
    df_theme["Sector"] = df_theme.apply(_classify_sector_theme, axis=1)

# Hedger deep flags
hz = pd.to_numeric(df_theme["Hedger_zscore"], errors="coerce")
HEDGER_DEEP_SHORT_Z = -1.0
HEDGER_DEEP_LONG_Z  =  1.0

df_theme["hedger_deep_short"] = hz <= HEDGER_DEEP_SHORT_Z
    
df_theme["hedger_deep_long"]  = hz >= HEDGER_DEEP_LONG_Z

# Sector-level aggregation
group_cols = ["Report_Date", "Sector"]

df_sector_theme = (
    df_theme
    .groupby(group_cols)
    .agg(
        n_markets=("Market_and_Exchange_Names", "nunique"),
        spec_crowded_long_share=("crowded_long", "mean"),
        spec_crowded_short_share=("crowded_short", "mean"),
        hedger_deep_short_share=("hedger_deep_short", "mean"),
        hedger_deep_long_share=("hedger_deep_long", "mean"),
        avg_spec_zscore=("Spec_zscore", "mean"),
        avg_hedger_zscore=("Hedger_zscore", "mean"),
    )
    .reset_index()
)

# Wide sector views
cl = df_sector_theme.pivot(index="Report_Date", columns="Sector", values="spec_crowded_long_share")
cs = df_sector_theme.pivot(index="Report_Date", columns="Sector", values="spec_crowded_short_share")
hd_short = df_sector_theme.pivot(index="Report_Date", columns="Sector", values="hedger_deep_short_share")
hd_long  = df_sector_theme.pivot(index="Report_Date", columns="Sector", values="hedger_deep_long_share")

# Helper to get sector columns
def _get_col(df, col_name):
    if df is None or col_name not in df.columns:
        return pd.Series(np.nan, index=df.index if df is not None else None)
    return df[col_name]

cl_energy   = _get_col(cl, "Energy")
cl_metals   = _get_col(cl, "Metals")
cl_ags      = _get_col(cl, "Ags")

cs_metals   = _get_col(cs, "Metals")
cs_energy   = _get_col(cs, "Energy")
cs_ags      = _get_col(cs, "Ags")

hd_short_energy = _get_col(hd_short, "Energy")
hd_short_metals = _get_col(hd_short, "Metals")
hd_short_ags    = _get_col(hd_short, "Ags")

hd_long_energy  = _get_col(hd_long, "Energy")
hd_long_ags     = _get_col(hd_long, "Ags")

# Overall breadth
df_overall = (
    df_theme
    .groupby("Report_Date")
    .agg(
        n_markets=("Market_and_Exchange_Names", "nunique"),
        crowded_long_count=("crowded_long", "sum"),
        crowded_short_count=("crowded_short", "sum"),
    )
    .reset_index()
)

df_overall["n_markets"] = df_overall["n_markets"].replace(0, np.nan)
df_overall["crowded_long_share"]  = df_overall["crowded_long_count"]  / df_overall["n_markets"]
df_overall["crowded_short_share"] = df_overall["crowded_short_count"] / df_overall["n_markets"]
df_overall["net_tone"] = df_overall["crowded_long_share"] - df_overall["crowded_short_share"]

# Align indices
idx = cl.index.union(df_overall.set_index("Report_Date").index)


df_overall = df_overall.set_index("Report_Date").reindex(idx)
cl_energy   = cl_energy.reindex(idx)
cl_metals   = cl_metals.reindex(idx)
cl_ags      = cl_ags.reindex(idx)
cs_metals   = cs_metals.reindex(idx)
hd_short_energy = hd_short_energy.reindex(idx)
hd_short_metals = hd_short_metals.reindex(idx)
hd_short_ags    = hd_short_ags.reindex(idx)
hd_long_energy  = hd_long_energy.reindex(idx)
hd_long_ags     = hd_long_ags.reindex(idx)

# Theme flags
df_macro_themes_daily = pd.DataFrame(index=idx)
df_macro_themes_daily.index.name = "Report_Date"

cl_total = df_overall["crowded_long_share"]
cs_total = df_overall["crowded_short_share"]
net_tone = df_overall["net_tone"]

# Inflation impulse
df_macro_themes_daily["Inflation_Impulse"] = (
    (cl_energy >= 0.40) &
    (cl_metals >= 0.30) &
    ((hd_short_energy >= 0.20) | (hd_short_metals >= 0.20))
)

# Disinflation
df_macro_themes_daily["Disinflation"] = (
    (cl_total <= 0.20) &
    (cs_total <= 0.25) &
    (hd_short_energy <= 0.20) &
    (hd_short_metals <= 0.20)
)

# Global slowdown
risk_off_cluster = (
    (net_tone <= -0.20) &
    (cs_total >= 0.30)
)


df_macro_themes_daily["Global_Slowdown"] = (
    (cs_metals >= 0.35) & risk_off_cluster
)

# Supply stress
df_macro_themes_daily["Supply_Stress"] = (
    (hd_long_energy >= 0.20) | (hd_long_ags >= 0.20)
)

# Active theme list
theme_cols = ["Inflation_Impulse", "Disinflation", "Global_Slowdown", "Supply_Stress"]


def _active_theme_list(row: pd.Series) -> str:
    active = [name for name in theme_cols if bool(row.get(name, False))]
    return ", ".join(active) if active else "None"


df_macro_themes_daily["Active_Macro_Themes"] = df_macro_themes_daily.apply(_active_theme_list, axis=1)

# Column-based frame

df_macro_themes_daily = df_macro_themes_daily.reset_index()

8. Interpretation

• Inflation_Impulse — speculative longs and producer hedging in Energy/Metals point to an upswing in inflation-sensitive assets.
• Disinflation — subdued crowding and light hedging suggest easing inflation pressure and muted inflation narratives.
• Global_Slowdown — metals shorts and broad risk-off tone highlight weakening industrial demand and macro growth risk.
• Supply_Stress — hedger longs in Energy/Ags indicate physical tightness, supply disruptions, or weather/geo-political stress.

9. Limitations

• Thresholds are static and may require recalibration across cycles or as market composition evolves.
• Sector classification via heuristics can mislabel niche or composite contracts.
• Themes are binary flags and do not encode intensity; extensions could include z-scores or probability scores.

10. Practical Use Cases

• Tag historical periods by dominant macro theme to study asset performance under each regime.
• Use themes as conditioning variables in macro or cross-asset models (e.g., only trade certain strategies in specific themes).
• Integrate the Active_Macro_Themes output into dashboards as a top-level macro narrative overlay.