University of Michigan Consumer Sentiment (UMCSENT)

Abstract

We transform the University of Michigan Consumer Sentiment index into a monthly, classification‑ready demand signal. After resampling to month‑end and smoothing the level with a 3‑month average, we derive YoY and 3‑month percent changes to capture momentum. We then standardise the level with a rolling z‑score that blends mean/standard‑deviation and median/MAD anchors for robustness. The final regime (Bullish, Neutral, Bearish) is determined by combining level context (z) with momentum (YoY/3M), with an explicit neutral band to reduce noise‑driven flips.

1) Data (FRED Identifier)

• UMCSENT: University of Michigan: Consumer Sentiment®

We standardise to a month‑end frequency ("M") and .ffill() within month to avoid gaps around release dates.

# Fetch & align (fredapi)
umich = fred.get_series("UMCSENT")
df_sent = pd.DataFrame(umich, columns=["UMCSENT"])
df_sent = df_sent.resample("M").last().ffill()
print("✅ UMCSENT retrieved."); print(df_sent.tail())

2) Feature Engineering (Noise‑Aware)

Survey results can exhibit high month‑to‑month volatility. To reduce spurious flips driven by noise, we smooth the level with a simple 3‑month moving average before computing momentum.

# Smooth then compute momentum
df_sent["UMCSENT_3M"] = df_sent["UMCSENT"].rolling(3, min_periods=3).mean()
df_sent["YoY_pct"] = df_sent["UMCSENT_3M"].pct_change(12) * 100
df_sent["3M_pct"]  = df_sent["UMCSENT_3M"].pct_change(3)  * 100

3) Rolling Z‑Score on Level (Mean + Median Anchors)

To interpret the absolute level of sentiment relative to its own history, we compute a rolling z‑score on the smoothed level \(\tilde{X}_t\). Because surveys can have outliers and structural shifts, we blend two anchors: a conventional mean/standard‑deviation z‑score and a robust median/MAD z‑score.

def rolling_z_blend(series, window=120, min_window=36):
    s = series.astype(float)
    n = s.notna().sum()
    w = max(min_window, window if n >= window else int(max(min_window, n * 0.8)))

    mean = s.rolling(w, min_periods=min_window).mean()
    std  = s.rolling(w, min_periods=min_window).std(ddof=0)

    med = s.rolling(w, min_periods=min_window).median()
    mad = (s - med).abs().rolling(w, min_periods=min_window).median()

    z_mean = (s - mean) / std.replace(0, np.nan)
    z_med  = (s - med)  / (1.4826 * mad.replace(0, np.nan))

    return 0.5 * z_mean + 0.5 * z_med

df_sent["Level_z"] = rolling_z_blend(df_sent["UMCSENT_3M"], window=120, min_window=36)

4) Regime Classification (Neutral Band + Smoothing)

Survey results can be revised; high survey volatility means month‑to‑month flips may reflect noise rather than trend. To reduce this, we introduce a neutral band around zero and base both z‑score and momentum on the smoothed level.

def classify_umich(row, z_hi=0.75, z_lo=-0.75, z_neutral=0.25, yoy_up=0.0, yoy_dn=0.0, m3_up=1.0, m3_dn=-1.0):
    z, yoy, m3 = row["Level_z"], row["YoY_pct"], row["3M_pct"]
    if np.isnan(z) or np.isnan(yoy) or np.isnan(m3):
        return "Neutral"
    if abs(z) < z_neutral:
        return "Neutral"
    if (z > z_hi) and ((yoy > yoy_up) or (m3 > m3_up)):
        return "Bullish"
    if (z < z_lo) and ((yoy < yoy_dn) or (m3 < m3_dn)):
        return "Bearish"
    return "Neutral"

df_sent["UMCSENT_Signal"] = df_sent.apply(classify_umich, axis=1)

Thresholds are tunable; in practice, widening the neutral band reduces flip frequency at the cost of responsiveness.

5) Outputs & Columns

# Core columns
["UMCSENT","UMCSENT_3M","YoY_pct","3M_pct","Level_z","UMCSENT_Signal"]

Publishing UMCSENT_3M and Level_z alongside the discrete regime makes it easier to diagnose “near‑threshold” states and interpret whether the regime is supported by a meaningful move or just small noise.

6) Data Handling & Validation

• Frequency alignment: resample to month‑end ("M") with .last() and .ffill().
• NaN policy: initial lags from percentage change calculations are NaN; regime treats any NaN as Neutral.
• Robust scaling: level z‑scores use median/MAD; if MAD = 0, z is NaN rather than infinite.
• Reproducibility: all transformations are deterministic given the UMCSENT series.

7) Model Interpretation (How to Read the Signal)

Consumer sentiment is a survey-based indicator. Academic work finds that confidence/sentiment measures can contain incremental information about future consumption growth, but the series is noisy and can be influenced by non-economic factors. Treat the regime label as a demand backdrop, not a deterministic forecast.

Academic & primary references (selected)

• Carroll, Fuhrer & Wilcox (1994) — sentiment adds forecasting content for household spending: JSTOR.
• Ludvigson (2004) — overview and evidence that confidence measures contain information about future consumption growth: AEA.
• Surveys of Consumers technical documentation (methodology/release materials): UMich docs.

Practical caveat

Survey results can be revised; high survey volatility means month‑to‑month flips may reflect noise rather than trend. The smoothing, z‑score standardisation, and neutral band are explicit choices to prioritise stability and interpretability.