Return residualization

resid builds a dated equity universe, computes factor exposures, estimates daily factor returns, and exports the unexplained component as residual return.

Walkthrough Design Setup

Setup

Install mise, clone marcap next to this repository, then install Python 3.14 and the locked dependencies:

git clone https://github.com/FinanceData/marcap.git ../marcap
mise install
uv sync

CLI

uv run resid ../marcap/data --config resid.toml

The positional data path is required. CLI values override matching TOML fields.

A compact configuration has this shape:

[run]
years = 10

[universe]
method = "daily"
size = 1000

[factors]
order = ["MKT", "SIZE", "HML", "MOM"]

[regression]
method = "sqrt-cap-wls"

The checked-in resid.toml adds the required factor windows, output directory, and validation settings.

See uv run resid --help for all overrides.

Outputs

The CSV files use standard factor-model notation:

Notation	File	Contents	Schema
ε	epsilon.csv	Residual returns	(date, ticker, specific_return)
r	r.csv	Returns used by the regression	(date, ticker, return)
X	X.csv	Normalized exposures	(date, ticker, <one column per factor>)
f	f.csv	Daily factor returns	(date, <one column per factor>)

epsilon.csv is the primary output. The other files support reconstruction and diagnosis. X and f store INTERCEPT, MKT, SIZE, HML, and MOM in removal order, with r = Xf + ε; sequential X columns use a weighted orthogonal basis.

Python API

from pathlib import Path

import resid

source = resid.MarcapDataSource(Path("../marcap/data"))
base_model = resid.CharacteristicFactorModel(
    (
        resid.size_factor(name="SIZE"),
        resid.long_term_reversal_factor(
            lookback_days=1008,
            skip_days=252,
            min_periods=756,
            name="HML",
        ),
        resid.momentum_factor(
            lookback_days=189,
            skip_days=63,
            min_periods=126,
            name="MOM",
        ),
    )
)
factor_model = resid.RecursiveMarketBetaModel(
    base=base_model,
    lookback_days=252,
    min_periods=126,
    decay=0.97,
    name="MKT",
)

result = resid.run_pipeline(
    window=resid.analysis_window(source, years=10, end=None),
    source=source,
    universe_builder=resid.DailyTopMarketCapUniverse(size=1000),
    return_calculator=resid.PercentageReturns(),
    factor_model=factor_model,
    residualizer=resid.SequentialWLSResidualizer(
        factor_order=("MKT", "SIZE", "HML", "MOM"),
        winsor_quantile=0.01,
        unscaled_factors=("MKT",),
    ),
    regression_weight_model=resid.SquareRootMarketCapWeights(),
    validations=(
        resid.RegressionCoverageValidation(minimum_date_coverage=0.99),
        resid.FiniteOutputValidation(),
        resid.ReturnReconstructionValidation(absolute_tolerance=1e-12),
        resid.ResidualOrthogonalityValidation(absolute_tolerance=1e-12),
        resid.SequentialOrthogonalityValidation(absolute_tolerance=1e-12),
    ),
)
resid.CsvArtifactWriter().write(result, Path("output/full"))

Key choices:

• Factor order sets attribution priority; changing it changes the model.
• Sequential WLS and joint OLS are available behind the same residualizer boundary.

Contracts

Stage output	Contract
Market data	DataFrame indexed by (date, ticker) with return_percent and market_cap
Universe	Boolean Series indexed by (date, ticker)
Returns	Series named return, indexed by (date, ticker)
Exposures	DataFrame indexed by (date, ticker), one column per factor
Regression weights	Positive Series indexed by (date, ticker)
Result	ResidualizationResult with ε, r, X, f, weights, diagnostics, universe, and validation results