Stop Fighting Scikit-Learn Pipelines: How Metadata Routing Fixes Sample Weights & Groups

A couple of months ago, I stumbled upon this video by Vincent D. Warmerdam about metadata routing in scikit-learn. I'll be honest, I had no idea what "metadata routing" even meant, but Vincent's explanation completely changed how I think about building ML pipelines.

The video showed me that one of the most frustrating problems in scikit-learn; passing sample weights and groups through complex pipelines finally had an elegant solution. It piqued my curiosity enough that I dove deep into the feature, tested it extensively, and honestly, I was surprised by how little coverage this gets in technical blogs and articles. So I figured, why not write about it myself and share what I learned?

If you've ever struggled with imbalanced datasets, grouped cross-validation, or just wanted to pass custom information through your pipelines, this article is for you. Let's start from the very beginning.

What is "Metadata" in Machine Learning?

Let's start with a concrete example. You're building a credit card fraud detection model with this data:

# Your training data
X = transaction_features  # Amount, merchant, time, location, etc.
y = is_fraud             # 0 = legitimate, 1 = fraud

# But you also have additional information:
sample_weights = [1.0, 1.0, 10.0, 1.0, ...]  # Fraud transactions weighted 10x
customer_ids = [101, 102, 101, 103, ...]      # Which customer made each transaction

Metadata is the "extra information" beyond your features (X) and labels (y):

• sample_weight: How important is each transaction? (Fraud = 10x more important)
• groups: Which customer does each transaction belong to? (For proper cross-validation)
• Custom metadata: Transaction timestamps, confidence scores, data quality flags, etc.

Why Metadata Matters: The Credit Card Fraud Problem

Imagine you're building a fraud detection system for a financial company. You have:

• Imbalanced data: 99% legitimate transactions, 1% fraudulent
• Time-series data: Transactions grouped by customer ID (can't split customers across train/test)
• Complex pipeline: Feature scaling → feature selection → classification
• Business requirement: False negatives (missed fraud) cost 10x more than false positives

The Challenge: Your model needs to:

1. Weight samples - Treat fraudulent transactions as 10x more important during training
2. Respect customer grouping - Keep all transactions from the same customer together during cross-validation (otherwise you're leaking information!)
3. Pass this information through pipelines - Your scaler, feature selector, and classifier all need access to these weights
4. Work with hyperparameter tuning - GridSearchCV needs to use both weights and groups

The problem?

This "metadata" (weights, groups) isn't part of your feature matrix X or labels y. It's auxiliary information that needs to flow through your entire ML pipeline.

Before scikit-learn 1.3, this was nearly impossible. Let's see why.

The Challenge: Why Metadata Routing Was Needed

Prior to metadata routing, you'd face multiple interconnected problems:

Problem 1: Can't Pass Weights Through Pipelines

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

# Your fraud detection pipeline
pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('clf', LogisticRegression())
])

# You have fraud weights (fraudulent transactions weighted 10x)
fraud_weights = np.where(y == 1, 10.0, 1.0)

# This doesn't work!
pipe.fit(X, y, sample_weight=fraud_weights)  # Error: unexpected keyword argument

Problem 2: Can't Use Groups in Cross-Validation

from sklearn.model_selection import cross_val_score, GroupKFold

# You have customer IDs (can't split customers across folds)
customer_groups = df['customer_id'].values

# This doesn't work with pipelines!
scores = cross_val_score(
    pipe, X, y,
    cv=GroupKFold(n_splits=5),
    groups=customer_groups  # Pipeline doesn't know what to do with this
)

Problem 3: Can't Combine Both in GridSearchCV

from sklearn.model_selection import GridSearchCV

# You need BOTH weights AND groups during hyperparameter tuning
grid = GridSearchCV(pipe, param_grid, cv=GroupKFold(n_splits=5))

# This is impossible - can't pass both!
grid.fit(X, y, sample_weight=fraud_weights, groups=customer_groups)  # Doesn't work

So you can begin to see the problem by now. Pipelines had no way to route this metadata to specific components. You'd have to use hacky workarounds like clf__sample_weight, which was inconsistent, broke with nested pipelines, and completely failed with cross-validation.

The Solution: Metadata Routing API

Metadata routing solves ALL three problems at once with a clean, explicit API. Here's how it transforms our fraud detection pipeline:

from sklearn import set_config
from sklearn.model_selection import GridSearchCV, GroupKFold

# Enable metadata routing globally
set_config(enable_metadata_routing=True)

# Build the fraud detection pipeline
pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('clf', LogisticRegression())
])

# Configure metadata routing - declare what each component needs
pipe['clf'].set_fit_request(sample_weight=True)
pipe['clf'].set_score_request(sample_weight=True)

# Problem 1 SOLVED: Pass weights through pipeline
pipe.fit(X, y, sample_weight=fraud_weights)

# Problem 2 SOLVED: Use groups in cross-validation
scores = cross_val_score(
    pipe, X, y,
    cv=GroupKFold(n_splits=5),
    groups=customer_groups  # Works perfectly!
)

# Problem 3 SOLVED: Combine weights AND groups in GridSearchCV
grid = GridSearchCV(pipe, param_grid, cv=GroupKFold(n_splits=5))
grid.fit(X, y, sample_weight=fraud_weights, groups=customer_groups)  # Both work!

print(f"Best model handles imbalance AND respects customer grouping!")

What changed? Each component explicitly declares what metadata it needs using set_*_request() methods. The pipeline then automatically routes metadata to the right places. Simple, explicit, powerful.

Here's what you need to know:

• set_fit_request(): Declares metadata needed during fit()
• set_score_request(): Declares metadata needed during score()
• set_predict_request(): Declares metadata needed during predict()
• Explicit routing: You must explicitly declare what metadata each component receives
• Selective propagation: Metadata is only routed to components that request it - not all components automatically receive it

Important:
The pipeline doesn't pass metadata to every step. Only components that explicitly call set_*_request(metadata=True) will receive that metadata. Components that don't request metadata won't receive it, even if you pass it to the pipeline.

# Example: Selective routing
pipe = Pipeline([
    ('scaler', StandardScaler()),        # Doesn't request sample_weight
    ('clf', LogisticRegression())        # Requests sample_weight
])

pipe['clf'].set_fit_request(sample_weight=True)  # Only clf gets weights

# When you call:
pipe.fit(X, y, sample_weight=weights)

# What happens:
# - scaler.fit(X, y) → NO sample_weight (didn't request it)
# - clf.fit(X_scaled, y, sample_weight=weights) → Gets sample_weight (requested it)

Implementation Guide

Example 1: Custom Transformer with Metadata

Let's build a custom transformer that uses sample weights during fitting. This is useful for weighted feature scaling or selection.

import numpy as np
from sklearn.base import BaseEstimator, TransformerMixin

class WeightedStandardScaler(BaseEstimator, TransformerMixin):
    """StandardScaler that respects sample weights during fitting."""

    def __init__(self):
        self.mean_ = None
        self.std_ = None

    def fit(self, X, y=None, sample_weight=None):
        """Fit scaler using weighted mean and std."""
        if sample_weight is None:
            sample_weight = np.ones(X.shape[0])

        # Normalize weights
        sample_weight = sample_weight / sample_weight.sum()

        # Compute weighted statistics
        self.mean_ = np.average(X, axis=0, weights=sample_weight)
        variance = np.average((X - self.mean_) ** 2, axis=0, weights=sample_weight)
        self.std_ = np.sqrt(variance)

        return self

    def transform(self, X):
        """Transform using fitted statistics."""
        return (X - self.mean_) / self.std_

    def get_metadata_routing(self):
        """Configure metadata routing for this transformer."""
        return (
            super()
            .get_metadata_routing()
            .add_self_request(self)
            .fit(sample_weight=True)  # Request sample_weight in fit()
        )

# Usage
from sklearn import set_config
set_config(enable_metadata_routing=True)

X = np.random.randn(100, 5)
weights = np.random.rand(100)

scaler = WeightedStandardScaler()
X_scaled = scaler.fit_transform(X, sample_weight=weights)

Here's what matters when building custom estimators:

1. Accept sample_weight parameter in fit() method
2. Implement get_metadata_routing() to declare routing requirements
3. Use add_self_request() and chain routing configuration
4. Handle None case when metadata isn't provided

Example 2: Pipeline with Metadata Routing

Now let's use our custom transformer in a pipeline with multiple metadata consumers.

from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

# Create sample data
X = np.random.randn(1000, 10)
y = (X[:, 0] + X[:, 1] > 0).astype(int)
sample_weights = np.random.rand(1000)

X_train, X_test, y_train, y_test, w_train, w_test = train_test_split(
    X, y, sample_weights, test_size=0.2, random_state=42
)

# Build pipeline with metadata routing
pipe = Pipeline([
    ('scaler', WeightedStandardScaler()),
    ('classifier', LogisticRegression(max_iter=1000))
])

# Configure routing: both steps need sample_weight
pipe.set_fit_request(sample_weight=True)
pipe['classifier'].set_fit_request(sample_weight=True)

# Fit with sample weights - they're routed to both steps
pipe.fit(X_train, y_train, sample_weight=w_train)

# Score also supports metadata routing
pipe['classifier'].set_score_request(sample_weight=True)
score = pipe.score(X_test, y_test, sample_weight=w_test)

print(f"Weighted accuracy: {score:.3f}")

Pipeline Routing Rules:

• Each step must explicitly request metadata via set_*_request()
• The pipeline itself can also request metadata to pass through
• Metadata is only routed to steps that request it
• You can route different metadata to different steps

Example 3: GridSearchCV with Metadata

Metadata routing shines in hyperparameter tuning scenarios where you need to pass weights or groups to cross-validation.

from sklearn.model_selection import GridSearchCV
from sklearn.datasets import make_classification

# Generate imbalanced dataset
X, y = make_classification(
    n_samples=1000, n_features=20, n_informative=15,
    n_redundant=5, weights=[0.9, 0.1], random_state=42
)

# Create sample weights to handle imbalance
sample_weights = np.where(y == 1, 10.0, 1.0)

# Build pipeline
pipe = Pipeline([
    ('scaler', WeightedStandardScaler()),
    ('clf', LogisticRegression(max_iter=1000))
])

# Configure metadata routing for both steps
pipe['scaler'].set_fit_request(sample_weight=True)
pipe['clf'].set_fit_request(sample_weight=True)
pipe['clf'].set_score_request(sample_weight=True)

# GridSearchCV with metadata routing
param_grid = {
    'clf__C': [0.1, 1.0, 10.0],
    'clf__penalty': ['l1', 'l2']
}

grid_search = GridSearchCV(
    pipe,
    param_grid,
    cv=5,
    scoring='accuracy',
    n_jobs=-1
)

# Fit with sample weights - they're used in both fitting and scoring
grid_search.fit(X, y, sample_weight=sample_weights)

print(f"Best params: {grid_search.best_params_}")
print(f"Best weighted score: {grid_search.best_score_:.3f}")

# Access the best model
best_pipe = grid_search.best_estimator_

GridSearchCV Routing Features:

• Metadata is automatically passed to all CV folds
• Both fitting and scoring can use metadata
• Works with custom scorers that accept metadata
• Supports groups parameter for GroupKFold and similar splitters

Using Groups for Cross-Validation:

from sklearn.model_selection import GroupKFold

# Create grouped data (e.g., multiple samples per patient)
groups = np.repeat(np.arange(100), 10)  # 100 groups, 10 samples each

# Configure pipeline to use groups
grid_search = GridSearchCV(
    pipe,
    param_grid,
    cv=GroupKFold(n_splits=5),
    n_jobs=-1
)

# Pass groups to ensure they're not split across folds
grid_search.fit(X, y, groups=groups, sample_weight=sample_weights)

Advanced Patterns

Metadata Aliasing: Routing Different Metadata to Different Steps

Sometimes you need to pass different metadata values to different pipeline steps. Metadata aliasing lets you route metadata under different names.

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

# Scenario: You have two types of weights
# - feature_weights: for weighted feature scaling
# - sample_weights: for weighted model training

# Create pipeline
pipe = Pipeline([
    ('scaler', WeightedStandardScaler()),
    ('clf', LogisticRegression())
])

# Configure aliasing: route 'weights' parameter to different metadata
pipe['scaler'].set_fit_request(sample_weight='feature_weights')  # Alias
pipe['clf'].set_fit_request(sample_weight='sample_weights')      # Alias

# Now you can pass both types of weights
pipe.fit(
    X, y,
    feature_weights=feature_importance_weights,  # Goes to scaler
    sample_weights=class_balance_weights         # Goes to classifier
)

Use cases for aliasing:

• Different weighting schemes for different steps
• Passing multiple types of metadata simultaneously
• Avoiding naming conflicts in complex pipelines

Important:
The parameter name you use in fit() must match the alias, not the internal parameter name.

Nested Pipelines: Routing Through Multiple Levels

Metadata routing works seamlessly with nested pipelines, automatically propagating metadata through all levels.

from sklearn.pipeline import Pipeline, FeatureUnion
from sklearn.decomposition import PCA
from sklearn.feature_selection import SelectKBest

# Build nested pipeline: preprocessing pipeline inside main pipeline
preprocessing = Pipeline([
    ('scaler', WeightedStandardScaler()),
    ('features', FeatureUnion([
        ('pca', PCA(n_components=10)),
        ('select', SelectKBest(k=5))
    ]))
])

main_pipe = Pipeline([
    ('preprocess', preprocessing),
    ('clf', LogisticRegression())
])

# Configure routing at any level
main_pipe['preprocess']['scaler'].set_fit_request(sample_weight=True)
main_pipe['clf'].set_fit_request(sample_weight=True)

# Metadata routes through all levels automatically
main_pipe.fit(X, y, sample_weight=weights)

What happens:

1. main_pipe receives sample_weight
2. Routes to preprocessing pipeline
3. preprocessing routes to scaler (which requested it)
4. preprocessing doesn't route to PCA or SelectKBest (didn't request it)
5. main_pipe routes to clf (which requested it)

A few things to remember about nested pipelines:

• Access nested components using chained indexing: pipe['outer']['inner']
• Metadata propagates through all levels automatically
• Each component at any level can independently request metadata
• No special configuration needed for nesting - it just works

Complex example with FeatureUnion:

# FeatureUnion with different metadata needs
feature_union = FeatureUnion([
    ('weighted_pca', WeightedPCA()),      # Needs sample_weight
    ('standard_select', SelectKBest())    # Doesn't need sample_weight
])

pipe = Pipeline([
    ('features', feature_union),
    ('clf', LogisticRegression())
])

# Only weighted_pca gets the weights
pipe['features'].transformer_list[0][1].set_fit_request(sample_weight=True)
pipe['clf'].set_fit_request(sample_weight=True)

pipe.fit(X, y, sample_weight=weights)
# weights go to weighted_pca and clf, but not to standard_select

Best Practices & Common Pitfalls

Best Practices

1. Always Enable Metadata Routing Explicitly

from sklearn import set_config
set_config(enable_metadata_routing=True)

2. Use Descriptive Metadata Names

# Good: clear purpose
estimator.set_fit_request(sample_weight=True, class_prior=True)

# Avoid: generic names
estimator.set_fit_request(metadata=True)

3. Configure Routing at Pipeline Creation

# Configure immediately after creating pipeline
pipe = Pipeline([...])
pipe['step1'].set_fit_request(sample_weight=True)
pipe['step2'].set_fit_request(sample_weight=True)

4. Handle None Gracefully in Custom Estimators

def fit(self, X, y=None, sample_weight=None):
    if sample_weight is None:
        sample_weight = np.ones(len(X))
    # ... rest of implementation

Common Pitfalls

Pitfall 1: Forgetting to Enable Metadata Routing

# This will fail silently or raise errors
pipe.fit(X, y, sample_weight=weights)  # Metadata routing not enabled!

Pitfall 2: Not Configuring All Steps

# Only configured classifier, scaler won't receive weights
pipe['clf'].set_fit_request(sample_weight=True)
pipe.fit(X, y, sample_weight=weights)  # Scaler doesn't get weights!

Pitfall 3: Mixing Old and New APIs

# Don't use both approaches
pipe.fit(X, y, clf__sample_weight=weights)  # Old way
pipe['clf'].set_fit_request(sample_weight=True)  # New way

Pitfall 4: Forgetting to Request Metadata in score()

pipe['clf'].set_fit_request(sample_weight=True)
# Forgot this:
pipe['clf'].set_score_request(sample_weight=True)
pipe.score(X, y, sample_weight=weights)  # Weights ignored in scoring!

Debugging Tips

Check Routing Configuration:

# Inspect what metadata a component requests
print(pipe['clf'].get_metadata_routing())

Verify Metadata is Being Used:

# Add logging to custom estimators
def fit(self, X, y=None, sample_weight=None):
    print(f"Received sample_weight: {sample_weight is not None}")
    # ... rest of implementation

Test with and without Metadata:

# Ensure your estimator works both ways
estimator.fit(X, y)  # Without metadata
estimator.fit(X, y, sample_weight=weights)  # With metadata

Performance Considerations

• Minimal Overhead: Metadata routing adds negligible computational cost
• Memory Efficient: Metadata is passed by reference, not copied
• Parallelization: Works seamlessly with n_jobs in GridSearchCV and similar
• Caching: Compatible with memory parameter in Pipeline for caching

When to Use Metadata Routing

Use metadata routing when you need to:

• Pass sample weights through pipelines
• Use groups for cross-validation with GroupKFold
• Route custom metadata to specific pipeline steps
• Build reusable components that accept auxiliary data
• Maintain clean, explicit code for complex workflows

Don't use metadata routing when:

• You're working with simple, single-estimator workflows
• Your scikit-learn version is < 1.3
• You don't need to pass auxiliary data
• Simple parameter passing suffices

Features vs. Metadata: When to Use Which?

When I first started working with metadata routing, I struggled to clearly demarcate what should be a feature versus what should be metadata. For instance, in the earlier credit card fraud use case we saw, I kept asking myself: "Should customer fraud history be a feature? What about transaction timestamps? Customer IDs?"

The line felt blurry, and I made several mistakes before understanding the distinction. Let me share what I learned, so you can avoid the confusion I went through.

The Key Distinction

Features (X): Information the model uses to make predictions

Metadata: Information about how to train/evaluate the model, but not used for predictions

Examples from Credit Card Fraud Detection

Let's look at some ambiguous cases:

Example 1: Transaction Amount

# Transaction amount as a FEATURE
X = [[100.50, 'online', 'electronics'],  # Amount is a feature
     [25.00, 'store', 'groceries']]
y = [0, 1]  # Fraud labels

# The model learns: "Large electronics purchases online are suspicious"

Decision:

Feature - The model uses amount to predict fraud

Example 2: Historical Fraud Rate for Customer

# Customer's fraud history as METADATA (sample weight)
X = [[100.50, 'online', 'electronics'],
     [25.00, 'store', 'groceries']]
y = [0, 1]

# Customer 1 has 0% fraud history → weight = 1.0
# Customer 2 has 50% fraud history → weight = 5.0 (pay more attention!)
sample_weights = [1.0, 5.0]

Decision:

Metadata - Tells the model "pay more attention to this sample" but isn't used for prediction

But wait! You could also make this a feature:

# Customer fraud history as a FEATURE
X = [[100.50, 'online', 'electronics', 0.0],   # Added fraud_history
     [25.00, 'store', 'groceries', 0.5]]
y = [0, 1]

Decision:

Feature - Now the model learns "customers with high fraud history are risky"

Here's How I Think About It

Ask yourself: "Should the model learn patterns from this, or does it tell the model how to learn?"

The Tricky Case: Information That Could Be Both

Some information genuinely could be either. Here's how to decide:

Case Study: Customer's Previous Fraud Count

Option 1: As a Feature

X = [[100, 'online', 2],  # 2 previous frauds
     [50, 'store', 0]]     # 0 previous frauds

• Model learns: "Customers with previous fraud are high-risk"
• Used for prediction on new data
• Use when: You want the model to learn this pattern

Option 2: As Metadata (Sample Weight)

X = [[100, 'online'],
     [50, 'store']]
sample_weights = [5.0, 1.0]  # Weight based on fraud history

• Tells model: "Pay 5x more attention to samples from risky customers"
• Not used for prediction
• Use when: You want to emphasize certain samples during training

Option 3: Both!

X = [[100, 'online', 2],
     [50, 'store', 0]]
sample_weights = [5.0, 1.0]

• Model learns the pattern AND pays more attention
• Most powerful but can overfit
• Use when: The information is both predictive and indicates importance

Practical Guidelines

Use as Feature when:

• The model should learn patterns from it
• It will be available at prediction time
• You want it to influence predictions directly
• Example: "High-value transactions are riskier"

Use as Metadata when:

• It indicates sample importance (sample_weight)
• It defines data structure (groups for CV)
• It's about the training process, not prediction
• It won't be available at prediction time
• Example: "This sample is from a high-quality data source"

Use as Both when:

• The information is genuinely predictive AND indicates importance
• You have enough data to avoid overfitting
• Example: Customer risk score (predicts fraud AND indicates which samples to emphasize)

Common Mistakes

Mistake 1: Using customer ID as a feature

X = [[101, 100, 'online'],  # Customer ID as feature
     [102, 50, 'store']]

Problem: Model memorizes customers instead of learning patterns. Use as metadata (groups) instead!

Mistake 2: Using sample importance as a feature

X = [[100, 'online', 5.0],  # Importance score as feature
     [50, 'store', 1.0]]

Problem: Importance score won't be available at prediction time. Use as metadata (sample_weight)!

Better Approach: Separate concerns

X = [[100, 'online'],
     [50, 'store']]
sample_weights = [5.0, 1.0]  # Importance
groups = [101, 102]           # Customer IDs

So What's the Takeaway?

Features = What the model learns from

Metadata = How the model learns

When in doubt, ask yourself: "Will this be available when making predictions on new data?" If no, it's probably metadata!

Conclusion

Looking back at everything we've covered, metadata routing really changes the game for building ML pipelines in scikit-learn. No more hacky workarounds with clf__sample_weight or struggling to pass groups through cross-validation. You just declare what each component needs, and the routing system handles the rest. It's cleaner, more explicit, and honestly just makes sense.

What you should remember:

• Enable metadata routing with set_config(enable_metadata_routing=True)
• Use set_*_request() methods to declare metadata requirements
• Configure routing for all pipeline steps that need metadata
• Handle None gracefully in custom estimators
• Test both with and without metadata

Next Steps:

• Explore the official scikit-learn metadata routing guide
• Implement metadata routing in your custom estimators

Author's Note: This article covers scikit-learn 1.3+. The metadata routing API is stable and recommended for all new projects. Legacy parameter passing (e.g., clf__sample_weight) still works but is discouraged.