Uncertainty-Aware QSAR

QSAR models that output only point predictions are insufficient for drug discovery decisions. Uncertainty quantification (UQ) transforms predictions into actionable confidence intervals: "LogD = 2.3 ± 0.4 (90% CI)" is far more useful than "LogD = 2.3".

When to Use This Skill

• Build QSAR models with calibrated prediction intervals (not just point predictions)
• Assess whether a query molecule is within the applicability domain (AD) of the model
• Design active learning loops: prioritize compounds with high epistemic uncertainty
• Rank compounds when model uncertainty is high (don't trust raw predictions alone)
• Regulatory/submission context requiring prediction confidence bounds
• Compare model calibration (is the stated 90% CI actually 90% coverage?)

Uncertainty Types

Type	What it means	How to reduce	Methods
Epistemic	Model doesn't know (lack of training data)	Add more training data	GP variance, ensemble disagreement, MC dropout std
Aleatoric	Intrinsic noise (measurement error)	Can't be reduced	Heteroscedastic models, learned noise σ
Total	Combined uncertainty	—	Epistemic + Aleatoric in prediction

Quick Start — Conformal Prediction (MAPIE)

from mapie.regression import MapieRegressor
from sklearn.ensemble import RandomForestRegressor
import numpy as np

# Fit + calibrate
base_model = RandomForestRegressor(n_estimators=100, random_state=42)
mapie = MapieRegressor(base_model, method="plus", cv=5)
mapie.fit(X_train, y_train)

# Predict with intervals (alpha = desired error rate)
y_pred, y_pi = mapie.predict(X_test, alpha=0.10)  # 90% CI
# y_pi shape: (n_samples, 2, n_alpha)
lower = y_pi[:, 0, 0]
upper = y_pi[:, 1, 0]

# Coverage check
coverage = np.mean((y_test >= lower) & (y_test <= upper))
print(f"Empirical coverage: {coverage:.2%}")  # should be ~90%

Quick Start — Tanimoto GP

import gpytorch
import torch
from rdkit.Chem import rdMolDescriptors

class TanimotoKernel(gpytorch.kernels.Kernel):
    """Tanimoto similarity kernel for binary fingerprints."""
    has_lengthscale = False

    def forward(self, x1, x2, **params):
        x1_sum = x1.sum(-1, keepdim=True)
        x2_sum = x2.sum(-1, keepdim=True)
        dot = torch.matmul(x1, x2.transpose(-1, -2))
        intersection = dot
        union = x1_sum + x2_sum.transpose(-1, -2) - dot
        return intersection / union.clamp(min=1e-8)

# Usage:
# fps = np.array([get_morgan_fp(smi) for smi in smiles_list])
# X = torch.tensor(fps, dtype=torch.float32)
# y = torch.tensor(activities, dtype=torch.float32)
# → see references/gaussian-processes.md for full GP model

Router — What to Read

Task	Reference
Theory: epistemic vs aleatoric, GP kernel validity, conformal coverage proof, AD concepts	references/uncertainty-theory.md
Conformal prediction with MAPIE: split/cross conformal, regression/classification, calibration	references/conformal-prediction.md
GP regression with Tanimoto kernel: GPyTorch, hyperopt, posterior sampling, active learning	references/gaussian-processes.md
Deep uncertainty: MC dropout, deep ensembles, evidential regression, Laplace approximation	references/deep-uncertainty.md
Applicability domain: distance-based AD, Williams plot (leverage), visualization, thresholds	references/applicability-domain.md

Software Stack

Package	Install	Role
MAPIE	pip install mapie	Conformal prediction (classification + regression)
gpytorch	pip install gpytorch	GP models on PyTorch (GPU-ready)
botorch	pip install botorch	Bayesian optimization (uses GPyTorch)
laplace-torch	pip install laplace-torch	Laplace approx for any PyTorch model
torch	pip install torch	Deep ensembles, MC dropout
sklearn	pip install scikit-learn	RF/SVM baselines, calibration
rdkit	conda/pip	Fingerprints (Morgan, MACCS) for features
uncertainty-toolbox	pip install uncertainty-toolbox	Calibration plots, ECE, sharpness

Key Concepts Summary

Concept	Formula / Key Point
Coverage guarantee	P(y ∈ Ĉ(x)) ≥ 1−α for conformal (finite-sample)
Prediction interval	[ŷ − q̂, ŷ + q̂] where q̂ = (1−α) quantile of calibration residuals
GP posterior variance	σ²(x*) = k(x*,x*) − k(x*,X)[K+σ²I]⁻¹k(X,x*)
Ensemble uncertainty	σ²_epistemic = Var{f̂_m(x)} across M models
MC dropout σ	std of T forward passes with dropout active
Leverage h_ii	h = x(XᵀX)⁻¹xᵀ; Williams plot flags h > 3p/n
AD distance	D = min_train Tanimoto; D < threshold → inside AD

Key Pitfalls

• Overconfident RF: RF standard deviation of tree predictions is NOT a calibrated CI — always use conformal on top
• GP scaling: naive GP is O(n³) — for >5000 training points use sparse GP (inducing points) or ExactGP + KeOps
• Conformal requires i.i.d. calibration: if training/calibration sets have distribution shift, coverage guarantee breaks
• MC dropout depth: dropout only captures epistemic if placed at multiple layers; single output layer dropout ≈ ensemble of linear heads only
• AD threshold is task-specific: there is no universal threshold for "inside AD" — always validate on held-out temporal/spatial split

Related Skills

• scientific-skills:scikit-learn — RF/SVM/GBM base models, cross-validation
• scientific-skills:pymc — full Bayesian QSAR (MCMC-based posterior)
• active-learning (upcoming) — using uncertainty for experimental design
• mmpa — MMPA + uncertainty: only apply transforms where model is confident
• generative-design — uncertainty filter on generated molecules (discard low-confidence predictions)