Scikit-learn

Overview

This skill provides comprehensive guidance for machine learning tasks using scikit-learn, the industry-standard Python library for classical machine learning. Use this skill for classification, regression, clustering, dimensionality reduction, preprocessing, model evaluation, and building production-ready ML pipelines.

Installation

Targets scikit-learn 1.9.1, verified with Python 3.13. The release requires Python 3.11+; use its published wheels for your interpreter/platform. See the 1.9 release notes. The bundled scripts and regression tests are executable examples. Reference snippets using caller-provided X, y, columns, or placeholders are illustrative adaptations, not complete standalone programs.

Install the PyPI package **scikit-learn** (not the deprecated sklearn package on PyPI). Import in code as sklearn.

# Install scikit-learn using uv
uv pip install "scikit-learn==1.9.1"

# Optional: plotting utilities and bundled script dependencies
uv pip install "scikit-learn[plots]==1.9.1" matplotlib pandas

# Commonly used with
uv pip install pandas numpy

Check your version:

import sklearn
print(sklearn.__version__)

When to Use This Skill

Use the scikit-learn skill when:

• Building classification or regression models
• Performing clustering or dimensionality reduction
• Preprocessing and transforming data for machine learning
• Evaluating model performance with cross-validation
• Tuning hyperparameters with grid or random search
• Creating ML pipelines for production workflows
• Comparing different algorithms for a task
• Working with both structured (tabular) and text data
• Need interpretable, classical machine learning approaches

Quick Start

Classification Example

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report

# Split data
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42
)

# Preprocess
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

# Train model
model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train_scaled, y_train)

# Evaluate
y_pred = model.predict(X_test_scaled)
print(classification_report(y_test, y_pred))

Complete Pipeline with Mixed Data

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn.ensemble import GradientBoostingClassifier

# Define feature types
numeric_features = ['age', 'income']
categorical_features = ['gender', 'occupation']

# Create preprocessing pipelines
numeric_transformer = Pipeline([
    ('imputer', SimpleImputer(strategy='median')),
    ('scaler', StandardScaler())
])

categorical_transformer = Pipeline([
    ('imputer', SimpleImputer(strategy='most_frequent')),
    ('onehot', OneHotEncoder(handle_unknown='ignore'))
])

# Combine transformers
preprocessor = ColumnTransformer([
    ('num', numeric_transformer, numeric_features),
    ('cat', categorical_transformer, categorical_features)
])

# Full pipeline
model = Pipeline([
    ('preprocessor', preprocessor),
    ('classifier', GradientBoostingClassifier(random_state=42))
])

# Fit and predict
model.fit(X_train, y_train)
y_pred = model.predict(X_test)

Core Capabilities

Five capability areas are documented in references/core_capabilities.md, with per-topic detail in references/supervised_learning.md, references/unsupervised_learning.md, references/model_evaluation.md, references/preprocessing.md, and references/pipelines_and_composition.md:

1. Supervised learning — classification and regression estimator families.
2. Unsupervised learning — clustering, decomposition, and manifold learning.
3. Model evaluation and selection — metrics, cross-validation, and hyperparameter search.
4. Data preprocessing — scaling, encoding, imputation, and feature selection.
5. Pipelines and composition — Pipeline and ColumnTransformer.

Always fit preprocessing inside a Pipeline so it is refit per cross-validation fold; scaling or imputing before splitting leaks test information into training.

Two worked workflows are in references/common_workflows.md.

Example Scripts

Run these commands from this skill directory; the clustering demo writes PNGs into the working directory. Its synthetic noise is seeded. The classification script assumes independent rows with enough observations per class for stratified CV; adapt both splits for grouped or temporal data.

Classification Pipeline

Run a complete classification workflow with preprocessing, model comparison, hyperparameter tuning, and evaluation:

uv run --no-project --with scikit-learn==1.9.1 --with pandas --with matplotlib python scripts/classification_pipeline.py

This script demonstrates:

• Handling mixed data types (numeric and categorical)
• Model comparison using stratified cross-validation and balanced accuracy by default
• Hyperparameter tuning with GridSearchCV
• Comprehensive evaluation with multiple metrics
• Impurity feature importances, with their high-cardinality bias made explicit

Clustering Analysis

Perform clustering analysis with algorithm comparison and visualization:

uv run --no-project --with scikit-learn==1.9.1 --with pandas --with matplotlib python scripts/clustering_analysis.py

This script demonstrates:

• Exploring candidate cluster counts (inertia/elbow and silhouette analysis)
• Comparing multiple clustering algorithms (K-Means, DBSCAN, Agglomerative, Gaussian Mixture)
• Reporting undefined metrics for degenerate clusterings and DBSCAN noise coverage
• Assessing internal geometry without treating it as proof of scientific clusters
• Visualizing results with PCA projection

Reference Documentation

This skill includes comprehensive reference files for deep dives into specific topics:

Quick Reference

File: references/quick_reference.md

• Common import patterns and installation instructions
• Quick workflow templates for common tasks
• Algorithm selection cheat sheets
• Common patterns and gotchas
• Performance optimization tips

Supervised Learning

File: references/supervised_learning.md

• Linear models (regression and classification)
• Support Vector Machines
• Decision Trees and ensemble methods
• K-Nearest Neighbors, Naive Bayes, Neural Networks
• Algorithm selection guide

Unsupervised Learning

File: references/unsupervised_learning.md

• All clustering algorithms with parameters and use cases
• Dimensionality reduction techniques
• Outlier and novelty detection
• Gaussian Mixture Models
• Method selection guide

Model Evaluation

File: references/model_evaluation.md

• Cross-validation strategies
• Hyperparameter tuning methods
• Classification, regression, and clustering metrics
• Learning and validation curves
• Best practices for model selection

Preprocessing

File: references/preprocessing.md

• Feature scaling and normalization
• Encoding categorical variables
• Missing value imputation
• Feature engineering techniques
• Custom transformers

Pipelines and Composition

File: references/pipelines_and_composition.md

• Pipeline construction and usage
• ColumnTransformer for mixed data types
• FeatureUnion for parallel transformations
• Complete end-to-end examples
• Best practices

Best Practices

Always Use Pipelines

Pipelines prevent data leakage and ensure consistency:

# Good: Preprocessing in pipeline
pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('model', LogisticRegression())
])

# Bad: Preprocessing outside (can leak information)
X_scaled = StandardScaler().fit_transform(X)

Fit on Training Data Only

Never fit on test data:

# Good
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)  # Only transform

# Bad
scaler = StandardScaler()
X_all_scaled = scaler.fit_transform(np.vstack([X_train, X_test]))

Match the Split to the Independent Unit

For independent classification rows, preserve class distribution as below. For repeated patients, specimens, sites, or related molecules, keep each group entirely in one partition using GroupKFold or StratifiedGroupKFold; class stratification alone does not prevent group leakage. For future prediction, use a chronological split and exclude features unavailable at prediction time. Apply the same grouping/time rule to both inner tuning and outer evaluation. See the cross-validation guide.

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42
)

Set Random State for Reproducibility

model = RandomForestClassifier(n_estimators=100, random_state=42)

Choose Appropriate Metrics

• Balanced data: Accuracy, F1-score
• Imbalanced data: Per-class Precision/Recall, Average Precision, Balanced Accuracy; include prevalence and threshold
• Cost-sensitive: Define custom scorer

Scale Features When Appropriate

Algorithms commonly sensitive to feature scale (scaling changes the modeled geometry):

• SVM, KNN, Neural Networks
• PCA, Linear/Logistic Regression with regularization
• K-Means clustering

Algorithms not requiring scaling:

• Tree-based models (Decision Trees, Random Forest, Gradient Boosting)
• Gaussian Naive Bayes; preserve the nonnegative count/proportion input expected by MultinomialNB

Troubleshooting Common Issues

ConvergenceWarning

Issue: Model didn't converge Solution: Increase max_iter or scale features

model = LogisticRegression(max_iter=1000)

Poor Performance on Test Set

Possible causes: Overfitting, distribution shift, leakage during selection, or an unsuitable metric Solution: Diagnose using training/validation results and the deployment split; do not repeatedly tune on the final test set. Use regularization, cross-validation, or a simpler model as appropriate

# Add regularization
model = Ridge(alpha=1.0)

# Use cross-validation
scores = cross_val_score(model, X, y, cv=5)

Memory Error with Large Datasets

Solution: Use algorithms designed for large data

# Use SGD for large datasets
from sklearn.linear_model import SGDClassifier
model = SGDClassifier()

# Or MiniBatchKMeans for clustering
from sklearn.cluster import MiniBatchKMeans
model = MiniBatchKMeans(n_clusters=8, batch_size=100)

Additional Resources

• Official Documentation: https://scikit-learn.org/stable/
• User Guide: https://scikit-learn.org/stable/user_guide.html
• API Reference: https://scikit-learn.org/stable/api/index.html
• Examples Gallery: https://scikit-learn.org/stable/auto_examples/index.html