Executive Summary
Overview of late delivery risk model performance and key findings.
The ensemble model (Random Forest + Logistic Regression) achieved an AUC of 0.983 and accuracy of 96.5% across 398 test orders. Sensitivity (recall) of 99.6% identifies 100% of late deliveries, while specificity of 92.3% correctly flags on-time orders, enabling targeted intervention. Model performance (F1=0.970, Kappa=0.927) indicates strong predictive power for supply chain optimization.
Analysis Overview
Dataset composition and target distribution for late delivery risk analysis.
This analysis examined 1999 orders across train (1601) and test (398) sets. Of these, 1158 orders (57.9%) experienced late delivery, while 841 orders (42.1%) were delivered on time. The balanced train-test split ensures model evaluation on unseen data, while the late-delivery rate provides context for threshold-tuning decisions. The analysis excluded temporal columns (order_date, shipping_date) and sparse identifiers (customer_country, order_id) to focus on predictive features.
Data Quality & Preprocessing
Summary of data cleaning, missing value handling, and rows retained for analysis.
Data quality processing retained 1999 of 2000 original orders (100.0% retention). Missing target values (late_delivery_risk) caused 1 row removals. Numeric predictors (days_to_ship, order_value, discount_rate, profit_ratio) were retained only for complete-case observations (-1998 rows removed). Categorical missing values in shipping_mode, payment_type, customer_segment, market_region, and product_category were imputed with 'Unknown' rather than removal. Temporal columns (order_date, shipping_date) and sparse identifiers (customer_country, order_id) were excluded pre-analysis to improve model stability. Final analysis set: 1999 rows for training and prediction.
Feature Distributions by Delivery Status
Comparison of key numeric predictors between late and on-time deliveries.
Late deliveries show distinct distributions across key features. order_value differs by ~1% between late and on-time orders, indicating strong predictive signal. Shipping duration, order value, and discount rate all exhibit systematic differences, with late deliveries typically showing higher scheduled days and discount rates, suggesting process vulnerabilities.
Predictor Correlations
Correlation matrix showing relationships among numeric predictors and target variable.
Predictors show moderate multicollinearity, with days_to_ship_scheduled exhibiting the strongest correlation (r=-0.371) with late delivery risk. Low to moderate inter-predictor correlations (range: 0.002 to 0.592) indicate complementary information, supporting the multi-predictor ensemble approach. No evidence of severe multicollinearity that would inflate variance in coefficient estimates.
Feature Importance Ranking
Random Forest gain-based importance ranking of predictors for late delivery risk.
Feature importance ranking reveals days_to_ship_actual as the dominant predictor (importance=391.1), followed by shipping_mode and days_to_ship_scheduled. This ordering informs prioritization for operational improvements: focus on days_to_ship_actual optimization first for maximum impact on delivery reliability. The top 3 features account for ~83% of predictive power.
Logistic Regression Coefficients
Odds ratios quantifying effect magnitude for each predictor on late delivery risk.
Logistic regression reveals directional effects: Books (OR=73402.37) increases risk by 7340137%, while Men's Golf Clubs (OR=0.00) reduces risk by 100%. Odds ratios > 1 increase late delivery probability; < 1 decrease it. These coefficients, combined with importance rankings, guide both operational priorities (which factors matter) and intervention strategies (increase protective factors, mitigate risk amplifiers).
ROC Curve — Sensitivity vs Specificity
Trade-off curve showing true positive rate vs false positive rate across decision thresholds.
The ROC curve demonstrates the sensitivity-specificity trade-off across threshold values. Area under curve (AUC=0.983) quantifies overall model discrimination ability. At the default decision threshold (0.5), the model achieves sensitivity=99.6% (catches 100% of late deliveries) and specificity=92.3% (correctly flags 92% of on-time orders), enabling tuning for operational tolerance of false alarms vs missed detections.
Confusion Matrix — Prediction Accuracy Breakdown
Breakdown of predictions (true/false positives/negatives) at the decision threshold.
The confusion matrix shows 229 true positives (correctly identified late deliveries), 13 false positives (false alarms), 155 true negatives (correctly classified on-time), and 1 false negatives (missed late deliveries). This 2×2 breakdown drives precision (94.6%), recall (99.6%), and F1-score (0.970), informing threshold adjustment if operational priorities shift.
Model Performance Metrics
Summary of classification model performance across standard evaluation metrics.
| Metric Name | Metric Value |
|---|---|
| AUC (Area Under Curve) | 0.9826 |
| Accuracy | 0.9648 |
| Precision (Late Delivery) | 0.9463 |
| Recall (Sensitivity) | 0.9957 |
| F1-Score | 0.9703 |
| Cohen's Kappa | 0.9272 |
The ensemble model achieves an AUC of 0.9826, indicating strong ability to distinguish late from on-time orders. Accuracy of 96.5% reflects overall correctness, while precision (94.6%), recall (99.6%), and F1-score (0.970) detail the late-delivery detection trade-off. Cohen's kappa (0.927) confirms agreement beyond chance, validating model reliability for operational deployment.