← All work

PREDICTIVE MODELLING / EXPLAINABLE ML

A predictive model
you can inspect.

An independently built workflow for preparing imbalanced data, explaining stroke classifiers and analysing their results.

CONTEXT
UCL COMP0172 · Healthcare ML project
IMPLEMENTATION
Python · scikit-learn · imbalanced-learn · DALEX
MY CONTRIBUTION
Independent project · Data preparation, model explanation & results analysis

Research case study using saved results. No clinical deployment or diagnostic use is claimed.

WHAT THIS WORK DEMONSTRATES

Build the model.
Make the result understandable.

The problem

Missing values and a small positive class make this more than a simple accuracy exercise. A useful comparison needs a consistent data workflow and visibility into errors.

What was built

I built the workflow end to end, with a focus on data preparation, explaining the models and interpreting their results. The notebook and report connect each stage to inspectable outputs.

Relevant to your project

Predictive modelling, structured model comparison and explainability — the same types of work I can help scope around your data and decision.

5,109prepared dataset rows
4classifier pipelines compared
1,022held-out evaluation rows

THE MODELLING WORKFLOW

Clear stages. Comparable results.

  1. 01

    Prepare the data

    Split training and test data; fit the BMI imputer on training data. Encode categories and standardise numeric inputs.

  2. 02

    Build comparable models

    Use shared preprocessing definitions and training-side random oversampling across four classifier pipelines.

  3. 03

    Evaluate the trade-offs

    Compare saved ROC curves, precision, recall and confusion matrices on the held-out test split.

  4. 04

    Explain model behaviour

    Inspect the classifier using saved permutation-importance and partial-dependence outputs.

The final notebook pipelines use RandomOverSampler. Resampling is a training step; the held-out test set keeps its original class imbalance.

THE SAVED EVALUATION

One score is never the whole picture.

Choose a model to inspect its ROC curve, scores and error counts. These are the project’s saved evaluation results.

MODEL COMPARISON

Inspect the result behind the score.

Saved results · No live prediction

How well does the model separate the classes?

Saved test-set ROC curves
ROC curve — Random ForestRandom Forest has a saved ROC AUC of 0.8484. The teal curve is selected. Other model curves are grey, with a diagonal reference line. Points come directly from saved notebook output.000.250.250.50.50.750.7511False-positive rateTrue-positive rate
Random ForestOther saved modelsDiagonal reference
SELECTED MODEL

Random Forest

ROC AUC
0.8484
Recall
88.7%
Precision
17.5%
F1 score
0.292
Confusion matrix · 1,022 test rows
Actual labelPredicted
negative
Predicted
positive
Negative700260
Positive755

Positive = stroke recorded in the dataset. Counts use the saved classifier decisions.

The trade-off

Random Forest identifies 55 of 62 positive labels, misses 7, and produces 260 false positives on this test split. AUC alone does not tell the whole story.

View all model scores and evaluation conditions
Saved held-out evaluation · one split, no confidence intervals
ModelROC AUCRecallPrecisionF1Accuracy
Random Forest0.848488.7%17.5%0.291873.9%
Logistic Regression0.846280.6%15.8%0.264672.8%
Gradient Boosting0.827071.0%15.9%0.260475.5%
XGBoost0.838782.3%16.1%0.269172.9%

The 1,022-row test split contains 62 positive and 960 negative labels. The split uses an 80:20 hold-out with seed 42 and no stratification. Scores apply to this saved evaluation, rather than new populations.

A calculated majority-class reference would achieve 93.9% accuracy by predicting every row as negative, while missing all 62 positive labels. This is why the comparison includes recall, precision, ROC and the confusion matrix.

EXPLAINABLE ML

Look inside the classifier.

Two complementary views of the Random Forest classifier: which inputs affect its score, and how its average output varies.

Which inputs does the model rely on?

Random Forest · permutation importance
Age+0.214
Average glucose level+0.010
Hypertension+0.009
Marital status+0.009
Smoking status+0.006
BMI+0.005
Work type+0.003
Gender+0.001
Heart disease+0.001
Residence type-0.002
Increase in saved validation loss after shuffling each feature. Larger positive changes indicate more reliance by this model. Signed values are preserved, including small negative changes.

Age has the largest effect on the model’s score in this diagnostic. This describes model behaviour, rather than a causal medical effect.

How does its output change?

Random Forest · partial dependence
Saved partial dependence — Age101 saved points show the Random Forest's average model output across values of age. These are model diagnostics, not calibrated personal risk estimates.00.5104182Age (years)Average model output
Original saved profile points, with no new inference. Correlated inputs can make partial-dependence profiles difficult to interpret; these curves are not clinical risk estimates.
View the saved profile values
Random Forest · Age
Feature valueAverage model output
0.0800.2126
0.8990.2126
1.7180.2026
2.5380.2015
3.3570.2015
4.1760.2015
4.9950.2015
5.8140.2015
6.6340.2015
7.4530.2015
8.2720.2015
9.0910.2015
9.9100.2015
10.7300.2015
11.5490.2015
12.3680.2015
13.1870.2015
14.0060.2022
14.8260.2013
15.6450.2013
16.4640.2013
17.2830.2013
18.1020.2013
18.9220.2013
19.7410.2013
20.5600.2013
21.3790.2013
22.1980.2013
23.0180.2013
23.8370.2013
24.6560.2013
25.4750.2013
26.2940.2013
27.1140.2013
27.9330.2013
28.7520.2013
29.5710.2013
30.3900.2013
31.2100.2013
32.0290.2023
32.8480.2023
33.6670.2023
34.4860.2023
35.3060.2023
36.1250.2023
36.9440.2023
37.7630.2337
38.5820.2295
39.4020.2295
40.2210.2279
41.0400.2280
41.8590.2360
42.6780.2349
43.4980.2349
44.3170.2352
45.1360.2724
45.9550.2731
46.7740.2714
47.5940.3494
48.4130.3499
49.2320.3592
50.0510.3615
50.8700.3628
51.6900.3628
52.5090.3632
53.3280.3633
54.1470.4309
54.9660.4345
55.7860.4477
56.6050.5034
57.4240.5034
58.2430.5040
59.0620.5043
59.8820.5060
60.7010.5046
61.5200.5025
62.3390.5025
63.1580.5209
63.9780.5213
64.7970.5210
65.6160.5235
66.4350.5235
67.2540.5343
68.0740.5728
68.8930.5829
69.7120.5894
70.5310.5944
71.3500.5946
72.1700.5993
72.9890.5992
73.8080.6065
74.6270.6071
75.4460.6071
76.2660.6100
77.0850.6138
77.9040.6147
78.7230.6157
79.5420.6174
80.3620.6174
81.1810.6175
82.0000.6126

METHODS & EVIDENCE

The details, when you need them.

Data preparation and evaluation scope

The source contains 5,110 rows. Removing one row in an unsupported categorical group leaves 5,109 rows and ten input features. The 80:20 split produces 4,087 training rows and 1,022 test rows; the split is not stratified.

BMI imputation is fitted on training data and then applied to test data. One-hot encoding, numeric scaling and training-side random oversampling feed the classifiers. The source compares Logistic Regression, Random Forest, Gradient Boosting and XGBoost.

This case uses one saved test split with only 62 positive labels. It does not establish generalisation to a new population, calibrated risk, clinical effectiveness or validated fairness.

Where the numbers and charts come from

ROC curves are redrawn from the original saved Plotly arrays. AUC is recomputed from those points. Confusion counts are transcribed from report Table 3.2 and checked against the notebook image; precision, recall, F1 and accuracy are recomputed from the counts and checked against the saved performance table.

Feature importance and profile curves come from the saved classifier diagnostic arrays. Only aggregates and diagnostic points are included here, with no individual health records or new prediction requests.

Audited commit: 3bf76140a327. The notebook blob also matches the current remote file checked for this case. No training or inference was rerun.

Inspect the saved notebook ↗Read the project report ↗
Keeping explanations attached to the right model

The importance plot and profiles shown here are attached to the Random Forest classification pipeline in the notebook. The separate SHAP experiment trains a RandomForestRegressor; it is not presented as an explanation of the classifier’s reported performance.

Permutation importance measures reliance by a fitted model. Partial dependence summarises average model behaviour and can be misleading when features are correlated. Neither plot establishes a causal medical effect.

Permutation importance: method reference ↗Partial dependence: interpretation limits ↗
Implementation notes and a reproducible next step

The report discusses SMOTE and other exploratory processing. The final notebook classifier pipelines use RandomOverSampler; outlier-removal calls are commented out, and the shown final pipeline does not execute grid search.

This is saved-output evidence, rather than a verified clean rerun. The notebook uses a local CSV path and has exploratory cell-order dependencies; its historical dependency environment is not locked.

A rebuild would first make the environment and execution order reproducible, then agree the split strategy, decision threshold, calibration and validation appropriate to the actual task.

APPLIED ML & STATISTICAL MODELLING

What could your data help you decide?

Bring the question, the available data and what a useful result would look like.

Discuss an ML Project