← All work

COMPUTER VISION / ML ENGINEERING

From image labels
to pixel-level models.

A computer vision workflow connecting data preparation, image classification and segmentation evaluation. My focus: the code that makes those stages work together.

PROJECT
UCL Applied Deep Learning · Group project
IMPLEMENTATION
Python · PyTorch · torchvision
MY CONTRIBUTION
Data processing, classifier implementation & the overall evaluation pipeline

THE PROBLEM

Useful vision models start
with the right training signal.

Image labels tell a model what is present. Segmentation asks where it is. The project explores how classifier activations can become pseudo-labels for training a foreground segmentation model.

The engineering task is to connect the data, models and evaluation into a workflow that can be inspected. A fully supervised route provides a reference for comparing the weaker training signal.

Task definitions from the source. Weak training uses pseudo-labels; validation and testing use ground-truth masks.

THE REPORTED RESULTS

Improve the training signal.
Measure what changes.

Three initialisations, baseline and refined pseudo-labels, and a fully supervised reference. The comparison shows the value of refinement and the remaining performance gap.

REPORTED EXPERIMENTS

Compare the training choices.

IMAGENET / REPORT TABLES 1 & 3

Refined Grad-CAM mIoU
0.766
Change from Grad-CAM
+0.132 mIoU
Full-supervision reference
0.869

Refinement improves the reported overlap. The fully supervised reference still scores higher, using pixel-level training labels.

Team project results from the supplied individual report, Tables 1 & 3. Values are reported scores; no training or evaluation was rerun for this page.

Read all scores and the metric definitions

Pixel accuracy counts correctly labelled pixels. mIoU measures region overlap. A large background region can make pixel accuracy look strong while foreground coverage is weaker.

The report uses IoU/mIoU labels. These values are reproduced as reported; raw run logs are unavailable to independently verify the aggregation. The current code’s two-class, batch-level definition is described in the source notes.

Individual report · Table 1 (page 5) and Table 3 (page 6)
InitialisationSegmentation routemIoUPixel accuracyTable
ScratchCAM0.54674.3%1
ScratchGrad-CAM0.49968.6%1
ScratchCAM + refinement0.58975.6%3
ScratchGrad-CAM + refinement0.58575.0%3
ScratchFull supervision0.78889.3%1
MoCoV2CAM0.57074.2%1
MoCoV2Grad-CAM0.55674.0%1
MoCoV2CAM + refinement0.59879.8%3
MoCoV2Grad-CAM + refinement0.62281.2%3
MoCoV2Full supervision0.83691.5%1
ImageNetCAM0.58977.6%1
ImageNetGrad-CAM0.63480.3%1
ImageNetCAM + refinement0.74486.6%3
ImageNetGrad-CAM + refinement0.76687.8%3
ImageNetFull supervision0.86993.2%1

Reported classifier accuracy: Scratch 41.5% · MoCoV2 65.9% · ImageNet 84.2%. This is a separate image-classification task.

Inspect the threshold experiment

One parameter changes the result.

Four reported thresholds with ImageNet initialisation. Grad-CAM reaches its highest listed score at 0.5; CAM reaches its highest listed score at 0.7.

Each marker is one table value. Connecting lines are visual guides between these four settings.

Reported threshold sensitivity — ImageNetCAM and Grad-CAM mIoU at thresholds 0.1, 0.3, 0.5 and 0.7, from report Table 2.0.500.550.600.650.700.10.30.50.7Pseudo-mask thresholdReport mIoU
● CAM● Grad-CAM
Individual report · Table 2, page 5 · ImageNet only
ThresholdCAM mIoUGrad-CAM mIoU
0.10.5610.588
0.30.5860.628
0.50.5890.634
0.70.6040.605

VISUAL EVIDENCE

See what the training mask captures.

The report’s original images connect the method to a visible result. These examples use ImageNet initialisation throughout.

ONE REPORTED EXAMPLE / IMAGENET

Switch methods to inspect how the pseudo-label covers the pet.

Original image — Grad-CAM + refinement, individual report Figure 5

Original image

Pet photograph from the report

Reference mask — Grad-CAM + refinement, individual report Figure 5

Reference mask

Ground-truth foreground

Generated pseudo-label — Grad-CAM + refinement, individual report Figure 5

Generated pseudo-label

Grad-CAM + refinement

Pseudo-label overlay — Grad-CAM + refinement, individual report Figure 5

Pseudo-label overlay

Coverage on the original image

Grad-CAM + refinement · Report Figure 5, page 8 · ImageNet initialisation.
Original embedded figure, displayed as four panels. This report figure shows a pseudo-label used for segmentation training. One example cannot establish general performance.
View the original report figure ↗

Image data: Oxford-IIIT Pet, Parkhi et al. (2012) · CC BY-SA 4.0. Copyright remains with the original image owners. Figure pixels are unchanged; the panel layout is adapted for display.

THE ENGINEERING WORKFLOW

Trace the work from input to evaluation.

Select a stage to see its responsibility, the artifacts it passes on and the supporting implementation.

FOLLOW THE WORKFLOWSource-based walkthrough

01 / MY CONTRIBUTION

Keep the image and its label aligned.

A shared dataset class serves classification, ground-truth segmentation and pseudo-mask training. The split utility stratifies the train/validation partition by breed; training image–mask pairs share crop and flip operations.

INPUT

Images, breed labels and optional masks

OUTPUT

Image tensors and the labels needed by each task

Implementation details

  • Training and validation lists are separate from the test list.
  • Image normalisation and spatial transforms are explicit.
  • Pet, background and boundary trimap values are mapped to two segmentation classes.
Read the complete workflow
Inputs and outputs defined by the source implementation
StageInputOutput
Prepare dataImages, breed labels and optional masksImage tensors and the labels needed by each task
Train classifierPrepared images and breed labelsA classifier checkpoint selected by validation accuracy
Build pseudo-labelsClassifier activations or a class-agnostic modelActivation arrays and binary pseudo-mask files
Train segmentationImages paired with pseudo-labels or ground-truth masksA two-class PSPNet checkpoint
Evaluate outputsA segmentation checkpoint and test images with reference masksMetric JSON and optional prediction visualisations

MY ROLE IN THE TEAM

Where I focused the engineering.

I primarily worked on the code: data processing, the classifier and the overall evaluation pipeline. The mask-generation and segmentation stages are shown as context for the team’s complete system.

01

Data that fits the task

Image loading, breed labels, foreground masks and paired transforms — connected through the dataset layer.

Inspect data handling
02

A configurable classifier

Model construction, training and validation checkpoint selection, with experiment settings carried through the workflow.

Inspect classifier training
03

An inspectable evaluation pipeline

Test-time predictions, explicit metric calculations and visualisation code for comparing predictions with reference masks.

Inspect model evaluation

METHODS & SOURCE

The detail behind the workflow.

Project scope and contribution

This is a group project on Oxford-IIIT Pet. My confirmed responsibilities are code implementation for data processing, classifiers and the overall evaluation pipeline. The full team pipeline also includes CAM-based pseudo-labels, class-agnostic refinement and PSPNet segmentation.

The project combines established model architectures and methods. ResNet, CAM, Grad-CAM, MoCo v2 and PSPNet are not presented as my original algorithms.

Training signals and evaluation boundaries

In weak mode, the segmentation training loader uses generated pseudo-masks. Validation and test evaluation use reference trimaps, mapping the pet label to foreground and both the background and boundary labels to background.

The split utility uses a seeded, per-breed 80:20 train/validation partition. Activation and class-agnostic stages can consume trainval images, so this is not a claim that every model is selected without exposure to those images. The held-out test split remains a distinct evaluation input.

The current repository metric is a sample-weighted average of batch-level two-class mIoU. The report uses IoU/mIoU labels; raw run logs are unavailable to independently confirm its aggregation. Displayed scores retain the report values and precision.

Reading the report alongside the code

Table values determine the displayed results. Table 1 gives CAM a higher IoU than Grad-CAM with Scratch and MoCoV2 initialisation, and Grad-CAM a higher IoU with ImageNet. The method comparison depends on the setting.

The report and current source differ in learning rate, early-stopping description, mask resolution and refinement thresholds. They are shown as related evidence, without claiming that the current checkout exactly reproduces every reported run.

The appendix figures show pseudo-labels. They illustrate mask coverage on one example; the aggregate model scores come from the separate result tables.

Source version and execution scope

This walkthrough follows commit ada679c79864. Report fingerprint: dd60ab4ecfe5. The code walkthrough follows the GitHub implementation. Results come from the supplied COMP0197 Individual Report: Tables 1–2 on page 5, Table 3 on page 6, and Figures 3–6 on page 8. The four figures are extracted directly from the PDF with identical pixel values; this viewer rearranges their panels for smaller screens.

No dataset, checkpoint, training or inference was run for this portfolio page. The source includes configuration and CLI inconsistencies; its clean execution has not been verified. A rebuild would first reconcile the experiment settings and checkpoint paths, then verify the image–mask transforms and evaluation definitions.

Inspect CLI orchestration ↗Inspect the metric definitions ↗

APPLIED ML & STATISTICAL MODELLING

Have an image-based problem to solve?

Start with the task, the images available and how a useful prediction should be evaluated.

Discuss an ML Project