
## The Goal

I needed a small on-device object detector that finds fingertips in photos.

For an iOS app first. Android to follow.

The requirements were simple:

- Run on-device, no server
- Fast enough for real-time use
- Small enough for phones

---

## Why YOLOX-Tiny

YOLOX-Tiny (Apache-2.0) was the right fit.

Small. Fast. Good enough for a two-class problem.

Fine-tuned from pretrained weights - no need to train from scratch.

One important decision: I used **LibreYOLO** (MIT) instead of Ultralytics for training.

The reason is licensing.

Ultralytics is AGPL-3.0. That creates complications in a commercial product.

LibreYOLO is MIT. Clean.

The model runs at about **6 ms per image on a Mac**.

That is only a rough guide for phone speed, but it is a good sign.

---

## The Data

About **1,500 labeled images**:

- 1,335 for training
- 166 for validation

Two classes: **fingertip** (palm side) and **fingernail**.

---

## The Hard Negative Class

The fingernail class deserves its own explanation.

Without it, the model fires on nails.

A fingernail looks similar to a fingertip from the model's perspective - same shape, same proximity.

So I added nails as a **hard negative class**.

The app ignores class 1 entirely. The model just needs to know the difference.

This was one of the biggest gains in accuracy.

---

## Data Collection Notes

Nail images came from public **CC BY 4.0** datasets.

One important rule: **images from the same person stay in the same split**.

This prevents validation scores from being inflated by person-level leakage.

---

## Training Recipe

- Apple Silicon GPU
- 416×416 input
- Batch 16, 100 epochs, early stopping
- SGD with a warmup-plus-cosine learning rate

Heavy augmentation:

- Mosaic
- Mixup
- Color jitter
- Flips and rotation

Augmentation is **switched off for the last 15 epochs**.

This lets the model settle on clean images before evaluation.

The full 100-epoch run took about **37 minutes on a laptop**.

---

## Results

| Iteration | Change | mAP50-95 |
|---|---|---|
| Baseline | Fingertips only | 0.70 |
| Combined | More data | 0.76 |
| v2 | Added the fingernail class, 100 epochs | **0.77** |

mAP50 is about 0.99 - saturated.

So **mAP50-95** (how tight the boxes are) became the number that matters.

One caveat: the validation set is small.

Differences like 0.757 vs 0.773 may be within noise.

---

## What Actually Made the Difference

More data and the hard-negative class gave the main gains.

The architecture stayed the same throughout.

> The model did not need to be bigger. It needed better data.

Consistent box labeling also matters a lot.

Sloppy labels cap mAP50-95 more than the architecture does.

---

## Shipping

- Export to **Core ML** (with NMS baked in) for iOS
- Export to **LiteRT/TFLite** for Android

---

## Final Thoughts

This project reinforced something I keep relearning.

Architecture choices matter less than I expect.

Data quality and thoughtful class design matter more.

The fingernail hard-negative class was a small idea that had a big impact.

Sometimes the fix is not a bigger model - it is a smarter label.


core ml

How the fingertip detector was trained