How it works

Give it a picture, get it back with the meme sunglasses on every face in it.

Original picture with several people — Several people, before

Processed picture with several people — After

Group photo: the 2013 class of NASA astronauts, by Robert Markowitz for NASA. Public domain.

Faces are found with YuNet, a small ONNX detector run through OpenCV. If it finds nothing it looks again at the picture doubled in size, then with its contrast equalised. Each face then goes to dlib's 68-point shape predictor for the landmarks the glasses are placed from.

The line between the outer corners of the eyes gives the tilt of the head and the size of the frame; how far the nose sits off that line gives the turn, so the glasses narrow and taper on a face seen from the side. Every point found comes back in the result, as faces. It also finds dogs now and then; cats, never.

Compositing is done with Pillow in a background worker managed by RQ: detecting faces takes long enough that doing it inside the request would block everything else. The page is FastHTML + htmx, the REST API is FastAPI.

webServes the page and the JSON API. Never touches an image.

workerPulls jobs off the queue, fetches and decodes the image, draws the glasses.

redisThe queue, and where results wait to be collected.

API

Documented at /api/docs. Processing happens in the background, so it takes two calls: one to submit the image, one to collect the result.

1Submit an image

POST /api/jobs with either a url or a base64 data URI, not both:

{
  "url": "https://deal-with-it.asrv.click/static/img/apollo_11_crew.jpg"
}

Copied

{
  "base64": "data:image/png;base64,..."
}

Copied

Responds 202 with the job to poll:

{
  "job_id": "9f2c...",
  "state": "queued",
  "status_url": "http://localhost:5000/api/jobs/9f2c..."
}

Copied

2Collect the result

GET the status_url until state is finished or failed:

{
  "job_id": "9f2c...",
  "state": "finished",
  "image": "data:image/png;base64,...",
  "error": null,
  "progress": 100,
  "step": "Done",
  "detection": "plain",
  "faces": [{"box": [101, 420, 402, 197], "score": 0.95,
             "points": {"left_eye": [249.7, 225.3], ...},
             "landmarks": [[199.0, 208.0], ...]}]
}

Copied

state is one of queued, started, finished or failed. A failed job carries a readable error: an unreachable URL, an undecodable image, or no faces found. An unknown or expired job id gives a 404. The result is a JPEG when the input was a JPEG and a PNG otherwise.

Submissions are limited per client and refused with 429 (too many in a minute) or 503 (the queue is full); both carry a Retry-After header.

A queued job also carries ahead: how many jobs are in front of it, with 0 meaning it is next. It goes null once a worker picks the job up, which is when progress and step take over.

While a job runs, progress and step report the stage it has reached. They are checkpoints rather than measurements: neither the detector nor Pillow reports how far through it is.

10Fetching the image

35Looking for faces

75Drawing glasses on 8 faces

90Encoding the result

100Done

Health

GET /api/health reports whether the broker is reachable. The status pill in the header reads it every 30 seconds.

{ "status": "ok", "redis": true }

Copied

License

MIT License. Created by Eric Magalhães.