Scope: face-engine-staging.aws.facetrustai.com, the real AWS Fargate deployment (2 vCPU / 4 GB, face_engine_desired_count = 2, no autoscaling), tested with k6 sending real JPEG uploads to POST /v1/embed — not a synthetic or local benchmark. See FaceTrust AI API’s load-testing page for how this fits alongside the other two services’ results.

​
Methodology

Staged ramp: 2 → 5 → 10 → 20 → 30 VUs, 30-45s per stage. Each iteration randomly picked between GET /v1/health (33%, cheap) and POST /v1/embed with a real 230KB test JPEG (67%, the actual CPU-bound detection+embedding work).

​
Results

Correctness held up well — only 2 failed embed calls out of 564 (0.35%) even at 30 concurrent VUs of genuine inference work. The problem is latency: median stayed at a reasonable 1.94s, but p95 climbed to 22 seconds, and the single worst request took 59 seconds.

​
The surprising part: CPU never saturated

CloudWatch CPUUtilization for the facetrust-staging-face-engine ECS service during the test window (2 tasks, 2 vCPU each): CPU peaked at under 30%, on a service whose entire job is CPU-bound face detection + embedding. If ONNXRuntime were using the full 2 vCPU allocated per task, sustained concurrent inference should drive CPU well past 30% — this strongly suggests the runtime is not configured to use multiple threads (ONNXRuntime’s intra_op_num_threads/inter_op_num_threads default to conservative values, sometimes effectively 1, unless explicitly set). Memory stayed low too (peak 10.3%), ruling that out as well. Practical implication: the 22-second p95 latency under load is not a “need bigger instances” problem — it’s very likely a single-threaded inference bottleneck that more vCPU wouldn’t fix without also tuning ONNXRuntime’s thread-pool configuration in InferenceEngine.cpp/main.cpp. Horizontally scaling face_engine_desired_count beyond 2 would help in the meantime (more independent single-threaded workers), but the underlying inefficiency — allocating 2 vCPU per task and using a fraction of one — is worth fixing before spending more on compute.

​
What this doesn’t cover

  • Only /v1/embed was load tested, not /v1/compare or /v1/liveness — those exercise different code paths (/compare runs detection+embedding twice, /liveness runs the heuristic PAD checks).
  • Single image reused for every request — no test of decode-time variance across real-world image sizes/formats (the endpoint accepts arbitrary JPEG/PNG).
  • No sustained soak test, only a single ~4-minute ramp up and down.