Benchmarks
Measured by us on 2026-11-03, one run each, on one 16 GB GPU worker. Treat these as a guide and test on your own workload.
qwen3-1.7b-abliterated
| Measure | Result |
|---|---|
| First request, starting a worker | 163 s |
| Warm request, to the full short reply | 1,119 ms |
| 1 client at once | 64 tokens/s |
| 8 clients at once | 406 tokens/s |
| 32 clients at once | 1,393 tokens/s |
| Refusals on mlabonne/harmful_behaviors | 0 of 100 |
qwen2.5-0.5b-abliterated
| Measure | Result |
|---|---|
| First request, starting a worker | 219 s |
| Warm request, to the full short reply | 1,110 ms |
| 1 client at once | 93 tokens/s |
| 8 clients at once | 618 tokens/s |
| 32 clients at once | 1,466 tokens/s |
| Refusals on mlabonne/harmful_behaviors | 1 of 100 |
Policy checks
Every policy check runs on qwen3-1.7b-abliterated. These are small sets of examples we wrote ourselves, and we chose the wording of the checks using those same examples, so expect lower results on real text and test your own rules on your own text. Thresholds are 0.5 for customer rules and 0.7 for the platform rule. For the platform rule this measures the model reading text the scan has sent it: wording the scan doesn't recognise never reaches the model.
| Rules | Examples | Correct | Matches caught | Fine text wrongly flagged |
|---|---|---|---|---|
| 9 sample customer rules | 52 | 98.1% | 96.2% | 0% |
| The platform rule | 32 | 100% | 100% | 0% |
How we measured
Throughput is 128-token replies at each number of clients at once. Latency was measured from a machine in India, so it includes that network delay, and start times vary with how soon a GPU is free. Refusals are counted from the first 24 tokens of each reply to the test split of mlabonne/harmful_behaviors, by matching refusal phrases.