Avg Latency
1.24s
Avg Throughput
342.5 t/s
Total Tokens
1.2M
Success Rate
98.7%
Latency vs Concurrency
Throughput vs Concurrency
Model Comparison
Raw Data
| Model | Concurrency | Status | Latency (s) | First Token (s) | Tokens | Throughput ↓ |
|---|
Avg Latency
1.24s
Avg Throughput
342.5 t/s
Total Tokens
1.2M
Success Rate
98.7%
| Model | Concurrency | Status | Latency (s) | First Token (s) | Tokens | Throughput ↓ |
|---|