Understanding p50 vs p99 Latency for AI Workloads

• Blog
• Docs
• Careers
• Get Support
• Contact Sales

• Products Featured AI Products Compute Build, deploy, and scale cloud compute resources Containers and Images Safely store and manage containers and backups Managed Databases Fully managed resources running popular database engines Management and Dev Tools Control infrastructure and gather insights Networking Secure and control traffic to apps Security Help protect your account and resources with these security features Storage Store and access any amount of data reliably in the cloud Browse all products
• Solutions AI/ML CMS Data and IoT Developer Tools Gaming and Media Hosting Security and Networking Startups and SMBs Web and App Platforms See all solutions
• Developers Community Documentation Developer Tools Get Involved Utilities and Help
• Partners Become a Partner Marketplace
• Pricing

• Log in Log in to:
• Community
• DigitalOcean

• Sign up for:
• Community
• DigitalOcean

• Log in Log in to:
• Community
• DigitalOcean

• Sign up for:
• Community
• DigitalOcean

• Tutorials
• Questions
• Product Docs
• Search Community

Contents Overview

Written by Anish Singh Walia

Senior Technical Content Strategist and Team Lead

Most published benchmarks for inference present a singular statistic: the median time for the first token or peak tokens-per-second. While these figures are accurate, they often reflect conditions that favor the provider, such as low concurrency and warm instances, which can misrepresent actual latency experienced in production.

This article presents two main arguments, with the latter being a novel insight. For individual interactive requests, the distribution of latency is more significant than the median; the p95 and p99 metrics reflect user experiences better than the median, which only accounts for the requests that go unnoticed.

The second argument is that for an agent executing a series of sequential model calls to complete a task, analyzing per-call latency alone can be misleading. The total time to complete a task is the cumulative latency across all calls, and the likelihood of encountering at least one latency spike increases with each additional call.

In this article, I will discuss how to accurately interpret percentiles, break down latency metrics according to workload requirements, the impact of agent chains on analysis, a critical checklist for evaluating published benchmarks, and a testing protocol I utilized with DigitalOcean’s Serverless Inference, including all measured data and scripts for your own testing.

Terminology Explained

If you're new to latency analysis, take a quick look and refer back as needed.

The only formula used in this article is P = 1 − (1 − q)^n, which calculates the probability that at least one of n calls experiences a tail event, given that each call has an independent chance q of doing so. This will be explained in detail when it first appears in the section about agent multipliers.

• A clean median with a bad tail is not a rounding error. It is a fixed fraction of your traffic having a bad experience, every day, by construction. At 10,000 daily requests, a 1 in 100 tail event means 100 users hit it daily, regardless of how good the median looks. This is a mathematical identity, not a benchmark claim; I verified it by direct calculation and show the formula in the section below rather than asking you to take it on faith.
• For a single request, three different metrics matter for three different workloads. Time to first token for streaming chat UX, inter-token latency for voice and long streamed output, and total completion time for agents and batch jobs. Definitions per Artificial Analysis’s published methodology . Naming “the fastest provider” without naming which of these three you mean is not a complete claim.
• For agent pipelines, the math changes. A chain of sequential calls sums their latencies, and the probability that at least one call lands in the tail rises with chain length. I checked a common claim about how quickly this reaches near-certainty and found the numbers do not support it: at a 1 percent per-call tail rate, a 10-call chain gives 9.6 percent, not near-certainty, verified by direct calculation and shown in full in the section below. The corrected math is still a real, material risk, just not the risk as commonly overstated.
• A model that wins the headline benchmark can lose the task, and I measured this happening rather than simulating it. On DigitalOcean Serverless Inference on July 27, 2026, Llama 4 Maverick won the single-call time-to-first-token comparison against GPT-OSS 120B by 4 times (300 ms median versus 1,202 ms). On the measured 10-call agent chain, the ranking inverted: GPT-OSS 120B finished the task in 20.9 seconds at the median against Maverick’s 26.7 seconds, and Maverick’s worst measured chain took over 20 minutes because one call’s stream stalled mid-generation.
• A repeatable test protocol beats trusting any single published number , including the ones in this article. I ran the full protocol on three models and a bounded follow-up on a fourth, more than 1,500 recorded requests from a GPU droplet in the same cloud, and every number, every caveat, and the exact script are included below so you can rerun it against your own traffic shape. The complete evidence base, harness, raw per-request data, console logs, analysis code, and charts, is public in the serverless-inference-tail-latency-study GitHub repository I created for this research.

The User Experience: Understanding Tail Metrics

To state it plainly: if your median time to first token is a favorable 300 milliseconds, but your p99 is 4 seconds, serving 10,000 requests daily means around 100 requests will experience waits exceeding ten times the median. This isn't just noise; a percentile is a defined metric, not a measurement fluke.

For a single request, this already warrants looking beyond the median. In a session with multiple requests, the exposure to latency issues compounds, as each request has a 1 percent chance of hitting a tail event.

The probability of experiencing at least one tail event during a session of 20 requests is:

P(at least one tail event) = 1 − (1 − 0.01)^20 ≈ 18.2%

This was verified through direct calculation, indicating that nearly one in five sessions of that length will include at least one problematic request, despite any single request having only a 1 percent chance of failure.

Understanding the Divergence Between p50 and p99 in Inference

The divergence has specific, concrete causes, each thoroughly discussed in a companion piece on inference architecture. In summary:

When the Median is the Appropriate Metric

High-volume, latency-tolerant batch workloads, evaluation suites, offline summarization, synthetic data generation, and large-scale content moderation prioritize throughput and cost per token over individual request wait times, as no user is monitoring the process. DigitalOcean’s batch inference explicitly separates this traffic from real-time serving.

Selecting the Right Metric for Your Workload: TTFT, Inter-token Latency, and Total Time

Latency is not a singular figure; it can be categorized into three distinct measurements that are relevant to different workloads.

Time to first token (TTFT) measures the delay from sending a request to receiving the first output token. This metric is crucial for chat interfaces, as it reflects the user's wait time before any output is displayed.

Inter-token latency, or time per output token, is the interval between subsequent tokens during generation. This is critical for voice interfaces and extended streamed responses, where the audience consumes tokens continuously.

Total completion time combines TTFT with the overall generation duration and is the key metric when the output isn't being observed, such as in batch jobs or intermediary calls within an agent pipeline.

Three distinct metrics for three different workloads mean that a provider optimized for one may not excel in the others.

There is No Universal Fastest Provider

Asking which provider offers the fastest TTFT lacks specificity unless you also clarify the concurrency, prompt length, and output length, as rankings can shift based on these variables.

This isn't a mere caution; it can be verified against published comparisons that differentiate speed based on input token counts.

The variation among providers serving the same model is substantial enough that the term 'fastest provider' becomes meaningless without context. Reports show significant differences in output speed for the same models.

The model may be identical, but the provider is not, and the disparity between the best and worst provider far exceeds any differences between competing models.

The Streaming Illusion

A provider might excel in TTFT but falter in the metric that truly determines user satisfaction regarding completion time. A quick first token followed by slow inter-token latency can create a poor user experience for lengthy responses.

The Agent Multiplier: Why Task Completion Time is the Key Metric

In this section, I'll ensure the math is precise, showcasing how to derive conclusions rather than merely asserting them.

Agent tasks usually require multiple model calls. A typical workflow involves several sequential calls, and task latency is the cumulative effect of all calls in the sequence.

Revisiting the Claim About Tail Probability in a Chain

Here’s a claim I want to verify: that a chain of 10 calls with a per-call p99 tail rate leads to a near-certainty of experiencing a tail event.

Using the same formula as before, with q as the per-call probability of a tail event and n as the number of calls:

P(at least one tail event in the chain) = 1 − (1 − q)^n

With a 1 percent per-call tail rate, a 10-call chain results in:

1 − (0.99)^10 ≈ 9.6%

This represents a significant risk, about ten times higher than any single call's 1 percent rate, but it does not equate to 'near-certainty'.

The accurate interpretation is that agent chains increase the likelihood of encountering tail events, but they do not guarantee them.

Longer chains increase the possibility of hitting a tail event, but the rate of occurrence is highly dependent on which percentile is defined as a tail event.

Demonstrating the Key Point: Benchmark Winners May Not Complete Tasks Efficiently

The essential insight is not merely about the risk of a single poor call but rather the total time across a chain when the model that excels in single-call benchmarks doesn't finish tasks the fastest.

I executed the complete testing protocol against actual models on DigitalOcean Serverless Inference.

Two models illustrate this point clearly. In the benchmark comparison, Llama 4 Maverick outperformed GPT-OSS 120B in TTFT, but when both models were subjected to the same agent workload, the ranking reversed.

The model that felt faster in initial token delivery ended up being slower in task completion compared to its competitor.

The charts illustrate the contrast between benchmark results and user experience.

Each chart panel presents the same question, “Which model is faster?” but evaluates different aspects of the performance.

The worst-case scenario in the measured data encapsulates the argument about tail latency succinctly.

Task completion time grows with the length of the chain, highlighting the importance of measuring real-world performance.

The chart indicates that as the number of sequential calls increases, the overall task duration also increases, emphasizing the significance of the model choice.

Tail Events Slow Tasks and Increase Costs

A tail event in an agent chain incurs costs beyond just delays, as retries can lead to paying for both the original and the retry tokens.

The stalled stream in the data exemplifies the type of request that triggers production timeouts.

DigitalOcean’s Inference Router can automatically switch to a faster model instead of resending the same slow request, optimizing both latency and costs.

Critically Evaluating Latency Benchmarks

Utilize these seven questions to assess any published latency claims critically.

• What percentile is reported? A median-only claim tells you about the typical request and nothing about the tail.
• What concurrency was the test run at? A number measured at concurrency one describes a best case that queueing under real load will not preserve.
• What prompt and output lengths were used? TTFT scales with prompt length because prefill is compute-bound, and total time scales with output length.
• Was the instance warm or cold? A cold-start-inclusive benchmark and a warm-instance-only benchmark answer different questions.
• What time window and duration was measured? A single burst of requests during off-peak hours can look nothing like sustained production traffic; DigitalOcean’s own consistency testing found that benchmarks run during business hours can look artificially good because other users have kept a model warm.
• Was streaming or non-streaming measurement used? TTFT is undefined without streaming.
• Was the measurement taken from a location that includes or excludes real network path? A benchmark run from infrastructure adjacent to the provider will understate the latency a real client experiences.

A latency claim that lacks any of these seven details is incomplete and requires further investigation.

Applying the Checklist Without Cynicism

A recent study highlighted the potential bias in benchmarking tools, emphasizing the importance of reproducible methodologies.

Defining Predictable Latency Properly

Companies that prioritize predictable latency have built architectures designed to minimize variability.

Tail latency is a critical concern for businesses that rely on it, with some reporting significant improvements in production metrics.

Executing the DIY Latency Test Protocol

This section outlines the design of the test and the results obtained.

Conducted under controlled variables to isolate the impact of traffic shape and concurrency.

• Two time windows. Run the full concurrency sweep once during business hours and once overnight or on a weekend, since cold-start behavior is worst when background traffic to a model is lowest and a daytime-only test can look artificially good.
• A chained multi-call test. Construct a 10-call sequential chain that mimics an agent loop, each call’s output feeding the next call’s input, and run at least 30 independent chains through it. Record the total completion time for each full chain, not just each call, and compute the chain-level p50, p95, and p99 directly from those 30 totals, rather than trying to reconstruct them from the per-call numbers.

What to Measure and Analyze

For each test cell: track median TTFT, p95 TTFT, p99 TTFT, and the p99:p50 ratio.

Two complementary tests reveal how queueing affects performance under load.

Checklist for Running Tests on DigitalOcean’s Serverless Inference

Every step in this checklist corresponds to actions taken during the testing process, ensuring accuracy.

#!/usr/bin/env python3 """ Latency benchmark for DigitalOcean Serverless Inference. Measures, per request: - TTFT (time to first streamed token), timestamped client-side - total completion time - prompt and completion token counts, from the usage object Workloads: 1. baseline_sweep - N requests at concurrency 1, 5, and 20 2. chained_session - M independent chains of `chain_length` sequential calls, each call&apos;s output feeding the next call&apos;s input, with total chain time recorded per chain Concurrency is implemented with a real OS-thread pool (ThreadPoolExecutor), not a single-process asyncio loop. This is a deliberate choice: a 2026 measurement-bias paper (Chandrasekar and Kramberger, arXiv, cited in this article) shows that single-process asyncio benchmarking clients can hit their own client-side queueing bottleneck under concurrency, which inflates the very TTFT numbers you are trying to measure. Real OS threads release the GIL during blocking network I/O, which avoids that specific failure mode at the modest concurrency levels used here. Run: export DO_MODEL_ACCESS_KEY=your_key_here python3 do_latency_bench.py --model <model-id> --out results_daytime.json python3 do_latency_bench.py --model <model-id> --out results_overnight.json """ import argparse import concurrent . futures import json import os import time import urllib . request def percentile ( sorted_vals , pct ) : """Linear-interpolation percentile, no external dependency required.""" if not sorted_vals : return None k = ( len ( sorted_vals ) - 1 ) * ( pct / 100 ) f = int ( k ) c = min ( f + 1 , len ( sorted_vals ) - 1 ) if f == c : return sorted_vals [ f ] return sorted_vals [ f ] * ( c - k ) + sorted_vals [ c ] * ( k - f ) def summarize ( ttfts_ms ) : s = sorted ( t for t in ttfts_ms if t is not None ) if not s : return { "n" : 0 } p50 = percentile ( s , 50 ) p95 = percentile ( s , 95 ) p99 = percentile ( s , 99 ) return { "n" : len ( s ) , "p50_ms" : round ( p50 , 1 ) , "p95_ms" : round ( p95 , 1 ) , "p99_ms" : round ( p99 , 1 ) , "p99_to_p50_ratio" : round ( p99 / p50 , 2 ) if p50 else None , } def streamed_request ( base_url , api_key , model , messages , max_tokens = 64 ) : """Send one streaming chat completion. Return TTFT and total time in ms.""" payload = { "model" : model , "messages" : messages , "max_tokens" : max_tokens , "temperature" : 0 , "stream" : True , "stream_options" : { "include_usage" : True } , } req = urllib . request . Request ( base_url + "/chat/completions" , data = json . dumps ( payload ) . encode ( ) , headers = { "Content-Type" : "application/json" , "Authorization" : f"Bearer { api_key } " , } , ) start = time . perf_counter ( ) ttft = None content_parts = [ ] usage = { } try : with urllib . request . urlopen ( req , timeout = 120 ) as r : for raw in r : line = raw . decode ( errors = "ignore" ) . strip ( ) if not line . startswith ( "data:" ) : continue body = line [ 5 : ] . strip ( ) if body == "[DONE]" : break chunk = json . loads ( body ) if chunk . get ( "usage" ) : usage = chunk [ "usage" ] for choice in chunk . get ( "choices" , [ ] ) : delta = choice . get ( "delta" , { } ) if delta . get ( "content" ) : if ttft is None : ttft = ( time . perf_counter ( ) - start ) * 1000 content_parts . append ( delta [ "content" ] ) except Exception as e : return { "error" : str ( e ) , "ttft_ms" : None , "total_ms" : None } total = ( time . perf_counter ( ) - start ) * 1000 return { "ttft_ms" : round ( ttft , 1 ) if ttft is not None else None , "total_ms" : round ( total , 1 ) , "prompt_tokens" : usage . get ( "prompt_tokens" ) , "completion_tokens" : usage . get ( "completion_tokens" ) , "content" : "" . join ( content_parts ) , } def baseline_sweep ( base_url , api_key , model , n_requests , concurrency_levels , prompt ) : results = { } messages = [ { "role" : "user" , "content" : prompt } ] for c in concurrency_levels : print ( f"[baseline] concurrency= { c } , { n_requests } requests" , flush = True ) records = [ ] with concurrent . futures . ThreadPoolExecutor ( max_workers = c ) as pool : futures = [ pool . submit ( streamed_request , base_url , api_key , model , messages ) for _ in range ( n_requests ) ] for f in concurrent . futures . as_completed ( futures ) : records . append ( f . result ( ) ) ttfts = [ r . get ( "ttft_ms" ) for r in records ] results [ f"concurrency_ { c } " ] = { "summary" : summarize ( ttfts ) , "records" : records , } return results def chained_session ( base_url , api_key , model , n_chains , chain_length , prompt_prefix ) : chains = [ ] for i in range ( n_chains ) : history = [ { "role" : "user" , "content" : f" { prompt_prefix } Chain { i } , step 0." } ] start = time . perf_counter ( ) calls = [ ] for step in range ( chain_length ) : r = streamed_request ( base_url , api_key , model , history ) calls . append ( r ) history . append ( { "role" : "assistant" , "content" : r . get ( "content" ) or "ok" } ) history . append ( { "role" : "user" , "content" : f"Chain { i } , step { step + 1 } . Continue." } ) total_chain_ms = ( time . perf_counter ( ) - start ) * 1000 chains . append ( { "chain_index" : i , "total_chain_ms" : round ( total_chain_ms , 1 ) , "calls" : calls } ) print ( f"[chained] chain { i } done: { total_chain_ms : .0f } ms total" , flush = True ) chain_totals = sorted ( c [ "total_chain_ms" ] for c in chains ) return { "chain_summary" : { "n_chains" : len ( chain_totals ) , "p50_ms" : round ( percentile ( chain_totals , 50 ) , 1 ) , "p95_ms" : round ( percentile ( chain_totals , 95 ) , 1 ) , "p99_ms" : round ( percentile ( chain_totals , 99 ) , 1 ) , } , "chains" : chains , } def main ( ) : ap = argparse . ArgumentParser ( ) ap . add_argument ( "--base-url" , default = "https://inference.do-ai.run/v1" ) ap . add_argument ( "--model" , required = True , help = "Exact model ID from GET /v1/models" ) ap . add_argument ( "--n-requests" , type = int , default = 75 ) ap . add_argument ( "--n-chains" , type = int , default = 30 ) ap . add_argument ( "--chain-length" , type = int , default = 10 ) ap . add_argument ( "--out" , default = "latency_bench_results.json" ) args = ap . parse_args ( ) api_key = os . environ . get ( "DO_MODEL_ACCESS_KEY" ) if not api_key : raise SystemExit ( "Set DO_MODEL_ACCESS_KEY before running." ) results = { "model" : args . model , "timestamp" : time . strftime ( "%Y-%m-%dT%H:%M:%SZ" , time . gmtime ( ) ) , } print ( "== Baseline sweep ==" , flush = True ) results [ "baseline_sweep" ] = baseline_sweep ( args . base_url , api_key , args . model , args . n_requests , concurrency_levels = [ 1 , 5 , 20 ] , prompt = "Reply with a two-sentence summary of why caching matters for LLM inference." , ) print ( "== Chained session ==" , flush = True ) results [ "chained_session" ] = chained_session ( args . base_url , api_key , args . model , args . n_chains , args . chain_length , prompt_prefix = "You are debugging a production incident." , ) with open ( args . out , "w" ) as f : json . dump ( results , f , indent = 2 ) print ( f"Wrote { args . out } " , flush = True ) if __name__ == "__main__" : main ( )

Execute the protocol as outlined, ensuring all raw data is captured during the run.

Conditions of the Test Execution

The following details outline the specific conditions under which the tests were conducted.

• Client: a single GPU Droplet ( gpu-h200x1-141gb , 24 vCPUs) in DigitalOcean’s NYC2 region. The network path is cloud-internal and short, which means these numbers could slightly understate what a client on the public internet would see. But the flip side is that this is also the realistic case for one class of reader: if your application already runs on DigitalOcean next to the inference endpoint, the short path measured here is the path your production traffic actually takes, with no cross-cloud hop or egress in between. The Droplet’s GPU is irrelevant to the test; it was simply the machine available.
• Endpoint and workload: https://inference.do-ai.run/v1/chat/completions , streaming enabled, temperature 0, max_tokens 64, the same short prompt for every baseline request, exactly as the script above sends it.
• Volume: the full protocol per model is 225 baseline requests (75 each at concurrency 1, 5, and 20) plus 30 independent 10-call chains, or 525 requests per model. Three models completed it: Mistral 3 14B, GPT-OSS 120B, and Llama 4 Maverick, for 1,575 recorded requests, with zero failed requests.
• Time window: I performend the test only for one window, 08:39 to 10:53 UTC on Monday, July 27, 2026, which is 4:39 to 6:53 a.m. US Eastern, a low-traffic period. The protocol calls for a second, contrasting window, and this run does not have one; treat every number here as a single-window measurement.
• One measurement caveat: the harness defines TTFT as time to first content token. A small number of GPT-OSS 120B responses (3 of 75 at concurrency 1, 10 of 75 at concurrency 5, 1 of 75 at concurrency 20) spent their entire 64-token budget on internal reasoning tokens and produced no content, so they have a total time but no TTFT, and the TTFT rows for that model are computed on the remaining requests.

Measured Results Overview

Time to first token results categorized by concurrency level.

Total completion time for individual calls at concurrency level 1.

Results from the measured 10-call agent chains, analyzing total task time.

Even after excluding the worst-case scenario, the remaining chains still show slower completion times.

The shape of the data tells a more compelling story than the starting point.

Interpreting the Charts: Understanding the Distribution of Results

The charts illustrate the performance of different models under varying conditions.

Medians remain stable as concurrency increases, while tail metrics fluctuate significantly.

Understanding the implications of the data visualizations for model performance.

Mistral 3 14B leads in performance across metrics, with a clear distinction in completion times.

The differences in model performance can often be attributed to their size and processing characteristics.

The Fourth Model: Unique Findings

I aimed to run the same protocol on a different model, with specific measures to prevent stalling.

The results indicate significant variability in performance across different models.

The stark contrast in performance metrics underscores the importance of total completion time.

Interpreting Findings: Understanding Patterns in Latency Data

Identifying patterns in latency data can reveal underlying issues affecting performance.

Mitigation strategies vary based on the identified patterns, emphasizing the need for tailored solutions.

Framework for Decision Making

Evaluate on p95 and p99 for interactive workloads, focusing on user experience.

For agent pipelines, assess task completion time and derive per-call requirements accordingly.

For batch workloads, prioritize throughput and cost, minimizing the impact of tail latency.

When routing requests, match latency-sensitive paths to their p99 profiles.

Choosing the right metrics is crucial for aligning with workload characteristics.

Verification of Claims in This Article

The calculations and measurements presented are based on direct testing and can be replicated.

Frequently Asked Questions on This Topic

1. Can a low median latency indicate issues?

Not necessarily; a low median with a favorable p99:p50 ratio is ideal.

2. Why is a 1 percent tail rate significant?

Because it compounds with repeated requests, leading to higher chances of encountering issues.

3. Should I always choose the provider with the tightest tail?

Not automatically; it depends on the specific workload requirements.

4. Are the results indicative of model superiority?

No; the measurements reflect specific conditions and should not be generalized.

Production challenges often reside in the distribution of latency rather than the median.

Conduct the testing protocol to understand your own production experience.

1. GitHub Repository for Test Protocol Execution

• serverless-inference-tail-latency-study on GitHub : the full evidence base for every measured number in this article — the measurement harness, orchestration and bounded-sweep scripts, raw per-request JSON for all 1,590 recorded requests, unedited console logs, the analysis script, and the four charts, under an MIT license.

2. DigitalOcean Documentation and Engineering Resources

• Prompt Caching and Cost Break-Even
• Inferentia2, TPU, Groq LPU, vs. GPU for LLM Serving
• Serverless Inference Metrics reference
• Inference Features, observability section
• Inference Limits
• Why Serverless Inference Consistency Varies on the Same Model
• Metrics that Matter with Serverless Inference
• DigitalOcean Inference Mode Comparison for Your Each Use Case
• Fine-Tuned LLMs on Serverless Architecture
• How to Use Inference Router
• DigitalOcean Serverless Inference: A Deep Dive
• Why is p99 Time to First Token (TTFT) High When Everything Else Looks Normal? Debugging Tail Latency in LLM Inference

3. External References

• Artificial Analysis, Language Model Benchmarking Methodology
• Artificial Analysis, GLM-5.2 (max) provider comparison
• Artificial Analysis, gpt-oss-120b (high) provider comparison
• Ashok Chandrasekar and Jason Kramberger, “ Identifying and Mitigating Systemic Measurement Bias in Production LLM Inference Benchmarks ,” arXiv, 2026.

Discover more about our product offerings.

About the Author

Anish is a Senior Technical Content Strategist with extensive experience in DevOps and technical writing.

This textbox defaults to using Markdown for formatting.

Use !ref to search our complete set of tutorials and documentation.

Highlighted Tutorials

• All tutorials
• All topic tags

Please fill in your information!

• Table of contents
• Terms Used in This Article
• TL;DR
• What users actually feel: the case for tail metrics
• TTFT, inter-token latency, and total time: pick the metric for your workload
• The agent multiplier: why task completion time is the real metric
• How to read a latency benchmark critically
• The DIY latency test protocol, executed
• Decision framework
• What this piece does and does not verify
• Common Questions on this topic?
• Conclusion
• References

• Ubuntu
• Linux Basics
• JavaScript
• Python
• MySQL
• Docker
• Kubernetes
• All tutorials
• Talk to an expert
• Featured tutorials SOLID Design Principles Explained: Building Better Software Architecture
• How To Remove Docker Images, Containers, and Volumes
• How to Create a MySQL User and Grant Privileges (Step-by-Step)
• All tutorials
• All topic tags

Join us in creating technical tutorials while supporting a tech-focused charity.

DigitalOcean Documentation

Comprehensive documentation for all DigitalOcean products.

Resources for Startups and AI Businesses

The Wave provides essential insights for building a business.

The Developer Cloud

Scale your infrastructure as your needs grow.

Start Building Today

Access everything you need to develop intelligent applications.

• About
• Leadership
• Blog
• Careers
• Customers
• Partners
• Referral Program
• Press
• Legal
• Privacy Policy
• Security
• Investor Relations

• Knowledge Bases
• GPU Droplets
• Bare Metal GPUs
• Inference Engine
• Data & Learning
• Evaluations
• Model Library
• Droplets
• Kubernetes
• Functions
• App Platform
• Load Balancers
• Managed Databases
• Spaces
• Block Storage
• Network File Storage
• API
• Uptime
• Cloud Security Posture Management (CSPM)
• Identity and Access Management (IAM)
• Cloudways
• View all Products

• Community Tutorials
• Community Q&A
• CSS-Tricks
• Currents Research
• DigitalOcean Startups
• Wavemakers Program
• Compass Council
• Open Source
• Marketplace
• Pricing
• Pricing Calculator
• Documentation
• Release Notes
• Code of Conduct
• Shop Swag

• AI Training GPU
• GPU Inference
• VPS Hosting
• Website Hosting
• VPN
• Docker Hosting
• Node.js Hosting
• Web Mobile Apps
• WordPress Hosting
• Virtual Machines
• View all Solutions

• Support
• Sales
• Report Abuse
• System Status
• Share your ideas

• About
• Leadership
• Blog
• Careers
• Customers
• Partners
• Referral Program
• Press
• Legal
• Privacy Policy
• Security
• Investor Relations

• Knowledge Bases
• GPU Droplets
• Bare Metal GPUs
• Inference Engine
• Data & Learning
• Evaluations
• Model Library
• Droplets
• Kubernetes
• Functions
• App Platform
• Load Balancers
• Managed Databases
• Spaces
• Block Storage
• Network File Storage
• API
• Uptime
• Cloud Security Posture Management (CSPM)
• Identity and Access Management (IAM)
• Cloudways
• View all Products

• Community Tutorials
• Community Q&A
• CSS-Tricks
• Currents Research
• DigitalOcean Startups
• Wavemakers Program
• Compass Council
• Open Source
• Marketplace
• Pricing
• Pricing Calculator
• Documentation
• Release Notes
• Code of Conduct
• Shop Swag

• AI Training GPU
• GPU Inference
• VPS Hosting
• Website Hosting
• VPN
• Docker Hosting
• Node.js Hosting
• Web Mobile Apps
• WordPress Hosting
• Virtual Machines
• View all Solutions

• Support
• Sales
• Report Abuse
• System Status
• Share your ideas