Migrate AI Cloud Inference

Blog Posts
Documentation
Career Opportunities
Support Options
Contact Sales Team

Featured Products with AI Capabilities
Compute Resources for Building and Scaling
Container and Image Management
Managed Database Solutions
Development and Management Tools

Login to Access
Community Login
DigitalOcean Login

Sign up for Access
Community Sign up
DigitalOcean Sign up

Tutorial Guides
Frequently Asked Questions
Product Documentation
Community Search

Table of Contents Overview

Written by Andrew Dugan, Senior AI Technical Content Creator

Andrew Dugan is a Senior AI Technical Content Creator with expertise in AI and machine learning

When developing Large Language Model applications, teams often find themselves locked into a single inference provider, making it difficult to incorporate new open-source models into their application

Migrating AI cloud inference tools is relatively straightforward, requiring only changes to the base URL, API key, and model, allowing teams to take advantage of wider model choice and lower costs

Migrating inference is primarily a configuration change
Wider model choice and lower costs are significant benefits
Building for portability ensures easy future migrations

Why Teams Migrate from Single Closed Labs

The primary benefit of migrating is accessing a wider range of models, including open-weight and frontier models, at lower costs, allowing teams to reduce costs by 10x to 50x

Running open-weight models on infrastructure providers keeps inference and data within the chosen cloud, next to the application and database, ensuring data control and security

Migrating also removes single points of failure, allowing teams to route requests across multiple models and providers, providing backup options in case of primary provider failures

Drop-In Replacement for Inference

The OpenAI-compatible API format has become an industry standard, making migration relatively straightforward, with only the base URL, API key, and model requiring changes

# Before — OpenAI client = OpenAI ( api_key = os . getenv ( "OPENAI_API_KEY" ) ) # After — any OpenAI-compatible provider client = OpenAI ( base_url = "https://<provider-endpoint>/v1/" , api_key = os . getenv ( "PROVIDER_API_KEY" ) , )

Teams can migrate without writing code by changing environment variables, and the OpenAI SDK can be used with minimal changes for standard chat completions

import os , requests resp = requests . post ( "https://<provider-endpoint>/v1/chat/completions" , headers = { "Authorization" : f"Bearer { os . getenv ( &apos;PROVIDER_API_KEY&apos; ) } " } , json = { "model" : "llama3-8b-instruct" , "messages" : [ { "role" : "user" , "content" : "Hello" } ] , } , ) print ( resp . json ( ) [ "choices" ] [ 0 ] [ "message" ] [ "content" ] )

Migrating from Anthropic requires adopting the OpenAI request format or using an Anthropic-compatible endpoint, if available

Standardizing on the OpenAI format is a clean migration path, as most inference clouds expose the OpenAI-compatible schema

Building for Future Portability

Teams can build for portability by keeping model configurations in environment variables and isolating provider-specific features behind helper functions

Storing prompts in versioned files and using a golden-dataset eval suite helps teams validate new models and ensure easy future migrations

Challenges and Trade-Offs of Migration

While migrating offers many benefits, teams may face challenges such as re-tuning prompts and adapting to new model behaviors

Teams may need to re-embed their entire vector store or refactor tool-calling schemas, depending on the initial architecture

Where DigitalOcean Fits in the Migration Process

DigitalOcean serves inference at a single endpoint, with one key, reaching over 70 open-weight and frontier models, offering serverless, batch, and dedicated deployment modes

Teams can start by creating a DigitalOcean account, generating a model access key, and updating the base URL, API key, and model in their OpenAI call

client = OpenAI ( base_url = "https://inference.do-ai.run/v1/" , # Added the DigitalOcean endpoint api_key = os . getenv ( "DIGITALOCEAN_INFERENCE_KEY" ) , # Added the DigitalOcean key ) resp = client . chat . completions . create ( model = "openai-gpt-5.5" , # Updated to DigitalOcean&apos;s catalog ID messages = [ { "role" : "user" , "content" : "Hello" } ] , )

Teams can test the migration by starting with the identical frontier model, then A/B testing an open-weight model, and verifying parity on companion tools

Migration is mostly a configuration change, offering wider model choice, lower costs, and easy future migrations, allowing teams to stay up-to-date with new model releases

Fine-tuning a smaller open-weight model on a dataset can result in a model that compares to a frontier model at a fraction of the cost, becoming an organizational asset

Frequently Asked Questions about Migration

How to migrate without writing code
Low-cost drop-in replacements for OpenAI
Migrating to open-source LLMs in production
Replacing OpenAI models with cheaper fine-tuned models

Using DigitalOcean's Serverless Inference with OpenAI SDK
DigitalOcean Gradient AI Platform Documentation
Fine-Tuning LLMs with LoRA for Custom Domains

Learn more about available products and services

About the Author of this Guide

Andrew is an NLP Scientist with 8 years of experience in designing and deploying enterprise AI applications

This text area supports Markdown formatting for easy answer formatting

Quickly search tutorials, documentation, and marketplace offerings using the !ref command

Featured Tutorials and Guides

All Available Tutorials
All Topic Tags

Please complete your information to proceed

Table of Contents
Key Takeaways
Why Migrate from Single Closed Labs
Drop-In Replacement for Inference
Building for Future Portability
Challenges and Trade-Offs
Where DigitalOcean Fits
Conclusion
Common Questions
Related Links

Ubuntu
Linux Basics
JavaScript
Python
MySQL
Docker
Kubernetes
All Tutorials
Talk to an Expert
Featured Tutorials

Get paid to write technical tutorials and support a tech-focused charity

DigitalOcean Documentation and Resources

Full documentation for every DigitalOcean product and service

Resources for Startups and AI-Native Businesses

The Wave provides everything needed to know about building a business, from funding to marketing

The Developer Cloud for Scaling

Scale up as you grow, whether running one virtual machine or ten thousand

Start Building Today with DigitalOcean

Get everything needed to build, scale, and deploy intelligent applications, from GPU-powered inference to managed databases

About
Leadership
Blog
Careers
Customers
Partners
Referral Program
Press
Legal
Privacy Policy
Security
Investor Relations

Knowledge Bases
GPU Droplets
Bare Metal GPUs
Inference Engine
Data and Learning
Evaluations
Model Library
Droplets
Kubernetes
Functions
App Platform
Load Balancers
Managed Databases
Spaces
Block Storage
Network File Storage
API
Uptime
Cloud Security Posture Management
Identity and Access Management
Cloudways
View all Products

Community Tutorials
Community Q&A
CSS-Tricks
Currents Research
DigitalOcean Startups
Wavemakers Program
Compass Council
Open Source
Marketplace
Pricing
Pricing Calculator
Documentation
Release Notes
Code of Conduct
Shop Swag

AI Training GPU
GPU Inference
VPS Hosting
Website Hosting
VPN
Docker Hosting
Node.js Hosting
Web and Mobile Apps
WordPress Hosting
Virtual Machines
View all Solutions

Support
Sales
Report Abuse
System Status
Share Your Ideas

About
Leadership
Blog
Careers
Customers
Partners
Referral Program
Press
Legal
Privacy Policy
Security
Investor Relations

Knowledge Bases
GPU Droplets
Bare Metal GPUs
Inference Engine
Data and Learning
Evaluations
Model Library
Droplets
Kubernetes
Functions
App Platform
Load Balancers
Managed Databases
Spaces
Block Storage
Network File Storage
API
Uptime
Cloud Security Posture Management
Identity and Access Management
Cloudways
View all Products

Community Tutorials
Community Q&A
CSS-Tricks
Currents Research
DigitalOcean Startups
Wavemakers Program
Compass Council
Open Source
Marketplace
Pricing
Pricing Calculator
Documentation
Release Notes
Code of Conduct
Shop Swag

AI Training GPU
GPU Inference
VPS Hosting
Website Hosting
VPN
Docker Hosting
Node.js Hosting
Web and Mobile Apps
WordPress Hosting
Virtual Machines
View all Solutions

Support
Sales
Report Abuse
System Status
Share Your Ideas