Skip to main content

Deploy with Cloudflare

Cloudflare is a web performance and security company that provides content delivery network (CDN), DDoS protection, Internet security, and distributed domain name server services. Cloudflare launched Workers AI, which allows developers to run LLMs models powered by serverless GPUs on Cloudflare’s global network.

Set-up

To set-up Workers AI on Cloudflare, you need to create an account on the Cloudflare dashboard, get your account ID, and generate a token with Workers AI permissions. You can then send a completion request:

curl https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/mistral/mistral-7b-instruct-v0.1 \
-X POST \
-H "Authorization: Bearer {API_TOKEN}" \
-d '{ "messages": [{ "role": "user", "content": "[INST] 2 + 2 ? [/INST]" }]}'

Here is the output you should receive

{'result': {'response': '2 + 2 = 4.'}, 'success': True, 'errors': [], 'messages': []}