EN ▾
Get API key

LLM Inference APIGuide

LLM Inference API: Streaming, Tools and JSON Mode

An LLM inference API lets you send a prompt over HTTPS and get generated text back, with no GPU on your side. This page shows a working setup against an OpenAI-compatible endpoint, from the first request to streaming, tools, JSON mode and rate limits.

Updated:

Isometric illustration of data packets flowing from a terminal into a cloud gateway, representing an inference API call

Key points

  1. Point any OpenAI SDK at https://api.llminferenceapi.com/v1, use the model ID uncensored and send your key as a Bearer token.
  2. Two endpoints are available: POST /v1/chat/completions and GET /v1/models.
  3. Streaming, tool calling and JSON mode work through the standard request fields.
  4. A full key allows 8 concurrent requests and 300 per minute; a 429 response tells you which limit you hit.

What you need before the first call

You need three things: a base URL, a key and a model name. The base URL is https://api.llminferenceapi.com/v1, the model ID is uncensored, and the key comes from the API key page after you sign up with Google or with email and password. New users can claim $0.50 of trial credit there with one click. It needs no card, lasts 7 days and is limited to one per person.

The service is text only. It exposes POST /v1/chat/completions and GET /v1/models. There is no embeddings route, no image or audio generation and no fine-tuning. It runs one open-weight model, operated by an independent provider and tuned to answer without content refusals for lawful adult use. It is not the official API of any model vendor. One hard rule applies: content that sexualizes minors is not allowed.

Keep the key in an environment variable rather than in source code. The examples below read it from API_KEY where the language allows.

Your first request

Start with curl, because it shows exactly what goes over the wire.

curl https://api.llminferenceapi.com/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "uncensored",
    "messages": [
      {"role": "system", "content": "You are a concise assistant."},
      {"role": "user", "content": "List three uses of a message queue."}
    ],
    "temperature": 0.7
  }'

The reply is the usual chat completion object: choices[0].message.content holds the text and usage holds the token counts. You can also list models with GET /v1/models, which is a quick way to confirm that the key and URL are right before you debug anything else.

In Python with the official OpenAI SDK, only the constructor changes:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.llminferenceapi.com/v1",
    api_key=os.environ["API_KEY"],
)

resp = client.chat.completions.create(
    model="uncensored",
    messages=[{"role": "user", "content": "List three uses of a message queue."}],
)
print(resp.choices[0].message.content)

And in Node:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.llminferenceapi.com/v1",
  apiKey: process.env.API_KEY,
});

const resp = await client.chat.completions.create({
  model: "uncensored",
  messages: [{ role: "user", content: "List three uses of a message queue." }],
});
console.log(resp.choices[0].message.content);

The completion may use whatever is left of the 100,000-token window after your prompt, so max_tokens is optional. Set it anyway when you want a hard ceiling on cost or length. Sampling controls such as temperature, top_p, stop, seed and the frequency and presence penalties are passed through.

Two settings are worth knowing from the start. seed makes sampling repeatable enough for testing, which helps when you compare two versions of a prompt and want the difference to come from the prompt rather than from chance. stop takes one or more strings that end generation as soon as the model writes them, a cheap way to cut off a reply at a delimiter and avoid paying for tokens you would throw away.

Streaming responses

With stream: true the server answers over server-sent events and sends the text in small deltas. Users see the first words quickly, which matters in chat interfaces. Usage figures arrive at the end of the stream.

stream = client.chat.completions.create(
    model="uncensored",
    messages=[{"role": "user", "content": "Write a four-line poem about a lighthouse."}],
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\n", chunk.usage.prompt_tokens, chunk.usage.completion_tokens)

Two points are easy to miss. First, the last chunk can carry usage and an empty choices list, so check that choices is not empty before indexing it. Second, a streamed request holds its concurrency slot until the stream ends. If a client opens a stream and never reads it to the end, that slot stays busy and your later requests can hit a concurrency error. Always close or fully consume streams.

Tool calling

Tool calling lets the model ask your code to run a function and use the result. You describe the functions in tools, optionally steer the choice with tool_choice, execute whatever the model requests, and send the output back as a message with role: "tool".

import json

tools = [{
    "type": "function",
    "function": {
        "name": "get_order_status",
        "description": "Look up the status of an order by its id.",
        "parameters": {
            "type": "object",
            "properties": {"order_id": {"type": "string"}},
            "required": ["order_id"],
        },
    },
}]

messages = [{"role": "user", "content": "Where is order A-1043?"}]
first = client.chat.completions.create(
    model="uncensored", messages=messages, tools=tools, tool_choice="auto"
)

msg = first.choices[0].message
if msg.tool_calls:
    messages.append(msg)
    for call in msg.tool_calls:
        args = json.loads(call.function.arguments)
        result = {"order_id": args["order_id"], "status": "shipped"}
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": json.dumps(result),
        })
    final = client.chat.completions.create(
        model="uncensored", messages=messages, tools=tools
    )
    print(final.choices[0].message.content)

Always parse function.arguments defensively. It is a JSON string produced by a language model, and on rare occasions it can be malformed or contain a field you did not define. Validate it before you act on it, particularly when the function changes data. Tool calls also work with streaming, in which case the tool_calls arrive as fragments you must join by index before parsing.

Managing conversation history

The API is stateless. It does not remember earlier requests, so a multi-turn chat works only because your code sends the earlier turns again each time. Keep a list of messages, append the user's new message, call the endpoint, then append the assistant's reply before the next turn.

history = [{"role": "system", "content": "You are a patient writing coach."}]

def chat(user_text):
    history.append({"role": "user", "content": user_text})
    resp = client.chat.completions.create(model="uncensored", messages=history)
    reply = resp.choices[0].message.content
    history.append({"role": "assistant", "content": reply})
    return reply

print(chat("Give me a prompt for a ghost story."))
print(chat("Make it set on a ferry."))

Because every turn resends the history, input tokens grow with the length of the chat. The window holds 100,000 tokens, which is plenty for most conversations, but cost climbs well before you reach the limit. A common pattern is to keep the system message and the last several turns, and replace older turns with a short summary that the model itself writes. Read usage.prompt_tokens after each call to see how large the history has become.

JSON mode

When you need machine-readable output, set response_format to {"type": "json_object"} and say in the prompt that you want JSON. Describing the keys you expect makes the result far more stable.

resp = client.chat.completions.create(
    model="uncensored",
    response_format={"type": "json_object"},
    messages=[
        {"role": "system", "content": "Reply with JSON only, using the keys title, tags and summary."},
        {"role": "user", "content": "Describe a short story about a lost robot."},
    ],
)
data = json.loads(resp.choices[0].message.content)
print(data["title"], data["tags"])

JSON mode makes the output parse as JSON, but it does not enforce your schema. Check the keys in code, and keep a fallback for the case where a value is missing. If the reply is cut off because it reached a token limit, the JSON will be incomplete, so do not set max_tokens too low for structured output.

A short checklist when something fails

Most first-day problems come from a handful of causes, and you can rule them out in order.

  1. Base URL. It must end in /v1. The SDK appends /chat/completions itself, so a URL with the path already included will not work.
  2. Key header. The key goes in Authorization: Bearer followed by the key. A stray space or newline copied with the key is a frequent cause of authentication errors.
  3. Model ID. Use uncensored exactly. Names of other vendors' models are not valid here.
  4. Request size. The body limit is 8 MB, and prompt plus completion must fit in 100,000 tokens.
  5. Balance. Look at the X-Balance-USD header on any response to see whether credit is the problem.
  6. Quote the request ID. If you contact support, include the X-Request-Id value from the failing response so the request can be found.

Keep in mind that a refusal is not the same as an error. Because the model is tuned to avoid content refusals for lawful adult use, you will see fewer of them than on filtered services, but the model can still decline or give a weak answer to a vague prompt. A clearer system message usually fixes that better than a higher temperature.

Limits, headers and retries

A full key allows 8 concurrent requests and 300 requests per minute. A trial key allows 2 concurrent requests and 60 per minute. The request body can be up to 8 MB. Each response carries X-Request-Id, X-Balance-USD, X-RateLimit-Limit-Requests and X-RateLimit-Limit-Concurrency, which are handy for logging and for showing a balance in an admin screen.

When you exceed a limit the API returns HTTP 429. The error code rate_limited means too many requests per minute, and concurrency means too many at once. On temporary overload you also get 429, with a Retry-After: 5 header. A simple retry wrapper covers all three:

import time
from openai import RateLimitError

def ask(messages, tries=4):
    for attempt in range(tries):
        try:
            return client.chat.completions.create(model="uncensored", messages=messages)
        except RateLimitError as e:
            wait = 5 if attempt == 0 else 5 * (attempt + 1)
            retry_after = getattr(e.response, "headers", {}).get("retry-after")
            time.sleep(int(retry_after) if retry_after else wait)
    raise RuntimeError("still rate limited after retries")

For a worker pool, cap the pool size at your concurrency limit instead of relying on retries. Eight workers on a full key will rarely see a concurrency error. If you do hit unexpected errors, the troubleshooting article goes through the common causes.

Paying for it

Billing is pay as you go from prepaid credit: $0.15 per 1M input tokens and $0.50 per 1M output tokens, no subscription. Failed or refused requests are not charged, and credit never expires. A request with 1,500 input tokens and 400 output tokens costs 1,500 / 1,000,000 x $0.15 + 400 / 1,000,000 x $0.50 = $0.000225 + $0.0002 = $0.000425.

Top-ups are any whole amount from $20 to $500, with a 5% bonus from $50, 10% from $100 and 15% from $200. You can pay with crypto (USDT on TRC20, USDC on Base). Prompts are not used for training. The pricing page lists the details and the quickstart repeats the setup steps in short form.

Questions and answers

Can I use the official OpenAI library with this API?

Yes. Install the OpenAI SDK for Python or Node, set the base URL to https://api.llminferenceapi.com/v1, pass your key, and use the model ID uncensored.

Does the API support function calling?

Yes. Pass tools and optionally tool_choice, then return each result as a message with role tool. Streamed tool_calls are supported as well.

What does a 429 error mean here?

You hit a limit. The code rate_limited means too many requests per minute, concurrency means too many at the same time, and a Retry-After: 5 header marks a temporary overload. Wait and retry.

Can I use it for embeddings or images?

No. The API is text only and offers chat completions and a model list.

Your key is one form away

Create an account, copy the key, change the base URL. That is the whole setup.

Get API key