Structured Output and Tool Calling

In post 4 we called an LLM the way most tutorials stop: send a prompt, read the assistant’s prose reply, print it. That is fine when a human reads the answer. It falls apart the moment your Go program has to act on the answer. resp.Choices[0].Message.Content is a string, and a string is exactly the wrong shape to feed into a database write, a switch statement, or a downstream API. You end up writing regexes to scrape a number out of a sentence, and the model breaks you the first time it adds “Sure, here’s the total: $42.00.” around it.

This post is about getting output you can rely on — machine-readable, typed, validated. There are two mechanisms, and the whole point of this post is that they are the same idea wearing two hats: constrain the model to emit JSON your Go code can decode and act on. The first, structured output, asks the model to fill in a shape you define. The second, tool calling, lets the model ask you to run a function and hand back the result. Both ride the same OpenAI-compatible chat JSON we already speak from post 4, so we build on the hand-rolled net/http client, not a vendor SDK.

We will keep one hard rule throughout: the wire is JSON typed by a schema, but the model is not bound by your types. It can still return malformed or schema-violating JSON. So every decode is defensive, and every structured call gets a validate-and-retry loop.

The shared foundation: our post-4 client

Everything below assumes the small chat client from post 4. As a refresher, the request and response envelopes look like this — plain structs over encoding/json, no framework:

type Message struct {
    Role       string     `json:"role"`
    Content    string     `json:"content,omitempty"`
    ToolCalls  []ToolCall `json:"tool_calls,omitempty"`
    ToolCallID string     `json:"tool_call_id,omitempty"` // set on role:"tool" replies
    Name       string     `json:"name,omitempty"`
}

type ChatRequest struct {
    Model          string          `json:"model"`
    Messages       []Message       `json:"messages"`
    Tools          []Tool          `json:"tools,omitempty"`
    ResponseFormat *ResponseFormat `json:"response_format,omitempty"`
}

type ChatResponse struct {
    Choices []struct {
        Message      Message `json:"message"`
        FinishReason string  `json:"finish_reason"`
    } `json:"choices"`
}

Assume a method client.Complete(ctx, ChatRequest) (ChatResponse, error) that marshals the request, POSTs it, and unmarshals the reply — exactly what we wrote last time. The new fields (Tools, ResponseFormat, ToolCalls, ToolCallID) are all omitempty, so a plain chat call serializes identically to post 4. We are extending the envelope, not replacing it.

Structured output, attempt one: just ask for JSON

The lowest-tech approach is to ask, in the prompt, for JSON and nothing else. Define the Go shape you want first, because the struct is your contract:

type Invoice struct {
    Vendor   string  `json:"vendor"`
    Total    float64 `json:"total"`
    Currency string  `json:"currency"`
    DueDate  string  `json:"due_date"` // ISO 8601, e.g. "2026-09-01"
}

Then instruct the model to produce that shape:

sys := `Extract the invoice fields. Respond with ONLY a JSON object, no prose, no
markdown fences, matching exactly:
{"vendor": string, "total": number, "currency": string, "due_date": "YYYY-MM-DD"}`

resp, err := client.Complete(ctx, ChatRequest{
    Model: "gpt-4o-mini",
    Messages: []Message{
        {Role: "system", Content: sys},
        {Role: "user", Content: rawInvoiceText},
    },
})

This works often enough to be tempting and fails often enough to hurt. The model may wrap the JSON in ```json fences, prepend “Here is the extracted data:”, or emit a trailing comma. You are back to scraping text.

The gotcha: prompting for JSON is a request, not a guarantee. Never json.Unmarshal the raw content directly and assume success. At minimum, strip markdown fences and locate the outermost {...} before decoding — and even then, treat a decode error as an expected branch, not a panic. Prompt-only JSON is the fallback for models or endpoints that lack a real structured-output mode, not the default.

Structured output, attempt two: response_format

Most OpenAI-compatible endpoints expose a response_format field that constrains the decoder rather than merely asking politely. Two modes matter:

• {"type": "json_object"} — the model is guaranteed to emit syntactically valid JSON (some object), but not any particular shape.
• {"type": "json_schema", ...} — you attach a JSON Schema and the model’s output is constrained to match it. This is the strong version.

Here are the Go types for the schema mode:

type ResponseFormat struct {
    Type       string      `json:"type"`                  // "json_schema"
    JSONSchema *JSONSchema `json:"json_schema,omitempty"`
}

type JSONSchema struct {
    Name   string                 `json:"name"`
    Strict bool                   `json:"strict"`
    Schema map[string]interface{} `json:"schema"`
}

And the schema for our Invoice struct, written by hand as a Go map so it marshals to exactly the JSON Schema the API expects:

invoiceSchema := map[string]interface{}{
    "type": "object",
    "properties": map[string]interface{}{
        "vendor":   map[string]interface{}{"type": "string"},
        "total":    map[string]interface{}{"type": "number"},
        "currency": map[string]interface{}{"type": "string"},
        "due_date": map[string]interface{}{"type": "string"},
    },
    "required":             []string{"vendor", "total", "currency", "due_date"},
    "additionalProperties": false,
}

req := ChatRequest{
    Model:    "gpt-4o-mini",
    Messages: []Message{{Role: "user", Content: rawInvoiceText}},
    ResponseFormat: &ResponseFormat{
        Type: "json_schema",
        JSONSchema: &JSONSchema{Name: "invoice", Strict: true, Schema: invoiceSchema},
    },
}

With strict: true and additionalProperties: false, a compliant endpoint returns JSON that decodes cleanly into Invoice. Writing the schema by hand keeps you honest about what “required” means and avoids pulling in a reflection library — but for larger structs you can generate the map from reflect once and reuse it.

The gotcha: json_schema support is provider- and model-specific. Older or self-hosted endpoints may accept only json_object, or silently ignore response_format entirely and return prose. Don’t assume the field did anything — the very next section validates the result regardless of which mode produced it. Structured mode narrows the failure rate; it does not eliminate it.

Decode defensively, then validate, then retry

This is the part most write-ups skip, and it is the part that makes structured output production-grade. Even with strict: true, you can get a total of -1, an empty vendor, or a due_date of "next Tuesday". Schema conformance is not the same as semantic validity. So we separate three concerns: decode (is it valid JSON of the right shape?), validate (do the values make sense?), and retry (feed the error back and let the model fix it).

func validateInvoice(inv Invoice) error {
    if strings.TrimSpace(inv.Vendor) == "" {
        return errors.New("vendor is empty")
    }
    if inv.Total <= 0 {
        return fmt.Errorf("total must be positive, got %v", inv.Total)
    }
    if _, err := time.Parse("2006-01-02", inv.DueDate); err != nil {
        return fmt.Errorf("due_date %q is not YYYY-MM-DD", inv.DueDate)
    }
    return nil
}

Now the loop. We give the model a bounded number of attempts; on each failure we append the model’s bad answer and a user message describing what was wrong, so the next attempt is a correction rather than a blind retry:

func extractInvoice(ctx context.Context, c *Client, text string) (Invoice, error) {
    msgs := []Message{{Role: "user", Content: text}}

    const maxAttempts = 3
    for attempt := 1; attempt <= maxAttempts; attempt++ {
        resp, err := c.Complete(ctx, ChatRequest{
            Model: "gpt-4o-mini", Messages: msgs,
            ResponseFormat: &ResponseFormat{
                Type:       "json_schema",
                JSONSchema: &JSONSchema{Name: "invoice", Strict: true, Schema: invoiceSchema},
            },
        })
        if err != nil {
            return Invoice{}, err // transport error: don't burn a retry
        }
        content := resp.Choices[0].Message.Content

        var inv Invoice
        decodeErr := json.Unmarshal([]byte(content), &inv)
        if decodeErr == nil {
            decodeErr = validateInvoice(inv)
        }
        if decodeErr == nil {
            return inv, nil // success
        }

        // Feed the failure back for the next attempt.
        msgs = append(msgs,
            Message{Role: "assistant", Content: content},
            Message{Role: "user", Content: "That response was invalid: " +
                decodeErr.Error() + ". Return corrected JSON only."})
    }
    return Invoice{}, fmt.Errorf("no valid invoice after %d attempts", maxAttempts)
}

Note the two error paths are treated differently. A transport error (err) means the HTTP call itself failed — retrying inside this loop would just hammer a broken endpoint, so we return immediately and let the caller’s own retry/backoff decide. A content error (decodeErr) means the model answered but got it wrong, which is exactly what the correction loop is for.

The gotcha: the retry budget must be bounded. An unbounded “loop until valid” can spin forever against a model that keeps making the same mistake, running up cost and latency with nothing to show. Three attempts is a reasonable default; past that, fail loudly and let a human or a fallback handle it. Also: keep the successfully-decoded value even if validation later fails — you want the error message to reference real values (“got -1”), which is what makes the correction prompt effective.

Tool calling: the model asks you to run code

Structured output makes the model fill in a shape. Tool calling flips the direction: you tell the model what functions exist, and it decides to call one, handing you typed arguments. You run the real Go function and hand the result back. It is structured output pointed at your codebase.

The request carries a tools array. Each tool is a name, a description, and a JSON-Schema description of its parameters — the same schema machinery as above:

type Tool struct {
    Type     string       `json:"type"` // always "function"
    Function ToolFunction `json:"function"`
}

type ToolFunction struct {
    Name        string                 `json:"name"`
    Description string                 `json:"description"`
    Parameters  map[string]interface{} `json:"parameters"`
}

Let’s expose one concrete tool: a currency converter backed by a fixed rate table (real Go, no fabricated service):

var rates = map[string]float64{"USD": 1.0, "EUR": 1.08, "GBP": 1.27, "INR": 0.012}

// The actual Go function the model's call will drive.
func convertCurrency(amount float64, from, to string) (float64, error) {
    rf, okF := rates[from]
    rt, okT := rates[to]
    if !okF || !okT {
        return 0, fmt.Errorf("unsupported currency pair %s->%s", from, to)
    }
    return amount * rf / rt, nil
}

var convertTool = Tool{
    Type: "function",
    Function: ToolFunction{
        Name:        "convert_currency",
        Description: "Convert an amount between USD, EUR, GBP and INR.",
        Parameters: map[string]interface{}{
            "type": "object",
            "properties": map[string]interface{}{
                "amount": map[string]interface{}{"type": "number"},
                "from":   map[string]interface{}{"type": "string", "enum": []string{"USD", "EUR", "GBP", "INR"}},
                "to":     map[string]interface{}{"type": "string", "enum": []string{"USD", "EUR", "GBP", "INR"}},
            },
            "required": []string{"amount", "from", "to"},
        },
    },
}

When the model decides to use it, the response comes back not as prose but with finish_reason: "tool_calls" and a tool_calls array on the message:

{
  "choices": [{
    "finish_reason": "tool_calls",
    "message": {
      "role": "assistant",
      "content": null,
      "tool_calls": [{
        "id": "call_a1b2c3",
        "type": "function",
        "function": {
          "name": "convert_currency",
          "arguments": "{\"amount\": 100, \"from\": \"EUR\", \"to\": \"USD\"}"
        }
      }]
    }
  }]
}

Look closely at arguments. It is a JSON string, not a JSON object — a string whose contents are JSON. Our Go types reflect that exactly:

type ToolCall struct {
    ID       string `json:"id"`
    Type     string `json:"type"`
    Function struct {
        Name      string `json:"name"`
        Arguments string `json:"arguments"` // JSON encoded as a string
    } `json:"function"`
}

The gotcha: arguments arrives as a string you must unmarshal a second time, not a map you can index. tc.Function.Arguments["amount"] does not compile — the field is a string. You json.Unmarshal([]byte(tc.Function.Arguments), &args) into a typed struct, and that decode can fail exactly like any other model output. The model can hallucinate an argument name, omit a required field, or pass a string where you wanted a number. Decode into a struct and check, never trust it blindly.

The full round-trip loop

One tool call is rarely the whole story. The model might call convert_currency, read the result, then call it again for a second pair, then finally answer in prose. So the driver is a loop: call the model, and while the reply contains tool calls, execute each one, append its result as a new message, and call again. You exit only when the model returns prose (finish_reason is not tool_calls).

func runWithTools(ctx context.Context, c *Client, userMsg string) (string, error) {
    msgs := []Message{{Role: "user", Content: userMsg}}

    const maxTurns = 8 // hard ceiling so a misbehaving model can't loop forever
    for turn := 0; turn < maxTurns; turn++ {
        resp, err := c.Complete(ctx, ChatRequest{
            Model: "gpt-4o-mini", Messages: msgs, Tools: []Tool{convertTool},
        })
        if err != nil {
            return "", err
        }
        reply := resp.Choices[0].Message
        msgs = append(msgs, reply) // the assistant turn (with its tool_calls) stays in history

        if len(reply.ToolCalls) == 0 {
            return reply.Content, nil // model answered in prose — we're done
        }

        // Execute every requested call and append one tool-result message per call.
        for _, tc := range reply.ToolCalls {
            result := dispatchTool(tc)
            msgs = append(msgs, Message{
                Role:       "tool",
                ToolCallID: tc.ID, // ties the result to the specific call
                Name:       tc.Function.Name,
                Content:    result,
            })
        }
    }
    return "", fmt.Errorf("tool loop exceeded %d turns", maxTurns)
}

dispatchTool is where the string-arguments decode and the real function call live. Crucially, a tool error is not a Go error you return up the stack — it is a result you hand back to the model, which can then apologize or try different arguments:

func dispatchTool(tc ToolCall) string {
    switch tc.Function.Name {
    case "convert_currency":
        var args struct {
            Amount float64 `json:"amount"`
            From   string  `json:"from"`
            To     string  `json:"to"`
        }
        if err := json.Unmarshal([]byte(tc.Function.Arguments), &args); err != nil {
            return fmt.Sprintf(`{"error": "could not parse arguments: %s"}`, err)
        }
        out, err := convertCurrency(args.Amount, args.From, args.To)
        if err != nil {
            return fmt.Sprintf(`{"error": %q}`, err.Error())
        }
        return fmt.Sprintf(`{"result": %.2f}`, out)
    default:
        return fmt.Sprintf(`{"error": "unknown tool %q"}`, tc.Function.Name)
    }
}

The gotcha: two things must be exact or the loop silently breaks. First, the assistant message with its tool_calls must go back into msgs before you append the results — the API rejects a tool message that answers a call it never saw. Second, each tool result is its own message with role: "tool" and a tool_call_id matching the id the model sent. If the model made three calls in one turn, you append three tool messages, each tagged with its own id. Mismatch the ids and the model can’t tell which result answers which call. And keep the maxTurns ceiling: without it, a model that loops on a failing tool will spin until your context window or your budget runs out.

The two mechanisms, side by side

Both are the same move: attach a JSON Schema, get back JSON, decode into a Go struct, and never trust it until you have checked it. Structured output is a one-shot fill-in-the-shape; tool calling is a conversation where the shape is “which function, with what arguments.” Once you see them as one idea, the defensive-decoding discipline transfers directly from one to the other.

Key takeaways

• A string is the wrong output type for software. Structured output and tool calling both exist to get you typed, machine-readable JSON instead of prose you have to scrape.
• Prefer response_format json_schema over prompt-only JSON, but know it is provider-dependent and still fallible — validate regardless of which mode produced the bytes.
• Decode, validate, retry — bounded. Schema conformance is not semantic validity; run a real validate step and feed failures back into a capped correction loop.
• Tool arguments is a JSON string, not a map. Unmarshal it a second time into a typed struct, and treat that decode as fallible.
• Tool calling is a loop, not a call. Append the assistant’s tool_calls turn, then one role:"tool" message per call tagged with its tool_call_id, and iterate until the model answers in prose — with a hard turn ceiling.
• The two mechanisms are one idea: constrain the model to emit JSON your Go code can act on, and never act on it blindly.

Next in the series we build on this loop to assemble a small agent — the same tool round-trip, plus memory and a planning step, so the model can string several tools together toward a goal.

Further reading

• OpenAI — Structured Outputs guide
• OpenAI — Function calling guide
• Anthropic — Tool use (function calling)
• Anthropic — Increase output consistency with JSON
• Go — encoding/json package documentation
• JSON Schema — specification and reference