Create Your Own AI Agent for Free

You can create your own AI agent for free by running an open model locally with Ollama. This guide uses a small Python agent that calls a local calculator tool. There is no hosted model bill, and your prompts stay on your machine.

"Free" has a precise meaning here: the software and model download cost nothing. Your computer still uses electricity, disk space, and memory, and a hosted GPU or paid API is no longer free. Be explicit about that trade-off before choosing a local stack.

What Free Means

• Software: Ollama and the example Python packages are free to install.
• Model: use an open-weight model whose license fits your project.
• Privacy: local inference keeps prompts and tool results on your device.
• Limits: local models can be slower, less capable, and more memory-hungry than paid APIs.

Start with a small model and a read-only task. If the model cannot reliably choose one tool, adding more tools will make the system less predictable, not more useful.

Choose a Local Stack

1. Install Ollama from its official distribution for your operating system.
2. Pull a model that your computer can run, such as llama3.2 or another supported instruct model.
3. Run Ollama's local HTTP service and verify that a simple prompt works.
4. Install the Python client and keep your agent code independent from model-specific prompts.

ollama pull llama3.2
python3 -m venv .venv
source .venv/bin/activate
pip install requests

Ollama exposes a local endpoint at http://localhost:11434. Do not expose that endpoint to the public internet without authentication, network controls, and a clear data policy.

Build the Agent

A local model can return a small JSON action such as {"tool":"calculate","expression":"19*7"}. Your program validates that action, executes the allowlisted tool, and asks the model for a final answer. If the JSON is invalid, stop and ask the user rather than guessing.

import json
import requests

MODEL = "llama3.2"
OLLAMA_URL = "http://localhost:11434/api/chat"

def calculate(expression: str) -> float:
    allowed = set("0123456789+-*/(). ")
    if not expression or any(char not in allowed for char in expression):
        raise ValueError("Only basic arithmetic is allowed")
    # Replace this with a real parser before production use.
    return float(eval(expression, {"__builtins__": {}}, {}))

def ask_ollama(messages: list[dict]) -> str:
    response = requests.post(
        OLLAMA_URL,
        json={"model": MODEL, "messages": messages, "stream": False},
        timeout=60,
    )
    response.raise_for_status()
    return response.json()["message"]["content"]

def run_agent(goal: str, max_steps: int = 4) -> str:
    messages = [{
        "role": "system",
        "content": "Return JSON action or final text. Action schema: "
                   "{tool, expression}. Use calculate only for arithmetic.",
    }, {"role": "user", "content": goal}]
    for _ in range(max_steps):
        text = ask_ollama(messages)
        try:
            action = json.loads(text)
        except json.JSONDecodeError:
            return text
        if action.get("tool") != "calculate":
            return text
        try:
            result = calculate(action["expression"])
        except (KeyError, TypeError, ValueError, SyntaxError):
            return "The tool request was rejected."
        messages.extend([
            {"role": "assistant", "content": text},
            {"role": "user", "content": "Tool result: " + str(result)},
        ])
    return "The local agent reached its step limit."

The eval line is deliberately marked as a teaching shortcut. Replace it with an AST-based parser before accepting untrusted expressions. The same boundary applies to shell, browser, file, and database tools.

Add Guardrails

• Allowlist tool names and validate every argument.
• Set maximum steps, request timeouts, and a local memory limit.
• Keep secrets out of prompts and redact them from logs.
• Require confirmation for network writes, purchases, and file deletion.
• Test prompt injection and malformed model output as normal failures.

Local execution reduces data exposure, but it does not make an agent automatically safe. A model with access to your filesystem can still damage it. Give the process a dedicated working directory and the minimum permissions required for the task.

The Real Costs

Local inference trades API spend for hardware resources. A laptop may run a small model comfortably but take longer on multi-step tasks. A larger model may need substantial RAM or a GPU. You also pay for electricity, storage, updates, and the time spent operating the stack.

Estimate context and storage needs with the memory cost calculator, and compare local usage with API budgets in the token budget calculator.

Frequently Asked Questions

Next Steps

For a hosted, more capable build, read the create your own AI agent guide. To run a stronger open model on your own hardware, see how to run Qwen3.8-27B locally. For the core mechanics, return to the How Do AI Agents Work? homepage guide.

create your own ai agent for free — return to the complete AI agent architecture guide.