Introducing Thales

JoeCode

Introducing Thales

Aug 30, 2026

The Thales project is a private AI inference cluster built on two Dell PowerEdge R650 servers, each equipped with an NVIDIA T4 GPU. The goal is to provide low‑latency, high‑throughput inference endpoints for a variety of models—fast, deep, and embedding—behind a LiteLLM router. This post summarizes the current status of the build, the steps completed, and the next milestones.

Thales?

Thales is one of the artificial intelligences in Arthur C. Clarke and Stephen Baxter’s A Time Odyssey series. In the story’s near-future setting (2037), Thales is a quietly capable, trustworthy machine intelligence working alongside humans serving humanity’s lunar operations. Learn more about the project here: Thales Project.

Overview

Two Dell R650 servers, each with an NVIDIA T4 GPU, form the core of the Thales cluster. A dedicated 25 GbE link (10.10.10.1/30 ↔ 10.10.10.2/30) connects the machines, while Tailscale provides secure remote access. The stack runs Ubuntu 24.04 LTS, with iDRAC firmware, BIOS, and storage (RAID1 boot, RAID5 bulk) fully updated.

Inference Stack

llama.cpp built from source (CPU‑only for now). The gpt‑oss‑20b model is on inf1; the Qwen3‑Coder model will live on inf2.
llama‑server units bind only to LAN and tailnet interfaces.
LiteLLM router on inf1 exposes three endpoints: fast (gpt‑oss), deep (Qwen3‑Coder), and embed (to be added). Master key is set; database and per‑device keys are pending.

Current Status

All hardware is online, networking is stable, and the CPU‑only inference stack is functional. GPU acceleration will be added once NVIDIA drivers and CUDA are installed. Security hardening is complete; only authorized devices can reach the services.

Next Steps

Install NVIDIA drivers and CUDA.
Enable GPU inference for both models.
Add LiteLLM database and per‑device keys.
Deploy the embedding model.
Finalize client tooling (Copilot) and run end‑to‑end tests.

The Thales cluster is now a fully functional, secure inference platform ready for GPU acceleration. With the foundation laid, the next phase will focus on performance tuning, scaling, and integrating the system into our development workflows. Stay tuned for updates as we move from CPU validation to GPU‑powered inference.

JoeCode