Why I wrote a client library for surrealdb

I work mostly on backend services: REST APIs and GraphQL ones, with whatever stack the day calls for. Yoga, Apollo, GraphQL Mesh, Hasura. After enough of that you start noticing the same arguments re-running in every code review. Someone wants a strict schema because the API surface is the contract. Someone else wants flexibility because upstream systems don't care that your model says address.line1 when they sometimes call it street. Both sides are right.

The lesson I keep coming back to: schema is king. But flexibility is queen. And I need both to be married. Strictness without room to bend produces APIs that take a sprint to add a field to. Flexibility without rules produces databases nobody can query reliably six months later.

That's the frame I brought to SurrealDB. I keep an instance running on my homelab; it started as a curiosity and turned into the thing I reach for whenever I need a database. This is the story of why I ended up writing a client library for it: first in Python, then TypeScript, then Go, and most recently Rust. The throughline of all four is that schema-and-flexibility tension.

What drew me in

The hook was that I didn't have to pick a side. SurrealDB stores relational rows, graph edges, and nested documents in one engine, and lets me decide schema strictness per-table rather than per-database. So the user table can be locked down while the metadata blob attached to it stays loose, in the same database.

The example that flipped the switch was a follows edge. In SurrealQL the relationship itself is a record:

RELATE user:alice->follows->user:bob SET created_at = time::now();

-- And then traversal is direct:
SELECT ->follows->user.* FROM user:alice;

That's many-to-many with per-edge metadata and graph traversal in two lines. I'd built the same thing in Postgres with a join table and in Neo4j with Cypher; neither felt as honest as just saying the edge exists and putting properties on it.

Why the schema needed to live in code

Raw SurrealQL is fine. DEFINE TABLE, DEFINE FIELD, DEFINE INDEX do what they say. It stops being fine somewhere around the fifteenth table, when your "truth" is a folder of .surql files drifting from whatever your application code thinks the shape is. I'd been bitten by that pattern with Postgres and Alembic and didn't want to repeat it.

So the first thing I built was a way to declare the schema in the language I was writing the application in. In Python that meant Pydantic-flavored builders:

from surql.schema.fields import string_field, int_field, datetime_field
from surql.schema.table import table_schema, unique_index, TableMode

user_schema = table_schema(
    'user',
    mode=TableMode.SCHEMAFULL,
    fields=[
        string_field('name'),
        string_field('email', assertion='string::is::email($value)'),
        int_field('age', assertion='$value >= 0 AND $value <= 150'),
        datetime_field('created_at', default='time::now()', readonly=True),
    ],
    indexes=[unique_index('email_idx', ['email'])],
)

That value compiles to the same DEFINE statements I'd otherwise write by hand, but the truth lives next to the code that uses it. The RELATE example from earlier has a companion. Edges are first-class:

from surql.schema.edge import edge_schema
from surql.schema.fields import datetime_field, int_field

follows = edge_schema(
    'follows',
    from_table='user',
    to_table='user',
    fields=[
        datetime_field('created_at', default='time::now()'),
        int_field('weight', default='1'),
    ],
)

Once the schema was a value I could hold, the rest of the library fell out of needing to do something with it.

Migrations were the next thing I couldn't avoid

Capturing the schema in code is half the problem. The other half is what happens when it changes. For a few months I was writing one-off migration scripts each time I added a column or relaxed an assertion. By the fourth or fifth hand-rolled update_v3.surql I was being dishonest about how often this was going to happen. So I built a diff engine: it compares the schema value in code against whatever is deployed and writes a .surql file with -- @up and -- @down sections.

surql migrate create "Add weight to follows edge"
surql migrate up
surql migrate status

I no longer think about migrations as a thing I write. I think about them as a thing the library notices when the schema drifts from the deployed shape.

Why four languages

I never set out to ship four ports. Python came first, for a private service backend leaning on SurrealDB for graph queries. TypeScript followed when this site needed the same ergonomics from a Deno Fresh app. Then Go, for a control-plane CLI that needed to share schema state with the Python service.

Same shape, different idioms. Here's a fluent select in TypeScript:

import { Op, SortDirection, SurQLClient } from '@oneiriq/surql'

const inactiveUsers = await client.query('users')
  .where({ status: 'inactive' })
  .where('last_login', Op.LESS_THAN, new Date('2024-01-01'))
  .orderBy('last_login', SortDirection.DESC)
  .execute()

And an aggregate in Go, which leans on options structs and explicit context the way a Go programmer would expect:

opts := query.AggregateOpts{
    Table: "match",
    Select: map[string]types.Operator{
        "count": query.CountAll(),
        "avg":   query.MathMean("score"),
    },
    GroupAll: true,
}
rows, _ := query.AggregateRecords(ctx, client, opts)

Each port made me confront a different question. Go pushed me away from method chaining toward options-functions, which I now think reads better for anything past three operators. Python came out cleaner than the others because it had three earlier mistakes to learn from.

Rust was the most recent and probably the most strategically important. I wrote it specifically to unblock things the other ports couldn't reach:

use surql::connection::{ConnectionConfig, DatabaseClient, LiveQuery};
use futures::StreamExt;

let mut live = LiveQuery::<serde_json::Value>::start(&client, "task").await?;
while let Some(notification) = live.next().await {
    println!("change: {notification:?}");
}

That's a LIVE SELECT subscription over WebSocket. The kind of thing I want for an IoT platform I'm building, where edge devices subscribe to schema-side changes rather than poll. The Rust port gave me that, plus a single static binary I can drop on a 4GB ARM SBC without dragging a runtime along.

The Go port has the same live-query surface in code, but the round-trip is currently gated by an upstream surrealdb.go shutdown race the integration test explicitly skips, so live queries from Go are still "works locally, don't ship it" until that lands. The ports are not at parity, and choosing the right one for a given service still matters.

What writing the library taught me

I didn't expect a client library to change how I thought about my own code, but it did. Schema-as-code gave me one place to look when I needed to know the shape of the system. The migration tool and the query builder both work because the schema is a value I can hold and pass around. Migration discipline turned out to be the thing that lets me change my mind about a model without panic.

The right language for the job matters. Python is where I do data work. Rust is where I land when I need tiny hardware or long-lived WebSockets. Go and TypeScript fill in around them.

The library was a side effect of using the database honestly. I just wanted to stop copy-pasting the same fifty lines of db.ts into every new project. Everything else happened because the first thing I wrote made it cheaper to write the second.