// journal
Written by the engineers doing the work, usually shortly after something broke in an interesting way. No trend pieces, no predictions about the next five years — just what we hit, what we tried, and what the numbers said afterwards.
Showing all 6 posts
Fixed-size chunking cuts a financial table in half and your retriever hands the model a column of numbers with no header attached. Structural chunking, table summarisation, and the recall numbers before and after.
Humans notice a pause at about 500ms and start talking over you around 1.2 seconds. The full budget hop by hop — endpointing, ASR, inference, TTS — and the three places we actually found the time.
One synchronous tokenizer call inside an async def endpoint took a
service from 400 requests per second to 12. How to find that call, how to fix it,
and why StreamingResponse alone won't save you.
A webhook that fires twice should not create two invoices. How we make every write in an automation safe to repeat, and the dead-letter queue that catches the ones that aren't.
Regulatory filings arrive as scanned PDFs, sometimes bilingual, frequently rotated. The OCR pass, the typed extraction schema, and why we let the model return null instead of guessing.
At what corpus size does a dedicated vector database start to earn its operational cost? Benchmarks at 10k, 500k and 8M chunks, plus the boring answer that fits most teams.
Nothing published in that category yet — show all posts.
// start here
We're usually happy to talk through an approach even if it doesn't turn into a project.
Book a call