Thoughts, stories and ideas.
A practical walkthrough of a durable, hybrid memory layer for LLM-backed products — and why the architecture matters beyond one domain. The problem: LLMs have no memory Large language models are brilliant in the moment and amnesiac between moments. Every request to a model is stateless: the weights don't
Anyone who has shipped an LLM-powered product runs into the same wall: conversations grow, context windows do not. The first ten turns are fast and pleasant. By turn eighty the prompt is bloated, latency creeps up, costs scale linearly with every new message, and eventually the model starts dropping the
I'm equal parts ML Engineer and backend engineer. I think in models, services, queues, and databases. The browser has always been someone else's problem. So when I decided to build PromptMask — a privacy-first Chrome extension powered by on-device AI — I expected the frontend world to feel
Governance, risk allocation, and system design are doing more work than your benchmark score. There is a particular kind of confidence that comes from a good safety benchmark score. Your model refused harmful prompts. It passed the red-team suite. The evaluation report is clean. You ship. [1] Then something goes
A lot of the best ML data is the least portable. Hospitals can’t centralize patient records. Banks can’t ship transaction logs. Phones generate rich signals that never leave the device. Even when data movement is technically feasible, it’s often blocked by privacy commitments, regulation, contracts, or internal