Mohith (Matt) Das
01 — Selected work
Projects
LLM Distillery
Shrinking a language model small enough to run on an 8 GB edge device without losing what makes it useful.
The constraint
Everything trains and runs on a single Jetson Orin Nano with 8 GB of unified memory. That ceiling ruled out full fine-tuning and forced sequential LoRA passes: one for cognitive distillation on teacher reasoning and code traces, a second for personal alignment, merged into one FP16 checkpoint.
Re-architecture
Every attention layer in the merged model was replaced with an alternating scheme of Sliding Window Attention (O(W) cache) and Gated Linear Attention (O(1) cache, learnable per-head decay), warm-started from the original Q/K/V/O projections and running a chunk-parallel recurrence for the linear layers.
Reading the paper, not the blog post
TurboQuant MSE and inner-product quantizers were reimplemented from the mathematics in arXiv:2504.19874 and validated against the distortion numbers the paper itself reports, then applied to K/V cache compression and a compressed retrieval index. Measured 97.5% token agreement with the FP16 baseline at 4-bit K/V.
CUDAQuant-Jetson
A quantitative trading research platform built around one problem: how do you stop a backtest from lying to you?
The actual hard part
Building a model that looks brilliant on historical data is easy, and worthless. The platform is built around preventing that: walk-forward validation with purge and embargo gaps, strict chronological signal computation, and three separate leakage checks for lookahead, target leakage, and future normalization.
Lifecycle, not just training
Models move through a candidate to challenger to champion registry with DuckDB-backed lineage and metrics. Regime detection splits the market four ways (trending or ranging, high or low volatility) and attributes strategy performance per regime, so a strategy that only works in one regime cannot hide inside an aggregate number.
Where the GPU actually helps
22 CPU feature functions and 9 CUDA C++ kernels, with dispatch thresholds set by measurement rather than assumption. Rolling min and max move to GPU above 1,000 points; rolling mean never does, because the CPU path is O(n) and always wins. Batch feature computation reaches 4.2x. 179 tests, live trading off by default behind explicit gates.
glass_box
A local LLM server that shows you what inference is actually doing: quantization, cache reuse, and per-layer activations, live.
Why two models
A hard 8 GB budget forced the design. One FP16 model plus activation visualization would not fit, so serving runs on a 1.5B model quantized to INT4 under llama.cpp while a separate 0.5B FP16 shadow model carries the PyTorch forward hooks. The split is the interesting decision, not a workaround.
What it measures
Session-pinned KV-cache reuse is surfaced per request as cached versus fresh token counts. Across one three-turn session the cache hit rate climbed from 62% to 79% and generation speed from 3.3 to 4.5 tokens per second as the prefix accumulated. Numbers observed, not estimated.
Seeing inside
Per-layer activations are projected through a precomputed PCA basis of 32 directions per layer and streamed as a 24 by 32 layer and cluster grid over WebSocket, so quantization behavior, cache reuse, and activation patterns are all visible in one place instead of inferred from throughput.
An MCP server that runs a full job search pipeline: portal scanning, JD scoring, resume rendering, outreach drafting. 51 tools, one SQLite file.
Read the case study →Many concurrent clients, one process, one database. Writes serialize through a single lock.
Architecture
Three planes in one process: MCP tools over stdio or streamable HTTP, an HTTP plane serving rendered artifacts and a live tracker dashboard, and a SQLite data plane in WAL mode with every write serialized through a single lock. One long-running server supports many concurrent MCP clients (Claude Desktop, Claude Code, Cursor, LibreChat) against the same database.
What's real
51 tools: ATS portal scanning (Greenhouse, Ashby, Lever, Workday) via Playwright, a two-step JD evaluator with an editable scoring rubric, LaTeX to PDF, docx and HTML resume rendering with custom themes, and a versioned career packet with bidirectional sync between the database and source files.
Design decision
Human in the loop is enforced in code, not just documented. No tool auto-submits an application or sends a message; every outward action stops at a draft or preview, and outreach drafts are validated against hard safety rails before they persist.
npx @mohith_das/jobops@latest initThe self-hosted platform behind this site: a portfolio with an AI concierge that recruiters and their agents can query over MCP.
Read the case study →Two endpoints, two trust levels. The public mount registers no write tools, so an injected prompt has nothing to call.
What it is
A self-hosted portfolio platform, and the software serving the page you are reading. Alongside the human-facing site, a public read-only MCP endpoint exposes the same project and case-study data to recruiter-side AI agents.
Design decision
Prompt-injection defense is handled server-side through tool allow-listing: the public concierge only ever sees a fixed, read-only tool surface, so no crafted prompt can escalate it into writes or reach private data.
Under the hood
Next.js monorepo with an Express orchestrator, SQLite storage, and multi-provider LLM support, so the concierge is not tied to any single model vendor.
git clone https://github.com/HireBridge/LivingCV && ./scripts/start.shA single-binary Go service that lets recruiter AI agents discover and search self-hosted candidate nodes.
Read the case study →Candidates keep custody of their data. The hub caches only snapshots it can cryptographically verify.
What it is
The recruiter-facing half of a federated hiring model: candidates keep their data on nodes they control (like LivingCV instances), and HireBridge gives recruiter agents one place to discover and search across them.
Under the hood
A single Go binary running in roughly 40 MB RSS. Search combines SQLite FTS5 keyword matching with sqlite-vec embeddings, merged by reciprocal rank fusion. Candidate snapshots are ed25519-signed so provenance is verifiable, and agent access uses the OAuth 2.1 device flow.
A git-backed markdown wiki served over MCP, built as durable long-term memory for LLM agents.
Every write becomes a git commit, so agent memory can be diffed, reverted, or read in any editor.
What it is
A personal wiki that LLM agents read and write over MCP. Every page is plain markdown in a git repository and every write is auto-committed, so agent memory has full history and can be diffed, reverted, or read by a human in any editor.
What's real
BM25-ranked retrieval and full-text search, graph tools that surface hubs, orphans, and shortest paths between pages, and a multi-user server mode with bearer-token auth so one wiki can back several agents or people.
pip install wikimcpA multi-agent simulation engine built around one question: how cheap can a swarm run?
Read the case study →Sparse activation: agents a tick does not need never leave storage, and never cost a token.
The question
Multi-agent systems burn tokens by default. Lean Swarm measures and minimizes that: tiered model routing sends cheap steps to cheap models, and sparse activation wakes only the agents a simulation tick actually needs instead of running everyone every tick.
Under the hood
Hybrid numeric plus text state, SQLite-backed hierarchical memory with vector search, and a benchmark harness that compares lean activation against the naive all-agents-every-tick baseline so the cost savings are measured, not asserted.
pip install leanswarm02 — Also shipped
Smaller tools
Hierocode
A hierarchical coding agent: a frontier model plans and reviews, a local model writes the code.
wiview
WiFi sensing on an ESP32-S3: presence, motion, and breathing detection from channel state information, no camera involved.
LinkWell Signature
A community platform connecting Western New York seniors and families to local providers and services, with a voice agent for people who struggle with web forms.
CliDeck macOS Tray
A native macOS menu-bar monitor for AI coding agents: Node core, compiled Swift launcher.
03 — Before this
Background
I build machine learning systems and the infrastructure that keeps them honest. Most of it runs on hardware with a hard memory ceiling, which is where the interesting decisions live: what to quantize, what to cache, and what a model actually needs versus what it was handed.
Before that, five years as a PM and analyst, most recently owning the Pulse BI product at Saras Analytics, the primary analytics surface for 100+ e-commerce brands, where I also built self-hosted AutoGen and LlamaIndex agents that removed roughly 20 hours a week of manual reporting.
Since July 2026 I have run client engagements independently through Annseraf, shipping production systems for a luxury retailer and a field-service crew. The through-line: ship the data product, then build the AI on top of it.
Timeline
Machine readable
04 — Machine readable
This portfolio speaks MCP
Everything on this page is also machine-readable. The site runs LivingCV, my self-hosted portfolio platform, which exposes a public, read-only MCP endpoint. Point Claude, ChatGPT, or any MCP client at it and your agent can query my career packet and story bank directly, the same way you would ask me.
One thesis across the projects: candidates keep custody of their data, and agents query it where it lives.