HiteshAI software engineer
LLM agents, retrieval and distributed ML, taken from the first sketch to a service that runs across many machines. Scroll and follow him down the runway — one build end to end, then open the book of roles at the end of the road.
01 / 14Design
Before any code I write down the expected load, the latency budget and the failure I fear most. The architecture comes from those limits. Authentication and authorization go into that first sketch, not onto the end of the project.
02 / 14Code
ReAct loops, MCP tool servers and retrieval pipelines, split into pieces that can be tested alone and scaled on their own. The agent only sees a model client, so a hosted model or a self-hosted one can sit behind it.
class Agent: async def run(self, goal): for _ in range(8): # step budget step = await self.llm.next(history) # think if step.final: return step.text obs = await self.mcp.call(step.tool) # act history.append(obs) # observe
03 / 14Data
Documents are chunked and embedded in parallel by Ray workers, then spread across index shards by key. Each shard keeps a copy of its neighbour, so one lost machine loses no data. Add a shard and very little has to move.
Each document hashes to one shard. The bar is the primary copy, the thin line below it is the replica of its neighbour.
04 / 14Tests
Unit tests cover the plumbing. Contract tests cover every tool an agent can call. Evaluation sets replay real tasks, so a prompt change, a new model version or a weaker retriever shows up as a failing test, not as a bad answer in production.
05 / 14CI/CD
A merge to main starts the pipeline. Nothing reaches users unless each gate passes, and nobody deploys by hand.
Push a commit and watch it walk the gates in the scene.
06 / 14Staging
The new version meets a small slice of production first. It earns more traffic only while error rate, latency and answer quality hold up against the old version — and the moment a gate fails, it rolls itself back.
Promoting automatically while every gate holds.
07 / 14Deploy
I built and scaled Ray-based AI infrastructure on Kubernetes across CPU and GPU clusters. LLM training, inference and agent workloads share it, and autoscaling adds or removes worker pods as compute demand moves.
Ray asks for worker pods. Kubernetes adds a node when there is no room, and removes idle ones afterwards.
08 / 14Distribute
Requests fan out across nodes and every shard lives on two of them. When a node dies, traffic reroutes and the missing copies are rebuilt from replicas. A failed request goes back to a durable queue and is retried until it is served, so nothing is lost. Set the per-node capacity, kill one node or all of them, then add nodes back and watch the queue drain.
Accepted 0 = served 0 + in flight 0 + queued 0. Dropped: 0.
Capacity is 3 req/s per node. Failure rate is 0.5 / node count, so one node fails about half the time and eight nodes about 6%. A failed request returns to the durable queue and retries until it is served — nothing is dropped.
09 / 14Scale
Autoscaling adds replicas as demand climbs and removes them when it falls. Scaling up is quick, scaling down is slow on purpose, so a spike never meets a half-empty fleet.
Scaling up is quick, scaling down is slow on purpose. Target is 70% utilization per replica.
10 / 14The record
PROVEN ATSiemens · Manentia AI · IEEE Xplore · Dalhousie
I grew up in Vadodara and studied computer engineering there, then crossed the ocean to Halifax for a Master's in Applied Computer Science at Dalhousie. Today I live and engineer in Saskatoon, building retrieval systems and the infrastructure they run on.
My work sits at the seam between machine learning models and the people who depend on them: RAG pipelines with verifiable citations, AST-aware chunking, asynchronous streaming endpoints, and interfaces that turn probabilistic outputs into answers an engineer can trust. At Siemens I architect enterprise RAG systems end to end; at Manentia AI I built high-throughput Django APIs for DICOM medical imaging; at Mobiuso I learned to profile and remove every bottleneck a web app can hide.
Python · TypeScript · JavaScript · Java · SQL
React · FastAPI · Django · Spring · Node.js
PyTorch · TensorFlow · OpenCV · YOLO · RAG / embeddings
AWS · Google Cloud · Docker · Jenkins · PostgreSQL
11 / 14Experience · the book
EX LIBRIS
Kept from 2021 to the present.
Three companies, one through-line:
making ML systems fast, resilient and dependable.
Flip the cover →
COLOPHON
Set in Instrument Serif and IBM Plex Mono.
Printed by WebGL in your browser.
Nothing here lives on a server — even the search runs on this page.
The road continues below ↓
A FLIPBOOK IN THREE CHAPTERS
Hitesh Gupta · Software Engineer, AI
2021 — PRESENT
CHAPTER I
Software Engineer Jan 2021 – Jul 2022
Angular · Node.js · Django · Profiling
CHAPTER II
Software Engineer, ML Ops Aug 2022 – Jul 2023
PyTorch · TensorFlow · DICOM · Django · AWS
CHAPTER III
Software Developer, AI Sep 2024 – Present
FastAPI · ChromaDB · React · Jenkins · RAG
CURRENT ROLE
INTERLUDE
Three roles · one habit: make it fast, then make it dependable.
FIELD NOTES
WHAT COMES NEXT
The certificates, the IEEE paper and the project index are on the shelves just past this page — each one linked and searchable.
Turn the last page →
THE END
…of the book, not the road.
The projects and credentials are waiting on the next pages.
Keep scrolling ↓
Keep scrolling — the pages turn with you. Or drag across the page.
12 / 14Selected work
Everything below was engineered the way production software is engineered — latency budgets, failure maps and observability decided before the first commit, then held to after launch. Press / and search dicom, yolo or chromadb — every system is in the semantic index.
LIVE DEMO · REAL RETRIEVALRAG Codebase Assistant
A retrieval-augmented codebase assistant: semantic search plus code-aware retrieval, so developers can navigate large repositories by asking questions and getting cited files, line ranges and confidence scores back.
ARCHITECTURE Local Vector Store (HNSW) + Tree-sitter AST Chunker + Streaming FastAPI
HURDLE SOLVED Hierarchical chunking that preserves function scope without context dilution.
Real-time waste detection and categorization from video surveillance feeds, architected for suburban municipal sanitation. Using a custom-trained Darknet-53 backbone and OpenCV telemetry, the system automates CCTV collection decisions at 30 FPS.
Read on IEEE Xplore↗
Peerreviewed
IEEE REPOSITORY ARCHIVE
13 / 14The evidence board
Ten exhibits, pinned and strung together — the certifications behind the cloud chapters, the courses behind the models. Walk the board, stamp what you visit, and open any exhibit to verify it at the issuer.
04 · GDSC CloudGoogle CloudVERIFIED
05 · PythonUdemyVERIFIED
06 · CNNsCourseraVERIFIED
07 · AWS MLCourseraVERIFIED
08 · NLPCourseraVERIFIED
09 · AI MedicalCourseraVERIFIED
EXHIBIT D · RESEARCHThe origin fileIEEE paper — Cleanliness Automation: YOLOv3open the folder ↓FILED
Exhibits A–C are the professional certifications, 04–09 the course record, and D the research that started this whole case.
14 / 14The lab
The workbench behind the shipped systems — a periodic table of real repositories, open on GitHub since 2019. Hover an element to inspect it, click to open the source.
Live from github.com/hmgtech. The six production systems live in the Work chapter — everything else is on this table.
ToolboxContact