# Ishan — full content

> Software engineer writing about systems, performance, distributed inference, and building reliable products.

This file concatenates clean text from site pages for LLM / agent context.
Prefer /llms.txt for a curated index; use per-page index.html.md for single pages.


---

# Reducing Bucket Costs: The Content-Type Mistake That Hides in plain sight

> Skipping Content-Type on S3 uploads defaults everything to application/octet-stream. Your app works fine—until a CDN, caching layer, or metadata cleanup turns a one-line fix into a real AWS bill.
Published: 2026-08-13
URL: https://hugo-portfolio-teal.vercel.app/blogs/reducing-bucket-costs/
Markdown: https://hugo-portfolio-teal.vercel.app/blogs/reducing-bucket-costs/index.html.md

In both of my first two companies, I observed a common mistake.
When uploading to S3 buckets, no Content-Type was being set on the object.
And what happens when you don’t pass it?
The bucket assigns application/octet-stream as the content type.
And everything continues to work perfectly.
Yaay 🎉
Your object is 2 MB.
You download it from the browser.
It downloads.
The UI works.
All the test cases pass.
Everything is beautiful.
Yaay.
No.
Naay.
The problem is that the journey from an S3 object to your browser is not simply:
S3 → Browser
There is a whole pipeline sitting in between.
The object gets fetched, transferred, potentially cached, processed by CDNs, passed through networking layers, and eventually interpreted by the browser.
And when all these systems see:
Content-Type: application/octet-stream
they basically say:
“Okay… it’s some binary data. 🤷”
They don’t really know what they’re dealing with.
But when you tell them:
application/json
or
image/jpeg
or
text/css
or any other appropriate content type, you’re giving them useful information.
Now the systems in the pipeline know what they’re dealing with.
And that allows them to use the optimized paths available for that particular type of content.
You get better handling.
You get better performance.
And the best part?
You didn’t pay anything extra for this optimization.
Just a tiny metadata field during upload.
Beautiful.
Until one day…
You decide to put a CDN in front of your bucket.
And suddenly you look at your S3 bucket and realize:
“Wait… why is everything application/octet-stream?”
😭
And now begins the cleanup operation.
You need to update the metadata of all those existing objects.
Which means touching a potentially huge number of objects.
Which means more requests.
More data transfer.
More processing.
And suddenly that tiny mistake you made months ago has become a very real line item on this month’s AWS bill.
AWS billing has entered the chat.
😭
This is one of those engineering mistakes that is particularly dangerous because nothing breaks.
Your application works.
Your tests pass.
Your users don’t complain.
And that’s exactly why it survives for months.
Until one day you introduce something new—usually a CDN, some caching layer, or a different way of serving the objects—and suddenly you discover the little decision you made during an upload has been following you around this entire time.
So, whenever you’re uploading objects to S3—or any object storage for that matter—be explicit about the Content-Type.
It’s a tiny thing.
It takes almost no effort.
And it can save you from a surprisingly expensive cleanup later.
Don’t let application/octet-stream become your technical debt’s hiding place.
Stay happy and stay blessed.
I know you are awesome ❤️


---

# I built an inference engine from scratch. Here is what I learnt.

> A practical build log of a coordinator-worker LLM inference engine: KV-cache locality, routing, retries, streaming, and why the hard part is not calling the model.
Published: 2026-08-05
URL: https://hugo-portfolio-teal.vercel.app/blogs/inference-engine-from-scratch/
Markdown: https://hugo-portfolio-teal.vercel.app/blogs/inference-engine-from-scratch/index.html.md

LLMs feel almost suspiciously simple from the outside.
You send some text to an API. A few seconds later, words start appearing on the screen. It is easy to think the interesting part is the model. Load it, call generate(), send the output back.
That is true until you try to serve it across more than one machine.
Then a slightly annoying question appears: which machine should generate the next token?
The answer is not “whichever one is free.” It is “the machine that already has the state needed to produce that next token.” That one constraint shaped nearly every decision in this project.
I built a distributed inference engine to understand that constraint from first principles. The project uses a TypeScript coordinator and Rust workers, loads GGUF models through llama.cpp, streams tokens with SSE, and keeps conversational state pinned to the worker that owns it.
This is not me trying to rebuild vLLM in a weekend. It is a deliberately small system for learning the systems problems hiding behind “generate text.”
First, what even is an inference engine? # An inference engine is the runtime layer between an application and an LLM. It accepts a prompt, prepares the model state, generates tokens, streams them back, and cleans up afterwards.
For one model on one laptop, that can be one process. For multiple workers, it becomes a distributed system with some awkward properties:
model weights are large and live on each worker a conversation creates state as it runs every generated token depends on the ones before it memory, especially KV-cache memory, becomes the resource that decides whether a request is safe to accept My initial goal was intentionally narrower than “distributed model parallelism.” I did not split one model across machines. Each worker runs a full model replica. The coordinator chooses a worker for a request, and that worker owns the inference session.
That gives horizontal capacity for independent conversations while keeping the data plane local.
flowchart TB C[Client] -->|POST /coordinator/infer| CO[CoordinatorTypeScript / Express] CO -->|select healthy worker| S[Scheduler] S --> CO CO -->|prefill + decode| W1[Rust worker 1model + KV cache] CO -->|prefill + decode| W2[Rust worker 2model + KV cache] CO -->|SSE tokens| C W1 -->|heartbeat + load| CO W2 -->|heartbeat + load| CO The important thing in that diagram is what the coordinator does not own: model weights, token-level execution state, and the KV cache. Those belong to workers.
The thing that changed the design: prefill is not decode # Before writing the services, I had to understand the inference loop well enough to draw boundaries around it.
There are two phases.
Prefill processes the prompt. It tokenizes the input, runs it through the model, and creates the attention state required for future generation. That state is called the KV cache.
Decode generates one token at a time. Each new token depends on the cache and on the tokens generated before it. This is the part users see streaming back.
sequenceDiagram participant Client participant Coordinator participant Worker Client->>Coordinator: prompt + conversation_id Coordinator->>Worker: prefill(session_id, prompt) Note right of Worker: build / extend KV cache Worker-->>Coordinator: prefill complete Coordinator->>Worker: decode(session_id) loop one token at a time Worker-->>Coordinator: SSE token(seq) Coordinator-->>Client: SSE token(seq) end The KV cache is not some optional optimisation I could bolt on later. In systems terms, it is the in-memory continuation state of a request. If Worker A created it, moving decode to Worker B means B has no idea where that conversation is. It either needs the cache moved over the network or it has to replay the prompt.
Both options are expensive. So I locked in an invariant early:
Decode always runs on the worker that owns the session’s KV cache.
That one invariant gave me sticky sessions, worker-local cache lifecycle, and very honest failure behaviour. It also explains why “stateless workers” are not enough for this job.
Starting with boundaries, not clever code # I began with the boring documents: architecture, state ownership, request lifecycle, invariants, and failure modes. It sounds slow, but it prevented the codebase from becoming a coordinator that secretly knew everything.
The three main pieces are simple.
Component Owns Explicitly does not own Coordinator request flow, conversation registry, streaming, admission control model weights, KV cache, model execution Scheduler a soft view of worker health/load and the selection policy sessions, cache, durable truth Worker model, tokenizer, session, KV cache, prefill and decode client requests, global routing I also wrote down a few rules that are more valuable than they first look:
KV cache never moves across workers. A conversation stays pinned to its worker while its session is alive. A worker failure affects that worker’s sessions, not everyone else. The scheduler makes routing decisions but does not mutate inference state. A slow client must not be allowed to slow model execution forever. These are not implementation details. They are guardrails. When an edge case shows up, I can ask which rule it would violate instead of improvising state transfers until the demo works.
Building the request path # With those boundaries in place, the first version was intentionally mocked. The worker accepted prefill, stored fake state keyed by session ID, and accepted decode calls. Only after the contracts existed did I connect a real GGUF model.
The Rust worker now uses llama_cpp to load a model, create an inference session, advance that session with the prompt during prefill, and generate completion tokens during decode. The coordinator exposes POST /coordinator/infer, then talks to worker endpoints for prefill and decode.
New conversations go through this path:
flowchart TD A[New request] --> B{System can admit it?} B -- no --> R[503: reject early] B -- yes --> C[Find healthy workers] C --> D[Filter workers at capacity] D --> E[Score available workers] E --> F[Prefill on selected worker] F -- worker rejects/fails --> G{Another worker left?} G -- yes --> E G -- no --> H[Return failure] F -- success --> I[Register conversation -> worker + session] I --> J[Decode and stream tokens] For existing conversations, the path is deliberately different. The coordinator first looks up the conversation ID. If the worker and session still exist, it sends a continue prefill request to that exact worker, extending the same context instead of starting over elsewhere.
If the session is gone or full, the API returns a clear reset-required response. Pretending that the conversation could seamlessly continue would be worse than admitting that the state is gone.
Scheduling: load balancing, but memory-aware # At first I thought scheduling would be the centrepiece: complex heuristics, lots of scores, maybe a clever algorithm.
The useful version is much more boring.
Workers send heartbeats containing their identity, URL, liveness, active-session count, and KV-cache usage. The coordinator marks a worker alive, stale, or dead based on the age of those heartbeats. Before routing a new request it does an inexpensive admission-control check, estimates the cache requirement, and refuses work if the system is already constrained.
For candidates that remain, the scheduler uses a weighted load score:
score = 0.6 × session utilisation + 0.4 × KV-cache utilisation The worker with the lowest score wins.
It is intentionally a pure function over soft state. If the scheduler restarts, it loses only its recent view. Heartbeats rebuild that view; it never owned a model session in the first place.
That separation matters because the system’s scarce resource is usually not the coordinator’s CPU. It is the memory occupied by live model sessions. A worker can look free in terms of requests while being unable to take another long context safely.
Failures were a feature, not a footnote # The first honest question I asked was: what happens when a worker disappears mid-request?
There is no magical answer. There are only trade-offs.
If a worker fails during prefill, the coordinator can try a different healthy worker. The session has not become useful yet, so redoing the prompt is acceptable.
If it fails during decode, the KV cache vanished with it. The stream fails and the conversation is invalidated. A future request must start a fresh session. In theory I could re-prefill the full conversation automatically; in practice, making that transparent needs a durable source of chat history and a conscious product decision about replay cost. I chose to surface the truth.
stateDiagram-v2 [*] --> Prefill Prefill --> Decode: session created Prefill --> RetryElsewhere: worker failure RetryElsewhere --> Prefill Decode --> Complete: final token Decode --> SessionLost: worker failure SessionLost --> [*] Complete --> Reusable: keep session for next turn Reusable --> Prefill: continue prompt Reusable --> Expired: TTL / eviction Expired --> [*] This approach is less flashy than claiming fault tolerance everywhere, but it gives each failure a defined blast radius. Other workers, their models, and their caches remain untouched.
Streaming is its own distributed-systems problem # Getting tokens from a worker to a client sounds easy until the client is slow.
The worker generates a numbered stream of SSE events. The coordinator reads that stream, checks sequence numbers, puts tokens into a bounded buffer, and writes them to the client with a deadline. It also records buffer occupancy, write latency, dropped tokens, and stream lifecycle events.
flowchart LR W[Worker generates token] --> CH[Bounded channel] CH --> WS[Worker SSE stream] WS --> R[Coordinator reader] R --> B[Bounded buffer64 tokens] B --> WR[Deadline-bound writer] WR --> CL[Client] B -. overflow .-> DROP[Drop oldest + log] WR -. timeout .-> END[End client stream] The decision here was important: protect the worker before protecting a stalled client. A slow browser should not make a model process accumulate an unbounded amount of output in memory. The coordinator can end that client stream while keeping the worker and other sessions healthy.
There is a second subtlety: a normal completed stream, a client disconnect, and a hard decode failure are not the same thing. A normal completion can leave the session alive for the next turn. A hard decode failure tears it down. Making those paths explicit stopped the conversation registry from becoming a pile of optimistic assumptions.
The implementation choices # The split was practical rather than ideological.
TypeScript + Express at the edge made the HTTP API, request validation, orchestration, and SSE forwarding quick to iterate on. Rust + Axum on workers provided a compact place for the model process, sessions, cache limits, heartbeats, and streaming. HTTP and SSE kept the protocol inspectable while I was learning. I can curl the health endpoints and watch a stream without a special client. GGUF + llama.cpp via Rust bindings made it possible to use small, quantized local models instead of making the project dependent on a GPU-only serving stack. Docker Compose gives the coordinator and worker a portable demo path, while the same URLs and heartbeat contracts can support more workers. There are deliberately approximate pieces too. For admission control, cache size is currently estimated from prompt length rather than measured from the backend’s exact allocation. That is good enough to learn the control flow, but real production capacity planning would use model-aware token counts and actual memory telemetry.
What I would change next # The system works as an educational inference platform, but “works” should not be confused with “finished.”
First, I would replace rough cache estimates with tokenizer- and model-specific accounting. Then I would add proper cancellation so a disconnected client can cancel generation work, not only stop receiving it. I would also make the coordinator highly available or persist the minimal session metadata needed for recovery.
For higher throughput, the next serious piece is continuous batching: admitting and scheduling prefill/decode work in small batches while preserving per-session ordering. That is much more interesting than blindly adding more workers.
I would still avoid tensor parallelism in this project’s first version. Splitting one model across devices introduces another layer of communication and synchronisation. It solves a different problem: fitting or accelerating a single model that cannot comfortably run on one worker. Here I wanted to understand request-level distribution and state locality first.
The biggest lesson # I started this project thinking distributed inference was mainly about sending requests to multiple machines.
It is not.
It is about deciding where state is allowed to live, making that ownership obvious, and being truthful when that state disappears. The model call is one part of the system. The harder part is everything that lets that call survive real clients, full memory, slow networks, stale workers, and the next turn in the conversation.
That is what made building this fun: every “simple” decision had a systems consequence hiding behind it.
The code and the design notes are open source at ishanjain1502/distributed-inference-engine. If you are building something similar, I would start with the invariants. They are much cheaper to change in a Markdown file than after your KV cache has quietly become everyone’s problem.


---

# AI solved a hardware issue that even replacing the monitor couldn't fix 🤯

> An AI assistant diagnosed summer monitor flicker as an electrical grounding issue after cables, drivers, firmware, and a full hardware replacement all failed.
Published: 2026-06-26
URL: https://hugo-portfolio-teal.vercel.app/blogs/debugging-monitor/
Markdown: https://hugo-portfolio-teal.vercel.app/blogs/debugging-monitor/index.html.md

One of my friends bought a new monitor a few months ago.
Everything worked perfectly in the beginning, but as soon as the summers started, the monitor began flickering randomly.
Not every few minutes, not after heavy usage, just… randomly.
Like every engineer, the first instinct was, “This has to be a software problem.”
So the debugging started.
The usual debugging steps # Before assuming anything fancy, we tried almost everything that normally causes display related issues.
Different HDMI and DisplayPort cables. Different operating systems. Different laptops. Updated GPU drivers. Updated the monitor firmware. Factory reset of the monitor. Nothing worked.
At this point, it looked like the monitor itself was faulty.
So we contacted customer support.
After explaining all the troubleshooting steps, the company agreed to replace the monitor with a completely new unit.
Problem solved?
Not really.
The brand new monitor started showing the exact same flickering.
Now things became interesting.
If two different monitors show exactly the same behaviour, then the monitor probably isn’t the problem.
So what is?
Hours of searching forums # The next few days were spent doing what every developer eventually does.
Searching StackOverflow.
Searching Reddit.
Searching random forums from 2014.
Watching YouTube videos.
Trying fixes that made absolutely no sense.
Still nothing.
By this point we had ruled out almost every obvious possibility.
One night, instead of searching another forum, he decided to ask Claude.
But instead of asking,
“Why is my monitor flickering?”
he described everything.
Every troubleshooting step.
Every replacement.
Every failed attempt.
Every observation.
Including one detail that almost everyone ignored.
The monitor only started having issues in the summers.
The question that changed the whole debugging direction # After reading everything, Claude didn’t immediately suggest another driver update or firmware patch.
Instead it asked a simple question.
Does anything else change in your setup during summers?
At first this sounded unrelated.
Then it asked another question.
What does the electrical switchboard where your monitor is plugged into look like?
The answer was something like this.
Monitor power plug Fan switch Fan regulator A few other switches And then Claude pointed out something that nobody had considered.
During winters, the fan regulator is almost never used.
During summers, it is.
It suggested that the older smooth rotary fan regulator could be introducing electrical noise or electromagnetic interference, affecting the monitor connected right beside it.
Honestly, this sounded too random to be true.
But after trying literally everything else, it was worth testing.
The fix # The next day, we replaced the old smooth fan regulator with a modern step regulator.
That was it.
The screen flickering completely disappeared.
No new monitor.
No new cable.
No firmware update.
Just replacing a fan regulator costing a few hundred rupees.
Was AI magically correct? # Not exactly.
The interesting part wasn’t that Claude somehow knew the answer.
The interesting part was that it noticed a pattern we completely ignored.
We were so focused on what was changing inside the computer that we never stopped to ask what was changing around the computer.
The monitor wasn’t behaving differently.
The environment was.
The exact electrical mechanism would require proper instruments to verify. The regulator may have been introducing electrical noise or electromagnetic interference that the monitor happened to be sensitive to. We can’t say with certainty that this was the only reason.
But replacing the regulator fixed the problem.
And that’s enough evidence to know that the environment mattered.
Final thoughts # This wasn’t a story about AI replacing engineers.
It was a story about asking better questions.
Sometimes debugging isn’t about knowing another command to run or another setting to toggle.
Sometimes it’s about noticing that the problem only happens in summer.
That one observation changed the entire debugging direction.
And honestly, that’s probably one of the best examples I’ve seen of using AI as an engineering partner rather than just a chatbot.


---

# The Janitor, The Librarian, and The Rustacean: How Languages Manage Memory

> Comparing garbage collection, manual memory management, and Rust ownership — how JavaScript, C-style languages, and Rust handle memory differently.
Published: 2026-06-14
URL: https://hugo-portfolio-teal.vercel.app/blogs/memory-management-frenzy/
Markdown: https://hugo-portfolio-teal.vercel.app/blogs/memory-management-frenzy/index.html.md

I’ve been writing JavaScript for years.
I’ve created objects, arrays, functions, closures, promises, maps, sets, and enough React components to make Chrome cry.
Yet if you had asked me a few years ago:
“What happens to all that memory once you’re done using it?”
My answer would’ve been something along the lines of:
“I don’t know. The browser figures it out.”
And honestly, that’s not entirely wrong.
The browser does figure it out.
But that realization led me down a rabbit hole of understanding Garbage Collection, why languages like JavaScript and Java have it, why languages like C++ and Rust don’t, and why memory management is probably one of the biggest philosophical differences between programming languages.
Let’s talk about it.
The Hotel Room Problem # Imagine you’re staying in a hotel.
Every time you need a room, the hotel gives you one.
Done with it?
Great.
Now someone needs to clean it.
There are essentially three ways to solve this problem.
Option 1: Clean It Yourself # This is the C/C++ approach.
You check into a room.
When you’re done, you explicitly tell the hotel:
“I’m leaving. Please clean this room.”
If you forget?
The room remains occupied forever.
If you accidentally tell them to clean the room twice?
Chaos.
This is effectively what malloc() and free() are doing in C.
int* number = new int(42); delete number; You allocated memory.
You freed memory.
Everything is your responsibility.
Maximum control.
Maximum power.
Maximum opportunity to shoot yourself in the foot.
Option 2: Hire A Janitor # This is JavaScript.
You don’t clean anything.
You simply stop using the room.
At some point, a janitor walks around the hotel and says:
“Nobody seems to be using this room anymore.”
And cleans it.
That’s Garbage Collection.
The language runtime periodically identifies memory that is no longer reachable and reclaims it automatically.
The developer never explicitly frees memory.
Option 3: Hire An Extremely Strict Librarian # This is Rust.
Instead of a janitor cleaning things later, Rust introduces a librarian who keeps track of exactly who owns every book.
The rules are simple:
Every piece of data has one owner. When the owner goes away, the data is cleaned up. Ownership can be transferred. Multiple readers are allowed. Only one writer is allowed. If you violate any of these rules:
The code doesn’t compile.
Not “throws an exception.”
Not “fails in production.”
It simply refuses to build.
So What Exactly Is Garbage Collection? # Let’s start with a simple example.
let user = { name: "Walter White" }; JavaScript allocates memory for that object.
Now imagine:
user = null; Nobody references the object anymore.
The object still physically exists in memory for a short period.
But eventually the Garbage Collector notices:
“Nobody can reach this object anymore.”
And removes it.
That’s the key concept:
Reachability # Modern JavaScript Garbage Collectors care about whether an object is reachable from the application’s roots.
Roots typically include:
Global variables Active function calls Local variables currently on the stack If an object can no longer be reached from any root, it becomes garbage.
The Mark And Sweep Algorithm # Most modern JavaScript engines use some variation of Mark and Sweep.
The algorithm is surprisingly simple.
Imagine the Garbage Collector as Thanos.
Not the snapping part.
The part where he scans the universe.
Step 1: Mark # Start from all root objects.
Mark everything reachable.
window.user -> address -> city Everything connected gets marked as alive.
Step 2: Sweep # Anything that wasn’t marked gets deleted.
Gone.
Reduced to atoms.
The collector walks through memory and frees everything unreachable.
Which is why the algorithm is called:
Mark and Sweep
Simple name.
Very expensive job.
Why Not Just Count References? # You might think:
“Why don’t we simply count how many references point to an object?”
Many older systems tried exactly that.
let a = {}; let b = a; Reference count = 2.
Remove one reference.
Count becomes 1.
Remove the second.
Count becomes 0.
Delete object.
Easy.
Except for one problem.
The Spider-Man Problem # Imagine Peter Parker and MJ storing each other’s phone numbers.
let peter = {}; let mj = {}; peter.friend = mj; mj.friend = peter; Now remove the external references.
peter = null; mj = null; The two objects still point to each other.
A simple reference counter sees:
“Each object still has one reference.”
So neither gets deleted.
Memory leak.
Mark-and-Sweep solves this beautifully.
The collector asks:
“Can I reach either of these from the roots?”
No.
Delete both.
Problem solved.
The Hidden Cost Of Garbage Collection # At this point GC sounds magical.
And honestly, it is.
But magic has a price.
Someone still has to stop and inspect memory.
Someone still has to traverse object graphs.
Someone still has to clean things up.
This introduces runtime overhead.
That’s why GC-heavy applications sometimes experience pauses.
Modern engines like V8 have become incredibly sophisticated with generational, incremental, concurrent and parallel collection strategies, but the core idea remains the same.
The janitor still needs to work.
He’s just gotten much faster.
Enter Rust # Now let’s switch universes.
Imagine if instead of hiring a janitor, we simply never allowed hotel rooms to become ambiguous in the first place.
That’s Rust.
Rust does not use a Garbage Collector.
Instead, it uses a system called Ownership.
{ let name = String::from("Jesse"); } The moment the scope ends:
} The memory is automatically released.
No Garbage Collector.
No runtime scan.
No janitor.
Ownership rules determine exactly when cleanup should happen.
So Which One Is Better? # Neither.
They’re solving different problems.
Garbage Collected Languages # Examples:
JavaScript Java C# Go (with GC) Pros:
Easier developer experience Faster iteration Fewer memory management mistakes Cons:
Runtime overhead Less predictable performance Potential GC pauses Ownership / Manual Memory Languages # Examples:
Rust C++ C Pros:
Greater control Predictable performance Lower runtime overhead Cons:
Steeper learning curve More responsibility Easier to introduce bugs (except Rust, which shifts the pain to compile time) Final Thoughts # One of the biggest realizations for me was that memory management isn’t just an implementation detail.
It’s a language philosophy.
JavaScript says:
“Trust the runtime.”
C++ says:
“Trust the developer.”
Rust says:
“Trust the compiler.”
And that single decision ends up influencing everything from developer experience to performance characteristics.
The next time you create an object in JavaScript, remember:
Somewhere deep inside V8, a tiny janitor is waiting patiently for you to stop using it.
And somewhere in the Rust ecosystem, a very angry librarian is making sure nobody loses track of a book.


---

# Web Crawler

> A small, dependency-minimal Rust web crawler that fetches a seed URL, extracts same-host links from the homepage, and saves HTML responses to disk.
Published: 2026-03-24
URL: https://hugo-portfolio-teal.vercel.app/projects/web-crawler/
Markdown: https://hugo-portfolio-teal.vercel.app/projects/web-crawler/index.html.md

crawler # A small, dependency-minimal Rust web crawler that fetches a seed URL, extracts same-host links from the homepage, and saves HTML responses to disk.
🔍 What it does # Accepts a seed URL (or hostname) as CLI input Fetches the homepage once Extracts <a href="..."> links in the first page Normalizes each link to an absolute URL Follows only same-host links Fetches each same-host page once Saves each response body in out_dir using a deterministic URL hash filename Logs crawl events to stdout with status and byte counts 🧩 Project structure # src/main.rs - CLI entrypoint using clap + tokio src/lib.rs - exposes reusable crawler API src/engine.rs - crawl orchestration src/fetch.rs - HTTP fetch wrapper with reqwest src/links.rs - HTML link extraction src/storage.rs - file path generation, save HTML src/url_util.rs - URL normalization and same-host checks src/log.rs - logging abstraction (stdout + pluggable) ▶️ Usage # Build and run from project root:
cargo run --release -- "https://example.com" --out-dir crawl_out Short form:
cargo run --release -- example.com -o crawl_out Defaults:
out_dir: crawl_out 🚦 Output # crawl_out/<url_hash>.html url_hash is derived from normalized final URL stdout log events include: seed, response, fetch, save, skip_links, link_skip, fetch_err, save_err 🛠️ Configuration # No configuration file. Use CLI args only.
🧪 Tests # No test files are currently included. The library is unit-test-friendly via Crawler::with_logger and CrawlConfig.
📦 Dependencies # reqwest (HTTP client) tokio (async runtime) anyhow (error handling) clap (CLI) url (URL parsing) 💡 Extending # add depth control (breadth-first / recursive crawl) add robots.txt + rate limiting add concurrency queue and dedupe URL set add filter rules (patterns, content types) instrument with structured logging / metrics 📝 Notes # The crawler is intentionally simple, for learning and small local crawl tasks. It is not a production spider and does not enforce politeness controls by default.


---

# Agent Circuit Breaker

> A lightweight Python circuit breaker for agent and LLM calls — monitors failures and temporarily disables expensive operations when a threshold is exceeded.
Published: 2026-03-15
URL: https://hugo-portfolio-teal.vercel.app/projects/agentic-circuit-breaker/
Markdown: https://hugo-portfolio-teal.vercel.app/projects/agentic-circuit-breaker/index.html.md

Agent Circuit Breaker # Circuit breaker pattern for agent/LLM calls: monitors failures and temporarily disables expensive operations when a threshold is exceeded.
Lightweight — zero dependencies for the core library. Sync and async — decorator, context manager, and call() / call_async(). Configurable — consecutive or sliding-window failure counting, custom predicate, fallback, excluded exceptions. Install # pip install -e . Or add to your project and use the package agent_circuit_breaker.
Quick start # Decorator # from agent_circuit_breaker import circuit_breaker @circuit_breaker(failure_threshold=5, recovery_timeout=60) def call_llm(prompt: str) -> str: # your LLM/agent call return response @circuit_breaker(failure_threshold=3, recovery_timeout=30) async def async_call_llm(prompt: str) -> str: return await some_async_client(prompt) Context manager # from agent_circuit_breaker import CircuitBreaker breaker = CircuitBreaker(failure_threshold=5, recovery_timeout=60) with breaker: result = agent.run(task) # async async with breaker: result = await agent.run_async(task) Class-based # from agent_circuit_breaker import CircuitBreaker, CircuitBreakerOpenError breaker = CircuitBreaker(failure_threshold=5, recovery_timeout=60) try: result = breaker.call(agent_function, arg1, arg2) except CircuitBreakerOpenError: result = "Service unavailable" Configuration # Parameter Default Description failure_threshold 5 Number of failures that open the circuit. recovery_timeout 60 Seconds the circuit stays open before a trial (half-open). failure_window None If set, use a sliding time window (seconds) instead of consecutive failures. failure_predicate None Callable (exc) -> bool: only count exception as failure when it returns True. fallback None Callable to run when the circuit is open instead of raising. excluded_exceptions None Tuple of exception types that never count as failures (still re-raised). Exception handling order: first check excluded_exceptions; if the exception is in that tuple, do not count it. Otherwise use failure_predicate if set, else treat as failure.
States # CLOSED — Calls allowed; failures are counted. OPEN — Calls blocked; CircuitBreakerOpenError (or fallback) until recovery_timeout has passed. HALF_OPEN — One trial call allowed; success closes the circuit, failure reopens it. Monitoring # Set callbacks on the breaker:
on_state_change(old_state, new_state) on_failure() / on_success() on_open() / on_close() / on_half_open() Requirements # Python 3.9+ License # MIT


---

# Inference Engine

> Distributed inference framework (coordinator + workers) with memory-aware admission control, backpressure, and failure resilience.
Published: 2026-01-07
URL: https://hugo-portfolio-teal.vercel.app/projects/inference-engine/
Markdown: https://hugo-portfolio-teal.vercel.app/projects/inference-engine/index.html.md

Inference Engine # A distributed inference framework for large language models that routes requests to workers, manages KV cache lifecycle, handles failures gracefully, and applies backpressure so memory — not compute — is the bottleneck.
Note: This is the infrastructure layer. LLM integration is not yet implemented. The system provides the distributed architecture, routing, and session management, but actual model inference needs to be integrated.
TL;DR # What this is: A production-ready distributed inference framework for scaling LLM serving across multiple workers with memory-aware admission control, backpressure handling, and automatic failure recovery.
What this isn’t: A complete LLM inference solution (model integration pending) or a single-server inference engine.
Tech Stack: TypeScript/Node.js (Coordinator) + Rust (Worker) with Express and Axum.
Key Features: O(1) admission control, horizontal scaling, backpressure, session management, heartbeat-based health monitoring.
Use Cases # This framework is designed for:
Scaling LLM inference across multiple GPU workers Memory-constrained environments where KV cache management is critical Production deployments requiring high availability and failure resilience Multi-tenant systems needing session isolation and capacity management Streaming inference with backpressure to handle slow clients gracefully Quick Start # git clone <repository-url> cd inference-engine ./start.sh Open the UI at http://localhost:1337, or test inference with python test_inference.py "What is the capital of France?" / the curl scripts below. See Setup and running for full prerequisites and options.
Setup and running # Prerequisites # Requirement Purpose Node.js 18+ Coordinator (TypeScript/Node) npm Install coordinator dependencies Rust 1.70+ Worker (Rust) — install from rustup.rs LLVM (Windows only) Worker build needs libclang, llvm-nm, and llvm-objcopy for the llama_cpp_sys crate. Install LLVM (e.g. 17.x) and set LIBCLANG_PATH to the LLVM bin directory (e.g. C:\Program Files\LLVM\bin). Also set NM_PATH to the full path to llvm-nm.exe and OBJCOPY_PATH to the full path to llvm-objcopy.exe in the same directory (e.g. C:\Program Files\LLVM\bin\llvm-objcopy.exe), or add that directory to PATH. start.sh derives NM_PATH from LIBCLANG_PATH if set. Docker (Compose v2) Optional: run coordinator + worker without local Node/Rust toolchains 1. Clone and install # git clone <repository-url> cd inference-engine Coordinator (one-time):
cd coordinator npm install cd .. Worker: No separate install step; start.sh (or cargo build) will compile it.
2. Model file (auto-download or manual) # On first start, ./start.sh and Docker Compose auto-download TinyLlama Q4_K_M into modelFiles/ if the file is missing (needs network + curl or wget).
Optional overrides:
MODEL_URL — download URL (default: TheBloke TinyLlama Q4_K_M on Hugging Face) SKIP_MODEL_DOWNLOAD=1 — never fetch; fail if the file is missing (air-gapped / CI) MODEL_PATH — path to an existing .gguf (forward slashes in Git Bash, e.g. E:/Projects/inference-engine/modelFiles/my-model.gguf) You can still download manually into modelFiles/ if you prefer; existing files are never re-fetched. For Docker, Compose mounts modelFiles/ at /models (read-write) and sets MODEL_PATH automatically.
3. Run the system # Option A – Start both with one script (recommended):
./start.sh This will:
Build and start the Coordinator on http://localhost:1337 Build and start the Worker on http://localhost:3001 Press Ctrl+C to stop both.
Option B – Run Coordinator and Worker separately:
Terminal 1 – Coordinator:
cd coordinator npm run build npm start Terminal 2 – Worker (from project root):
# Optional: set model path (use forward slashes on Windows in Git Bash) # export MODEL_PATH="E:/Projects/inference-engine/modelFiles/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf" cd worker cargo build cargo run Option C – Docker Compose (recommended for portable demos):
Prerequisites: Docker with Compose v2. An empty modelFiles/ is fine on first run — the worker entrypoint downloads the default GGUF into the mounted volume (may take several minutes). The volume is read-write so the file persists on the host.
docker compose up --build Open the UI at http://localhost:1337. Press Ctrl+C to stop (or docker compose down if detached).
Only the coordinator port (1337) is published. Compose starts two workers (worker-1, worker-2) with unique heartbeat IDs and URLs (http://worker-1:3001, http://worker-2:3001) so the coordinator can pin sessions and fail over if one process dies. Both containers use restart: unless-stopped.
Do not run docker compose up --scale worker=2: a shared service name would collide on WORKER_ID / WORKER_URL. To add a replica, copy a worker-N service block, give it a new WORKER_ID and WORKER_URL, and list it under the coordinator’s depends_on. Each worker loads its own copy of the model — RAM scales with N.
Override the model with MODEL_PATH / MODEL_URL in docker-compose.yml or a Compose .env if you use a different file or URL.
4. Test the API # Browser UI: open http://localhost:1337 — enter a question, optional model / max tokens (max 1000), and watch tokens stream into the response panel. Live thread and worker stats are at http://localhost:1337/stats.
Health checks:
curl http://localhost:1337/coordinator/health curl http://localhost:1337/coordinator/health/workers Worker ports are not published; the workers list should show worker-1 and worker-2 alive after heartbeats. With start.sh (single local worker), curl http://localhost:3001/worker/health still works.
Streaming inference (curl):
curl -N -X POST http://localhost:1337/coordinator/infer \ -H "Content-Type: application/json" \ -d '{"conversation_id":"550e8400-e29b-41d4-a716-446655440000","prompt":"What is the capital of France?","model":"tinyllama-1.1b","max_tokens":1000}' Python test script:
python test_inference.py "What is the capital of France?" 1000 Shell test script:
./test_inference.sh "What is the capital of France?" 1000 Benchmark & stress # Requires the coordinator and worker already running, plus:
pip install -r scripts/requirements.txt Fixed-concurrency benchmark:
python scripts/bench.py --mode bench --concurrency 8 --requests 40 --max-tokens 50 Concurrency ramp stress (stops when reject rate hits the threshold):
python scripts/bench.py --mode stress --max-concurrency 64 --step 8 --requests-per-step 16 Stress runs use a unique conversation_id per request; the coordinator may retain them until idle expiry (~5 minutes). Back-to-back ramps can accumulate capacity pressure—wait for idle TTL or restart the coordinator between independent runs.
Optional JSON output and gates:
python scripts/bench.py --mode bench --concurrency 4 --requests 20 \ --out results.json --fail-on error_rate=0.1,p95_ttft_ms=10000 5. Environment variables (optional) # Variable Where Description MODEL_PATH Worker / ensure Path to GGUF model file. Use forward slashes in Git Bash. start.sh defaults to $ROOT/modelFiles/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf; Docker default: /models/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf MODEL_URL Worker / ensure Override GGUF download URL (default: TheBloke TinyLlama Q4_K_M) SKIP_MODEL_DOWNLOAD Worker / ensure Set to 1 to disable auto-download LIBCLANG_PATH Worker build (Windows) LLVM bin directory (for libclang), e.g. C:\Program Files\LLVM\bin. NM_PATH Worker build (Windows) Full path to llvm-nm.exe. Can be derived from LIBCLANG_PATH (see start.sh). OBJCOPY_PATH Worker build (Windows) Full path to llvm-objcopy.exe, e.g. C:\Program Files\LLVM\bin\llvm-objcopy.exe. PORT Coordinator Coordinator port (default 1337). HOST Coordinator Coordinator host (default 0.0.0.0). WORKER_ID, WORKER_URL, COORDINATOR_URL Worker Override worker identity and URLs if running multiple workers or custom topology. Architecture # System Type: Distributed coordinator-worker architecture with stateless scheduling.
Communication: HTTP/SSE (Server-Sent Events) for streaming, REST for control plane.
Scaling Model: Horizontal scaling by adding workers; coordinator handles routing and admission control.
The system consists of three components, each with a single responsibility:
┌─────────────────────────────────────────────────────────────────┐ │ CLIENT │ └─────────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ COORDINATOR │ │ • Admission control (O(1)) │ │ • Session tracking │ │ • Backpressure + streaming │ └─────────────────────────────────────────────────────────────────┘ │ ┌───────────────┼───────────────┐ ▼ ▼ ▼ ┌───────────────────┐ ┌───────────────────┐ ┌───────────────────┐ │ WORKER 1 │ │ WORKER 2 │ │ WORKER N │ │ • KV Cache │ │ • KV Cache │ │ • KV Cache │ │ • Model Weights │ │ • Model Weights │ │ • Model Weights │ │ • Decode Loop │ │ • Decode Loop │ │ • Decode Loop │ └───────────────────┘ └───────────────────┘ └───────────────────┘ Components # Coordinator (TypeScript/Node.js)
Entry point for all client requests Streams tokens from worker to client Applies backpressure — buffers fill, clients get dropped, not workers Tracks sessions for real-time capacity awareness Never touches model weights or KV cache Scheduler (Pure function)
Selects which worker handles each request Scores workers by session count (60%) and KV cache usage (40%) Rejects early if system is at capacity (O(1) check) Worker (Rust)
Designed to own the model — weights, tokenizer, KV cache (LLM integration pending) Prefill: Tokenize prompt, build initial KV cache (infrastructure ready) Decode: Autoregressive token generation (infrastructure ready) Enforces local limits — max sessions, max KV per session No client awareness — just produces tokens into a bounded channel API Reference # POST /coordinator/infer # Start an inference request. Returns streaming tokens via Server-Sent Events.
Request:
{ "conversation_id": "uuid", "prompt": "string", "model": "string", "max_tokens": number } conversation_id is required (client-generated UUID). Reuse it across turns for multi-turn chat; concurrent requests with the same id are queued.
Response: text/event-stream
Each SSE event:
{ "token": "string", "finished": boolean } Status Codes:
200 - Success (streaming) 400 - Missing required fields 409 - Conversation reset required (reason: session_full or session_gone) 502 - Worker unreachable or failed 503 - System at capacity See protocol/inference.http.md for complete API documentation.
Configuration # Coordinator # Environment variables (optional):
PORT - Server port (default: 1337) HOST - Server host (default: 0.0.0.0) Worker # Environment variables:
MODEL_PATH - Path to GGUF model file. Use forward slashes (e.g. E:/path/to/model.gguf) when setting in Git Bash. start.sh defaults to $ROOT/modelFiles/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf; Docker/Compose default: /models/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf MODEL_URL - Override GGUF download URL when auto-download runs SKIP_MODEL_DOWNLOAD - Set to 1 to disable auto-download (fail if file missing) Supported models: The worker uses the llama_cpp Rust crate (v0.3), which bundles llama.cpp. Default is TinyLlama 1.1B (TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF); use any quant e.g. Q4_K_M.gguf. Other supported architectures include Llama, Gemma 2, Phi, Mistral, etc. Gemma 3 is not yet supported by the bundled llama.cpp.
WORKER_ID - Unique identifier (default: worker-1) WORKER_URL - Reachable URL for coordinator (default: http://localhost:3001) COORDINATOR_URL - Coordinator base URL (default: http://localhost:1337) System Limits # Per Worker:
100 max sessions 512 MB max KV per session 8 GB total KV cache System-wide:
1000 total sessions 64 GB total KV cache Project Structure # inference-engine/ ├── coordinator/ # TypeScript/Node.js coordinator service │ ├── src/ │ │ ├── server.ts # Express server setup │ │ ├── infer.ts # Inference request handling │ │ ├── scheduler.ts # Worker selection logic │ │ ├── health.ts # Health check endpoints │ │ └── ... │ └── package.json │ ├── worker/ # Rust worker service │ ├── src/ │ │ ├── main.rs # Entry point │ │ ├── model.rs # Model loading & inference │ │ ├── cache.rs # KV cache management │ │ ├── stream.rs # Token streaming │ │ └── ... │ └── Cargo.toml │ ├── docs/ # Detailed documentation │ ├── ARCHITECTURE.md # System design deep dive │ ├── COORDINATOR.md # Coordinator implementation │ ├── WORKER.md # Worker implementation │ ├── FAILURE_MODES.md # Failure handling strategies │ └── ... │ ├── protocol/ # API specifications │ └── inference.http.md │ ├── start.sh # Quick start script └── README.md Key Features # Distributed Architecture: Framework for scaling inference across multiple workers Memory-Aware Admission Control: O(1) capacity checks prevent overload Backpressure: Slow clients are dropped, not workers Failure Resilience: Automatic retries for prefill failures Session Management: KV cache lifecycle infrastructure with TTL-based cleanup Real-time Health Tracking: Heartbeat-based worker monitoring Streaming Infrastructure: Server-Sent Events with bounded channels for backpressure Keywords: distributed inference, LLM serving, KV cache management, backpressure, admission control, worker scheduling, session management, horizontal scaling, memory-aware load balancing, token streaming, Server-Sent Events, coordinator-worker pattern, failure resilience, health monitoring, heartbeat protocol
Documentation # For detailed information, see:
System Overview - High-level design and problem statement Architecture - Deep dive into system design Coordinator - Coordinator implementation details Worker - Worker implementation details Failure Modes - Failure handling strategies Streaming - Token streaming and backpressure KV Cache - KV cache management Scheduler - Worker selection algorithm Observability - Metrics and monitoring Current Status # Project Phase: Infrastructure complete, LLM integration pending.
This project provides the infrastructure layer for distributed LLM inference:
✅ Implemented:
Coordinator with admission control and session tracking Worker framework with health monitoring and heartbeat Scheduler for worker selection Streaming infrastructure with backpressure Session management and KV cache lifecycle (infrastructure) Failure handling and retry logic 🚧 Pending:
LLM model integration (model loading, tokenization, inference) Actual KV cache implementation tied to a specific model backend Token generation logic Integration Requirements: To complete LLM integration, implement model loading, tokenization, and inference logic in the worker’s model.rs module. The infrastructure for session management, streaming, and KV cache lifecycle is ready.
Troubleshooting # Worker fails to start # Check ports: Ensure port 3001 is not in use Verify Rust installation: rustc --version should show 1.70+ Check build errors: Review cargo build output for dependency issues Coordinator returns 503 “System at capacity” # Check worker health: curl http://localhost:3001/worker/health Verify worker registration: curl http://localhost:1337/coordinator/health/workers Check system limits: Review session and KV cache limits Ensure worker is running: Worker must be running and sending heartbeats Coordinator can’t reach worker # Verify worker URL: Check WORKER_URL environment variable matches actual worker address Check network: Ensure coordinator can reach worker on the specified port Check heartbeat: Worker should be sending heartbeats every 10 seconds Development # Building # Coordinator:
cd coordinator npm install npm run build Worker:
cd worker cargo build --release Testing # See individual component documentation for testing instructions.
Contributing # Contributions welcome! Please read the architecture documentation before making significant changes.
For AI/LLM Parsing # Project Summary: Distributed inference framework for LLM serving with coordinator-worker architecture, memory-aware admission control, and backpressure handling.
Primary Technologies: TypeScript, Node.js, Rust, Express, Axum, Server-Sent Events.
Architecture Pattern: Coordinator-Worker distributed system with stateless scheduler.
Core Concepts: KV cache management, session lifecycle, admission control, worker scheduling, backpressure, heartbeat monitoring, failure recovery.
Current State: Infrastructure layer complete; LLM model integration pending.
Related Documentation: See docs/ directory for detailed architecture, failure modes, streaming, and component-specific documentation.


---

# Similar Player Finder

> A data analytics project that finds FIFA 22 players similar to a custom profile using one-hot encoding, feature scaling, and cosine similarity — exposed via a FastAPI endpoint.
Published: 2025-01-05
URL: https://hugo-portfolio-teal.vercel.app/projects/similar-player-finder/
Markdown: https://hugo-portfolio-teal.vercel.app/projects/similar-player-finder/index.html.md

Data Analytics Project: Player Similarity Analysis # Table of Contents # Overview Problem Statement Datasets Data Cleaning and Preprocessing Feature Selection Encoding Techniques Similarity Analysis Conclusion Overview # This project analyzes player similarities using data from Kaggle. The goal is to identify comparable players based on selected features using cosine similarity.
Problem Statement # The idea was to get user build a player for them, and get similar players with regards to attributes filled by the user. Which can then be used for building more use cases on top of it.
Datasets # Source: Kaggle Selection Process: Multiple datasets were analyzed based on completeness, relevance, and alignment with the problem statement. Final Dataset: [FIFA 22 Dataset] I first fbref website and some other places like trnaferMarkt to scrape the data for the project, but then realised maybe it shouldnt be so in the first place, as what i wanted was to have a simulation game like experience, and this would have made it really complex. So went with really basic more enjoyable by general public, player dataset from FIFA 22. Data Cleaning and Preprocessing # Removed missing and inconsistent entries. Normalized and standardized numerical values for consistency. Handled outliers and imbalanced data where necessary. Feature Selection # Selected 7 features critical for player comparison, such as: Position Speed Passing Dribbling Defense Physic Shooting Rationale: These features were chosen for their direct influence on the objective of the analysis (e.g., performance metrics, position-based attributes) and are easy for a generic user to fill up and make a deicion upon. Encoding Techniques # Why Encoding? To convert categorical data into numerical format for similarity calculations. Why One-Hot Encoding? Preserves category uniqueness. Avoids ordinal assumptions inherent in label encoding. Similarity Analysis # Approach: Cosine Similarity Why Cosine Similarity? Measures similarity based on the orientation rather than magnitude. Effective for high-dimensional sparse data. Ensures that the results are not skewed by the scale of features. Conclusion # Identified players with high similarity scores. The methodology demonstrates the utility of cosine similarity in sports analytics. Insights can support team formation, player recruitment, and performance comparison. we will need to add more features as search does gives the result, but they are not always what we want, we want more filters on top of it to get a much better selection.


---

# Mongo Easy Migration

> A MongoDB plugin that simplifies writing and managing database migrations with a structured API, per-record callbacks, and built-in logging.
Published: 2024-12-04
URL: https://hugo-portfolio-teal.vercel.app/projects/easy-migration/
Markdown: https://hugo-portfolio-teal.vercel.app/projects/easy-migration/index.html.md

Mongo Easy Migration Plugin # Easy Migration is a MongoDB plugin designed to simplify the process of writing and managing migrations in MongoDB databases. It streamlines the development workflow by providing a clear and structured approach to database changes. Features # Intuitive API: Write migrations with ease using simple and well-documented methods. Rollback Support: Provides an easy way to undo migrations (coming soon) Customizable: Supports custom migration logic to suit your application needs. Error Handling: Robust error reporting and logging to help debug issues. Setup # install library from npm using
npm i mongodbplugin Usage # What you need is your Your mongoDB URI Your primary collection on which migration has to take data from Your list of remaining collections, or the collection on which the migration is to be performed on Callback function with logic of what is required to be done const processMigration = require("mongodbplugin"); let mongoDB_URI = process.env.DB_URL let primaryCollection = require("./models/primaryCollection"); let secondaryCollection = require("./models/secondaryCollection"); let ternaryCollection = require("./models/ternaryCollection"); let callbackFn = require("./migrations/updateNewFieldsInDB); // the callbackFn should consists of what logic you want to impliment on per record basis, as this will be called inside a loop. processMigration( { uri:mongoDB_URI, options: { // this includes further options that you want to pair up with mongodb }}, primaryCollection, [ primaryCollection, secondaryCollection, ternaryCollection ], callbackFn( data, primaryCollection, secondaryCollection, ternaryCollection, writeLog ) // spread apart your collections here // you can use writeLog function to write your logs on system which will get saved inside a folder migrationLogs ) /** writeLog(action, logContent); you can directly call this function inside your callback */ Coming Soon # Improved overall Performance. Improved logging capabilities. Rollback support.


---

# Improving UX - Web Vitals Story

> A practical look at Core Web Vitals (LCP, CLS, INP): what they measure, how to measure them, and techniques that improve real product UX.
Published: 2024-08-14
URL: https://hugo-portfolio-teal.vercel.app/blogs/improving-web-vitals/
Markdown: https://hugo-portfolio-teal.vercel.app/blogs/improving-web-vitals/index.html.md

Why should you even bother with UX as an engineer, isn’t that what designers are paid for? # Today the user experience lies at the forefront and is the biggest defining quality for the value of a product. You can’t have a product that is slow to load, with constant layout shifting and unstable elements, as these hurt the user’s overall experience. So to address these problems we have to monitor and optimize for certain metrics during the whole lifecycle of the product. To monitor these metrics we use core web vitals
What are Web Vitals # Web Vitals is an initiative by Google to provide unified guidance for quality signals essential to delivering a great user experience on the web.
Google provides several tools, usable directly from your Chromium browser or other external tools., that can help identify the metrics that matter the most, which are known as core web vitals
The core web vitals include:
LCP (Largest Contentful Paint): It gives us the time taken to render the largest object/image/text block on the page(visible in the viewport), since the time, the user navigated to the page.
CLS (Content Layout Shift): It gives us the value of the largest change in layout that occurs during the entire lifecycle of the page
INP (Interaction to Next Paint): INP asses the overall responsiveness to user interaction( not screen size problem), observing the latency of mouse/keyboard clicks and other user interactions, the interaction with the worst latency defines the value of INP
Let’s discuss about two of these metrics, LCP and CLS one by one in detail
But how do you calculate these web vitals?🧐 # If it is a web app you can directly use chrome’s lighthouse tool, to calculate values of web vitals, or else you can use this javascript library for ‘web-vitals’, for apps like which are being run inside an iframe or any other kind of sandboxed environment inside a browser.
Largest Contentful Paint (LCP)? # For this first we need to understand what LCP is, LCP reports the render time of the largest image or text block visible in the viewport, relative to when the user first navigates to the page. A good LCP score is when your LCP is 2.5 seconds or less. Technically LCP score is the 75th percentile of page loads, across all devices.
Elements considered for LCP
<image> (element inside SVG) <img> <video> url() (CSS background images) block-level elements loading text or inline text Optimizing for LCP - # While there’s no single approach, these fundamental techniques can be useful:
Using Lazy Loading: Helps in delaying the downloading of the media not present in the viewport
Utilizing a CDN (Content Delivery Network) for static assets significantly improves LCP (Largest Contentful Paint) by reducing the time it takes to deliver assets. CDNs distribute content across multiple servers, allowing assets to be loaded from a server geographically closer to the user, reducing latency. Webp-based images are a superpower as they are small in size.
Use Hashes in File Names with Cache-Control By using hashed file names with cache control headers, you ensure that assets are only re-downloaded when changes are made. This reduces unnecessary requests and ensures that users always get the most up-to-date resources, improving LCP by minimizing revalidation. Big Companies like shopify use this to make sure the fastest delivery and only the latest version gets delivered to the user Example - You are to serve a style.css file from your server, make sure its name is not style.css but style<some_hash>.css, which changes after each file update, this is done so that every time the file updates, a new hash is created, which makes it a new file name for browser, and it doesn’t use the previously cached on ( though our typical react/next builds, automatically create hashed files for our static files).
Investigate and Optimize Network Requests Analyzing the network terminal helps identify slow or redundant requests. Reducing or optimizing these requests can decrease the time needed for the browser to render the page’s largest content, directly improving LCP.
Remove Unnecessary Frontend Requests Removing or deferring non-essential frontend requests reduces the overall load on the network, allowing critical resources to load faster. This prioritization enhances LCP by ensuring that the main content is rendered promptly.
Optimize Database Queries Streamlining database queries can drastically reduce server response times. Efficient queries ensure that the server can deliver necessary data faster, which contributes to a quicker LCP as the page can render its largest element sooner.
Increase Inline Requests from Frontend Increasing inline critical CSS and JavaScript within the HTML reduces the number of render-blocking resources. With fewer external requests, the browser can render the page faster, improving LCP.
Run Background Tasks for Non-Essential Operations Ensuring that the server focuses solely on delivering the web page while offloading other tasks to background processes prevents delays in page rendering. This prioritization ensures that the LCP is not hindered by non-critical server tasks.
Content Layout Shift(CLS)? # First, we need to understand what CLS is, CLS is a measure of the largest burst of layout shift scores for every unexpected layout shift that occurs during the entire lifecycle of a page.
A layout shift occurs any time a visible element changes its position from one rendered frame to the next.
For a good user experience, sites should strive to have a CLS score of 0.1 or less. To ensure you’re hitting this target for most of your users, a good threshold to measure is the 75th percentile of page loads, segmented across mobile and desktop devices.
Simply put, CLS occurs when divs jump unexpectedly, To calculate the layout shift score, the browser looks at the viewport size and the movement of unstable elements in the viewport between two rendered frames. The layout shift score is a product of two measures of that movement: the impact fraction and the distance fraction.
What causes these movements in div👁️👁️:
Dynamic pages take some time to load the content, and the div in which the content is loaded also does not have a fixed size. You can use skeletons, but even they will have a layout shift if they are not of the exact size as that of the resultant div. Screen size changes also affect this variedly
Optimizing for CLS # There are no steps or paths to follow for CLS at least in my experience till now, or some practices, what you need to do is iteratively work on decreasing the delta in change of layout
First principle Techniques you can use
Using a skeleton, to be closest to the resultant layout, your design should also allow, the first page of the app, to have a fixed layout at least for the viewport, so that you can give a much better skeleton for it.
Using Loaders, these should be used rather carefully, as they will result in a definite layout shift, it’s just that they indicate the user of the loading state. a smart way to use loader is to use it on secondary pages/states, like when clicking on a button requires a redirection to another page/state which requires an API call to render information, rather than using a skeleton on a secondary page/state, make a loader on the button you pressed, give the loading state to current page, make the API call get the data, and when the page is ready only then redirect user, and since the final page will be loaded directly there will be no layout shift.
Make sure there is no element on the page, which is unstable and moves with the viewport
The size(aspect ratio) of image and other media should be constant, they should not change with time, or when it gets completely loaded, affects the layout shift adversely.
Additional Tips - # Optimization for web vitals is an iterative process, and heavily context-dependent, you can also take the help of softwares like mixpanel, etc, to know which page is visited most often and can optimize your APIs and design systems while keeping that in mind
Do not do regular deployments of frontend applications, as you are usually serving your application pages from CDN, but they need to be invalidated due to regular deployments. A new asset needs to be served, which will require your server to deliver the page, this interference from the server can increase LCP. Reasoning: Even if your files are hashed, and you do very frequent deployments the first time the static files are called on the browser they will take more time then cached ones, and all of users will be required to download the latest files, maybe with just more planned product lifecycle you can avoid these very frequent deployments and save some time there.
Try testing your pages on slow network speed (you can do this directly on chrome from network tools), will help you discover key breakages in your app, which usually get missed since we are working with good internet
Make sure your artifacts are the smallest possible in size, as it always helps to have a lighter page
Conclusion # Maintaining these metrics within specific thresholds is essential, to make sure your application’s SEO score is good and it ranks high and also from the perspective of general user experience
If you are an app developer, who develops apps for platforms like, app store, play store, shopify app store, then this is something you must be careful about, as they take these metrics under consideration not only while reviewing your app for publishing but also for giving it badges and featuring it on their platforms.
There is a lot more to discuss regarding the two of these, but that will need a separate blog of its own, so till then stay tuned 🤗


---

# About Me

> Software engineer, Sports Enthusiast.
URL: https://hugo-portfolio-teal.vercel.app/about/
Markdown: https://hugo-portfolio-teal.vercel.app/about/index.html.md

Hi! I’m Ishan. When I was born, my parents thought what can be the most powerful name that we can have for our child, they tried to get intersection of 4 hindu dieties -> The Sun, Lord Shiva, Goddess Parvati & Lord Vishnu and thought, great, “Ishan” can mean any of these at any time, this will be best for our child
Born and raised in Kota, Rajasthan, I completed my schooling at Bakshi’s Spring Dales School (2019) and graduated with a degree in Computer Science from RTU (2023).
I enjoy taking ambiguous, messy problems and turning them into simple, reliable systems. My work usually sits at the intersection of product intent, system design, and real-world constraints. I prefer boring tech at scale, obsess over system boundaries, and optimize for correctness before cleverness.
I am a die-hard fan of FC Barcelona and the Indian Cricket Team. Whether consciously or not, I’ve always tried to model myself after Lionel Messi, Sachin Tendulkar, and Rahul Dravid—striving to be supremely calm under immense pressure, fiercely consistent, and the best at what I do.
Beyond tech, I’ve always loved testing my limits in different arenas. I used to write original plays and dramas, and even represented my district in debate competitions and the state of Rajasthan in a National-Level quiz. That competitive drive eventually spilled over into entrepreneurship, where my startup was selected as one of just 12 across India for T-Hub’s exclusive 1:1 mentorship program. When I’m not building things, I’m usually reading fiction, learning random trivia, or meeting new people.
This is my personal space where I share my thoughts, projects, and experiences.
Career # Skailama # Software Engineer → Engineering Lead | June 2023 – April 2026
Build products end-to-end across high-performance web apps, backend systems, AI automation, and SaaS Improved Core Web Vitals by 70% Optimized critical systems with Rust Automated internal operations using AI Modernized production platforms and helped take products from zero to profitability Focus on problems where product, performance, and business outcomes intersect Grayswipe # Co-Founder | October 2021 – August 2024
Started as a Blog Platform and then pivoted to a Consumer App to book Salons for weddings and other events Handled its technical end as well as the Social Media handles to post memes 🤣 And Finally pivoted to a SaaS Built and launched a B2B SaaS marketplace for India’s textile industry Owned product architecture and full-stack development Scaled to 50+ manufacturers representing over ₹50 crore in annual turnover Led a team of engineers across technical strategy, delivery, and product execution Selected among the top 12 startups (out of 375 nationwide) for the T-Hub RubriX Cohort Early Experience & Internships # Software Engineer / Freelance
Built and shipped a promotional game for Flipkart Big Billion Days, integrating directly with Flipkart systems for a large-scale consumer event Developed blockchain analytics software on the Polygon ecosystem, contributing to the company securing a Polygon fellowship Implemented distributed authentication using Shamir’s Secret Sharing for secure trust distribution and resilient key management Other Experiences # Goal.com & Red Rants # Sports Writer | 2019 – 2020 (pre-COVID)
Wrote for Goal.com and RedRants.com covering football transfers, tactical breakdowns, and pre- and post-match reports