Skip to content
View Aravind0403's full-sized avatar
3 followers · 5 following

Achievements

Achievement: YOLOAchievement: QuickdrawAchievement: Pull Shark

Achievements

Achievement: YOLOAchievement: QuickdrawAchievement: Pull Shark

Block or report Aravind0403

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
Aravind0403/README.md

Aravind Sundaresan

AI Systems Engineer · Heterogeneous LLM Inference & Engine Internals · Distributed GPU Systems

LinkedIn arXiv Lab & Research Upstream Dossiers OKC Hopper H200 MLCommons Email

⚡ About Me

AI Systems Engineer specialising in heterogeneous LLM inference: hardware-aware scheduling, disaggregated serving, GPU kernel engineering, and engine internals across vLLM, SGLang, and MLCommons.

  • 7+ years of distributed-systems infrastructure at Microsoft and Amazon combined with 3 years of self-funded inference systems research.
  • Upstream contributor to core engines (vLLM, SGLang) and official benchmarking consortia (MLCommons / MLPerf): KV connectors, asynchronous schedulers, prefix-caching hot paths, and Marlin / Quark quantization kernels.
  • Competitor in the Open Kernel Challenge (Meta × Paradigm × Andromeda): Ranked #1309 Global (Top Tier) with 4/4 verified solutions optimizing the 816B Muse Spark MoE on NVIDIA Hopper H200 (up to 2.33× GEMV speedup).
  • Author of HeteroDisagg (hardware-adaptive asymmetric disaggregated serving & Community RFC) and Clairvoyant (ML-driven SJF inference scheduling, arXiv:2606.07248).
  • Founder of Anavind Labs — an independent systems practice publishing open benchmarks, architectural specs, and upstream engine hardening.

🚀 Upstream Open-Source Contributions & RFCs

Core engine internals contributed to vLLM, SGLang, and MLCommons (detailed technical dossiers published at anavindlabs.com/upstream.html):

Ecosystem Pull Request / Spec Subsystem & Component Architectural Impact & Root Cause Solved Status
vLLM PR #65584 KV Connector & P/D Disagg Resolved sliding-window KV cache block drops in P/D serving (#65550) via prompt token backoff across NIXL and Mooncake, eliminating silent null_block reads. In Review
vLLM PR #65159 V1 Async Scheduler Fixed async scheduling placeholder underflow crash on sync KV load failure (#65131); properly drains in-flight stale output tokens and resets placeholders. In Review
SGLang PR #39623 P/D Disaggregation Eliminated unbounded memory leaks and worker OOMs under network jitter via admission guards, thread-safe RLock purge, and an $O(1)$ min-heap TTL sweeper across Mooncake, Mori, and Nixl. In Review
SGLang PR #39741 Quantization & Marlin Prevented Marlin GEMM tile-divisibility CUDA aborts ($N \pmod{64} == 0$) on Qwen3.5 GDN hybrid models via pre-flight shape validation and dynamic unquantized fallback. In Review
vLLM PR #51574
(Co-Authored) 		Core V1 Scheduler Eliminated P99 tail latency inflation from preempted-request starvation in PriorityRequestQueue (#41951, #56843) via bounded starvation caps and anti-thrashing victim selection. In Review
vLLM PR #56964 MoE & Speculative Decoding Resolved runtime termination when target models are Marlin-quantized but speculative draft layers are unquantized via dynamic kernel execution fallback. In Review
vLLM PR #65536 Quantization (Quark) Fixed AMD Quark quantization exclusion parsing for module targets with parameter suffixes (weight, bias), preventing invalid kernel binding. In Review
vLLM PR #64967 Attention Kernels Enforced power-of-two head size validation in TurboQuant fused attention kernels to prevent invalid memory indexing. In Review
SGLang PR #37067 Prefix Cache / RadixTree Optimized RadixCache.total_size() from $O(N)$ recursive tree traversal to $O(1)$ via incremental mutation tracking, eliminating GIL and DFS polling overhead. Closed (Adopted)
vLLM / SGLang Community RFC HeteroDisagg Architecture Accelerator Capability Descriptor (ACD) & Hardware-Adaptive Serving Policies: Roofline-driven dynamic chunking and bipartite GQA re-sharding across mismatched TP ranks. Proposed
MLCommons Contributor MLPerf Inference & Endpoints Benchmarking harness integration across reference LLM SUTs (vLLM / SGLang backends) and high-QPS serving endpoint evaluation. Active

🛠️ Featured Systems & Research Projects

GPU Kernel Engineering & High-Performance Inference

Project Description Key Performance & Benchmarks
Open Kernel Challenge (OKC)
Hopper H200 GPU Kernel Engineering 		Frontier LLM inference optimization on NVIDIA Hopper H200 for Muse Spark Open (816B Parameter Sparse MoE). Sponsored by Meta × Paradigm × Andromeda. • Rank #1309 Global (Top Tier): 4/4 verified solutions on official H200 silicon.
• 2.329× Router GEMV (4.4 µs vs 10.3 µs) & 1.777× Unpooler GEMV via 128-bit LDG.E.128 loads and register warp shuffles.
• 1.518× MoE W4A16 Decode (50.8 µs vs 77.1 µs) via tuned Marlin tiling.
• Serving Track: Multi-GPU 8×H200 SGLang server with DFlash Speculative Decoding.
HeteroDisagg
Disaggregated LLM Serving 		Research prototype for hardware-adaptive disaggregated LLM serving across mixed-silicon clusters (A100 + RTX 3090). Features asymmetric TP and wire quantization. • Asymmetric TP: HeteroKVConnector for GQA bipartite P2P re-sharding across mismatched prefill/decode ranks (prefill_tp != decode_tp).
• In-Flight FP8: Vectorised FP8 E4M3 quantization halved wire payload (128 MB → 64 MB) at 99.98% cosine fidelity and $\Delta\text{PPL} \le 0.0018$.
• Accelerator Capability Descriptor (ACD): Roofline planner deriving chunk sizing and asymmetric TP across silicon.
Clairvoyant Scheduler
📄 arXiv:2606.07248 		Zero-overhead Go sidecar proxy eliminating Head-of-Line blocking in LLM inference via ML-driven Shortest Job First (SJF) scheduling. • 70–76% P50 latency reduction under burst load.
• 83.6% latency reduction under realistic trace replay.
• 0.029 ms inline ONNX inference overhead with zero backend modifications.
AetherControl
Full LLM Lifecycle Control Plane 		4-stage platform: dataset pre-flight guard (trainsight) → GRPO post-training (rlhf-pipeline) → vLLM serving (vllm-engine) → K8s governance. • 1,141 tok/s throughput · TTFT P50 22.1ms.
• 100% OOM prevention via pre-flight profiler.
• GSM8K: 42% → 70% via GRPO · −98.6% DCGM mutex contention.

Cluster Orchestration & Infrastructure

Project Description Key Performance & Benchmarks
Kube-Sentinel
Zero-Trust K8s GPU Scheduler 		Native Go Scheduling Framework plugin (Score/Reserve/Bind) + Python gRPC telemetry daemon cross-checking node telemetry against scheduler cache truth. • 46.3% GPU cost reduction ($120/hr → $64.44/hr) on 32-node Alibaba cluster trace.
• 1,252 pods/sec peak throughput with sub-millisecond P99 latency.
• 100% failover availability under daemon SIGKILL.
ACO Orchestrator Predictive job scheduler for heterogeneous compute using Ant Colony Optimization + LSTM spike prediction. • <10ms latency · 95%+ SLA adherence across 202 test scenarios.
ServiceScope v2 AI-native blast-radius analysis for Python microservices: AST parsing + local LLM inference. • 190 files/sec · 0% inference failure · Zero external API dependencies.

🔬 Research & Preprints

  • Clairvoyant: ML-Driven Shortest Job First Scheduling for LLM Inference
    Aravind Sundaresan — arXiv:2606.07248
    Target: NeurIPS 2026 Workshop → MLSys 2027

💻 Technical Competencies

  • Inference Engines & Serving: vLLM internals, SGLang internals, PagedAttention, RadixAttention, Continuous Batching, Chunked Prefill, Disaggregated Prefill/Decode, Asymmetric TP, GQA re-sharding, KV-cache lifecycle & wire quantization.
  • GPU Kernel Engineering & Acceleration: NVIDIA Hopper H200, 128-bit LDG.E.128 vectorization, register warp shuffles (__shfl_down_sync), Marlin GEMM kernels, AMD Quark, GPTQ, AWQ, FP8 (E4M3), MoE kernel dispatch, Triton kernels, CUDA C++, CUDA IPC, NCCL, Roofline analysis.
  • Languages & Frameworks: Python, Go, C++, CUDA, Java, PyTorch, Triton, ONNX Runtime.
  • Orchestration & Distributed: Kubernetes (custom Go scheduler plugins), Docker, DCGM, Prometheus, Grafana, AWS, Azure, GCP, InfiniBand/RoCE, NVLink.

📬 Connect

Pinned Loading

  1. aether-control-llm-infra aether-control-llm-infra Public

    Enterprise LLM Serving Platform, Control Plane & Post-Training Pipeline (vLLM, GRPO, K8s).

    Python

  2. Kube-Sentinel Kube-Sentinel Public

    Trust-weighted, cost-optimal Kubernetes scheduler plugin with zero-trust telemetry verification.

    Python

  3. clairvoyant-scheduler clairvoyant-scheduler Public

    Go sidecar proxy that eliminates Head-of-Line Blocking in LLM inference via ML-driven SJF scheduling — zero backend modification. Paper in preparation

    Python 1

  4. ServiceScope-v2 ServiceScope-v2 Public

    AI-native blast-radius analysis for Python microservices — AST parsing + local LLM inference + dependency graph. No service mesh needed

    Python

  5. ACO_Adaptive_Compute_Orchestrator ACO_Adaptive_Compute_Orchestrator Public

    Predictive job scheduler for heterogeneous compute — ACO + LSTM spike prediction + intent-aware routing. <10ms latency, 95%+ SLA adherence, 202 tests

    Python