shashwat.dev — portfolio '26
0
Data Engineer · PhD Scholar — Explainable AI

Building pipelines people can trust.

I build large-scale data platforms by day — streaming systems, lakehouses, enterprise RAG — and research explainable AI by night, making ML systems transparent enough to bet real decisions on. Scroll — the particles will follow.

Scroll · click the void
PySpark ·Kafka ·Databricks · Airflow ·Snowflake ·dbt · AWS ·Azure ·Delta Lake · LangChain ·SHAP ·Terraform ·
IMPACT

Numbers that ship.

0
Years building data platforms across fintech & banking
0
Records flowing daily through pipelines I engineered
0
Uptime on real-time streaming monitoring I built
0
Production pipelines shipped end-to-end
WORK

Things I've built.

↗
01

Enterprise RAG for Financial Docs

A retrieval-augmented pipeline for financial document summarization — LangChain + vector databases wired into governed banking data flows, cutting manual review effort by 40%.

LangChainVector DBsLLMOps
↗
02

10x Lakehouse Migration

Moved RDBMS workloads to an Azure Databricks lakehouse — PySpark pipelines over 5M+ daily records running 10x faster, plus a 100GB+ Synapse warehouse consolidating 10+ vendor feeds.

DatabricksPySparkAzure Synapse
↗
03

ExplainOps — Open Source

An ops layer for ML explanations: lineage-linked attribution tracing, drift monitoring, and trust checks. Part of an XAI × Data Engineering series I'm shipping repo by repo on GitHub.

XAIOpen SourcePython
↗
04

Trustworthy AI Research

Survey of Trustworthy & Explainable AI accepted at IEEE WorldSUAS 2026 — alongside PhD research on making LLM systems transparent, bias-aware, and auditable in enterprise data platforms.

IEEE 2026PhDXAI
OPEN SOURCE

Shipping in public.

One repo at a time, an XAI × Data Engineering stack — each with tests, CI and a runnable demo. Every claim below is something you can clone and check.

WRITING

Thinking out loud.

Research, long-form and a Hinglish SQL series — the same ideas, pitched at whoever needs them.

JOURNEY

Where I've shipped.

Jul 2024 — Present · Pune

Data Engineer · IBM

Client: a large banking client

Optimized AWS Glue ETL to cut compute cost by 25%. Refactored PySpark jobs 25% faster, saving 200+ compute-hours a year. Built Kafka + PySpark streaming monitoring holding 99.9% uptime, and an enterprise RAG pipeline (LangChain + vector DBs) that cut manual review of financial documents by 40%.

Feb 2021 — Jul 2024

Software Engineer, Data Engineering · Pine Labs

Migrated RDBMS workloads to Azure Data Lake and Databricks — PySpark pipelines over 5M+ daily records running 10x faster. Built a 100GB+ airline data warehouse on Azure Synapse consolidating 10+ vendor feeds, real-time Spark Streaming + Kafka pipelines, and 15+ production pipelines end-to-end.

2020 — 2021

Data Science Intern · NuvoRetail

Billing optimization and sales forecasting models hitting 90% accuracy — my first taste of data products that people actually bet decisions on.

Ongoing — Expected 2029

PhD Scholar, Trustworthy & Explainable AI · VIT

Researching how to make LLM-era systems transparent, bias-aware and ethically compliant inside enterprise data platforms. First survey paper accepted at IEEE WorldSUAS 2026.

AWS Certified Data Engineer — Associate PCAP — Certified Python Programmer MCA · VIT
CONTENT

Where I talk data.

↗
NEWSLETTER

Explainable Pipelines

Weekly deep-dives where data engineering meets explainable AI — architecture patterns, trust checks, and the code behind them.

WeeklyLinkedInDE × XAI
↗
YOUTUBE

Bits to Insights

"SQL ratte nahi — samajhte hain." A Hinglish SQL series: concept → practical → interview questions, built for the next generation of data engineers.

SQLHinglishSeries
↗
INSTAGRAM

pipeline.diaries

The diary of a data pipeline — "AI gets the credit. The pipeline keeps the receipts." One brutal truth from desi tech life, daily.

HumorReelsDaily
↗
MEDIUM

Long-form Notes

Newsletter editions and engineering write-ups, republished for readers who live outside LinkedIn.

EssaysXAIData Eng
↗
KAGGLE

Notebooks

Reproducible XAI experiments — one question, one experiment, one honest finding each. Negative results included.

NotebooksXAIPython
↗
X

Building in Public

Threads on what I'm shipping — open-source XAI tools, pipeline war stories, and research notes as they happen.

@I_m_ShashThreadsOSS

Explainable Pipelines

One idea a week on building data platforms that can explain themselves — pipelines, lineage, attribution, and the LLM era's trust problem. No spam, just engineering.

Subscribe on LinkedIn
Edition #1 is live — "Your AI Can't Explain Itself. Your Pipeline Is Why."

Let's build something
worth trusting.

Open to collaborations in data engineering, explainable AI research, and technical content. The particles just spelled my name — now you know who to email.