Archive

1046 posts · Page 71 of 88. ← Blog

Pratik Dhanave · ·8 min read

Data Modeling for Analytics

The same data can be organized in ways that make analytical queries fast, intuitive, and cheap — or slow, confusing, and expensive. Data modeling is the craft of that organization, and it's where a counterintuitive truth lives: the careful normalization that's correct for application databases is often exactly wrong for analytics. Analytical data wants to be shaped differently, around how questions are asked rather than how data is written, and understanding dimensional modeling is understanding how to make a warehouse actually usable.

The same data, organized differently, makes analytical queries fast and intuitive — or slow and confusing. Data modeling is the craft of that organization, and it holds a counterintuitive truth: the normalization that's correct for application databases is often exactly wrong for analytics.

Pratik Dhanave · ·7 min read

Where Data Lives: Warehouses, Lakes, and Lakehouses

"Just put it in a database" stops working the moment you're dealing with analytics at scale — because the database that runs your application is optimized for exactly the wrong thing. Analytical data needs different storage: systems built to scan and aggregate huge volumes, not to serve fast individual transactions. The landscape of analytical storage — data warehouses, data lakes, and the newer lakehouses — is where data engineering decisions about where data lives get made, and understanding the differences (starting with OLTP vs OLAP) is essential.

'Just put it in a database' stops working the moment you're doing analytics at scale — the database that runs your app is optimized for exactly the wrong thing. Analytical data needs different storage: warehouses, lakes, and lakehouses. Understanding the differences (starting with OLTP vs OLAP) is essential.

Pratik Dhanave · ·7 min read

Data Pipelines and ETL/ELT

The core artifact of data engineering is the pipeline: an automated flow that pulls data from somewhere, reshapes it, and lands it somewhere useful. And the single most consequential shift in modern data engineering is captured in three reordered letters — ETL became ELT — a change driven by cheap, powerful cloud data warehouses that flipped when and where transformation happens. Understanding pipelines, and the ETL-to-ELT shift, is understanding how data actually moves and gets made usable.

The core artifact of data engineering is the pipeline: an automated flow that pulls data from somewhere, reshapes it, and lands it somewhere useful. And the single most consequential modern shift is captured in three reordered letters — ETL became ELT — driven by cheap, powerful cloud data warehouses that flipped when and where transformation happens.

Pratik Dhanave · ·6 min read

What Data Engineering Is

Every dashboard, every analytics query, every machine-learning model, and every "data-driven decision" rests on an invisible foundation: someone built the pipelines that collect, move, clean, and organize the data so it's actually usable. That someone is a data engineer, and their work is the unglamorous, essential plumbing beneath everything data. When it works, no one notices; when it breaks, every downstream report and model breaks with it. Understanding data engineering is understanding how raw data becomes something a business can actually use.

Every dashboard, analytics query, and ML model rests on an invisible foundation: someone built the pipelines that collect, move, clean, and organize the data. That someone is a data engineer, and their work is the essential plumbing beneath everything data. When it works, no one notices; when it breaks, every downstream report and model breaks with it.