In 2017, a team of five data engineers worked in Melbourne. They were passionate and used classic ETL. Their tools included PostgreSQL, Airflow, and Python.

They extracted data nightly from Shopify APIs and more. Then, they transformed it using Python scripts. The clean data went into a PostgreSQL warehouse.

It worked well for a while. Clients loved the dashboards. The team felt confident. The CTO, Harsha, said, “We own the entire flow end-to-end. No surprises. No black boxes.”

But, they didn't know their system would soon be a bottleneck.

The Problem Wasn’t Obvious… At First

As more customers joined, data volumes grew. What was once 100MB a night became 15GB, then 50GB. Scripts started failing, and Airflow DAGs took longer.

But the worst part was yet to come.

The team spent more time fixing issues than creating new things. Marketing and product teams had new requests. The founders wanted predictive dashboards.

“Just add one more table,” they'd say. But adding a table meant rewriting scripts and testing. It was like renovating a house while people lived there. Every request felt like a risk.

Harsha worked late every Friday, optimising queries. He knew they were stuck due to the architecture's brittleness.

Then Came the Breaking Point

One afternoon, a big client signed up. They needed hourly data syncs and detailed analysis. The team panicked.

They tried to scale their ETL system. They added more compute and parallelized jobs. But it only helped temporarily.

During a critical demo, the DAGs failed. The CTO's face turned pale. The client asked, “Is this data even real?”

That night, Harsha realised they needed a complete rethink. Not just patches.

Discovering a New World: ELT

That weekend, Harsha found a blog post about Snowflake. He also read about dbt and companies moving to ELT.

He wrote in his notebook:
What if we stopped transforming data before loading it?
What if we just dumped everything into the warehouse first?
What if the warehouse became the engine?

The next morning, he called an emergency team meeting.

“We’re doing this backwards,” he said. “We should load raw data into the warehouse first, then transform it with SQL.”

The team was skeptical. “That sounds lazy. What about data quality? What about governance?” they asked.

“I thought the same,” Harsha admitted. “But this isn’t laziness — it’s leverage. We’ve been wasting our energy transforming data on weak compute. It’s time we let the warehouse do what it’s built to do.”

And so, they started shifting to ELT.

Rebuilding From Scratch

They moved quickly. They picked BigQuery as their new cloud data warehouse. It was fast and easy to use, with no setup needed.

They changed how they handled data. Now, data goes straight into BigQuery tables. This made things faster. No need to clean or rename data. They just used it as is, with all its details.

Then, they discovered dbt.

It was a game-changer. They used SQL models instead of complex code. This made their work easier to manage and understand.

Each model explained itself. They could see how everything connected. They organized data in three steps: raw, staging, and mart.

They could track data back to its source easily. This made everyone happy, including Harsha.

They could now answer data requests quickly. They were no longer worried about changing data or adding new sources.

The Unexpected Benefits

They started sleeping better.

But there were more surprises.

They could now go back in time to fix data issues. This was great for solving problems or adding new features.

Their clients were happy too. Dashboards were updated faster and were more accurate. The team could handle complex requests without stress.

The team was excited again. They were making progress, not just fixing things. This made everyone more productive.

Reflections: Why ELT Won

Looking back, Harsha explained why they chose ELT:

"ETL is like pre-cooking all the ingredients before you even know what dish you're making — time-consuming, rigid, and hard to adapt. ELT is like bringing fresh ingredients into the kitchen and deciding how to cook them based on what your guests want. It's agile, responsive, and works with real-world complexity."

ETL was good when the data was small. But now, with cloud warehouses, ELT is better. It's like using a laptop instead of handwriting.

ELT changed DataNova's culture. They went from fixing problems to creating solutions. This led to big improvements.

Closing Thoughts

DataNova's story is not unique. Many data teams face similar challenges. The key lesson is:

If your transformation logic slows you down, it's time for ELT. It's not just new; it's better.