A I & C h i l l

A dev journal featuring me and my ever-loyal AI minions.
Exploring the limits of agent driven development aka vibe coding

Custom Azure Functions ETL vs Azure Data Factory: Which One Should You Use?

Published 07 Jun 25 by vasileioskk5 min read

The Case for Azure Functions: Code, Control, and Chaos (but in a good way)

When I go the Azure Functions route, it’s usually because I want:

  • Complete control over the transformation logic

  • To work in my comfort zone (C#, LINQ, async/await — yes please)

  • A more DevOps-friendly setup with versioned, testable code

  • Lower overhead for lightweight, highly customized pipelines

For example, in one of my projects, I had CSVs dropped into a Blob container that needed:

  • Parsing based on custom rules

  • Field mappings that changed depending on the timestamp

  • Real-time validation

  • Logging to both App Insights and custom audit tables

Trying to do that in ADF felt like forcing a screwdriver into a USB port.

💡 Bonus: With Durable Functions, I got retry logic, checkpoints, and orchestration for long-running jobs — with way less hassle than you'd expect.


But Then There’s Azure Data Factory: ADF for the Win (Sometimes)

Now, ADF isn’t bad — far from it. If your use case looks like:

  • Copying data between known sources (SQL to Blob, API to Lake, etc.)

  • Running basic transformations with minimal code

  • You want built-in monitoring, scheduling, lineage, and retry logic

  • You’re working with a team that isn’t heavy on dev resources

Then ADF is a dream. Click around, fill out some wizard forms, link some datasets, and boom — pipeline done.

I’ve used it for internal data syncing tasks where I just needed to pull nightly snapshots from partner APIs into Azure SQL. No need to build or deploy anything — it just worked.

Cost Comparison: They’ll Both Charge You... Differently

Azure Functions

  • Charged per execution and runtime (consumption plan = cheap, durable = still affordable)

  • Cost scales with usage and memory

  • Debugging issues might cost you time more than money

Azure Data Factory

  • Charged per pipeline run, data movement (per activity), and integration runtime hours

  • Costs can explode if you move large volumes or run frequent syncs

  • Easier to predict for simple jobs, riskier for complex ETL

⚠️ TL;DR: ADF can feel cheap — until it’s not.
Functions can feel dev-heavy — until they’re not.

What’s More Performant?

Azure Functions generally win in performance when:

  • You’re dealing with lots of logic-heavy transformations

  • You can parallelize things smartly

  • You want to tightly control how and when things scale

ADF performs better when:

  • You’re moving big chunks of data across Azure-native services

  • Your logic is simple but your data volumes are high

  • You’re OK with the "black box" nature of some activities


My Personal Rule of Thumb

If it’s highly custom, developer-led, and requires logic beyond what ADF easily supports →
🛠️ Azure Functions all the way

If it’s data movement, low-code-friendly, or part of a larger BI/data team pipeline →
📊 ADF makes life easier

Wrapping Up

It’s not about which one is “better.” It’s about what fits the problem you have.

For my personal projects (and even some team ones), Azure Functions give me the flexibility and ownership I want — especially when paired with things like Blob triggers, queues, and SQL.

But when I need something quick, visual, and not dev-heavy? ADF is a solid pick.

Just... watch that pricing page 👀

ETL and chill. Pick your poison wisely.