Mir's Data .Report

Share this post

Control Panels, Data Modeling, and Snowflake

mirdata.substack.com

Control Panels, Data Modeling, and Snowflake

In an Interview with the Co-Founder of Mode Analytics, Benn Stancil

Sep 3, 2022
1
Share
Share this post

Control Panels, Data Modeling, and Snowflake

mirdata.substack.com

Last week, I had the pleasure of chatting with Benn Stancil from Mode Analytics
 
If you ever read Benn's blog, you already know that this is going to be an engaging discussion.

As always, covered lots of stuff. We talked about:
- Control Panels for Data Stack
- Possible new Metrics for Data Engineering Tooling
- How to think in terms of “Edges” for Required Deliverables
- The importance of Central Scheduler as an Architect
- And what data teams could learn from Engineering

But most pleasant of all was what we arrived at a conclusion of our Podcast:
> Data Teams should Do a Lot More Telling...
> and stop focusing on "being helpful"
> “this is how it is going to work, not that way” 😎💪


SQL is all the hype right now. With companies such as DBT going “all in” on SQL and making a big thing out of it, SQL has made a big comeback over the last decade.

So it was quite interesting to hear Benn make this comment about Modeling Languages, and their “declarative” advantage. (Yours truly agrees)

“When Data Modeling Languages actually make more sense than SQL”


If you haven’t yet seen this post on LinkedIn, you should definitely watch the video. We’ve all known this, but Benn spoke it first.

“There is incentive to pile things into Snowflake”

Some additional talking points on this same subject:

  • We can be more precise about running Data Building jobs

  • Why not have data work the same way as Flight Scheduling


The idea of a Global Control Panel for Data is all the rage, but no one really knows what would that entail. Benn had a pragmatic point of view here:

“Need someone, a team, or a system that acts like an architect, so that when someone adds "a room for the house", this system can say "not a good place for that room"“

More on this:

  • Data tooling has gotten increasingly complicated. You have 100 different systems, and they all do different things

  • We should start thinking in terms of "edges" or final requirements when designing data pipeline schedulers


Finally, forget Tooling, no Data talk is truly complete without addressing Best Practices related to “how we work”.

“There needs to be a clear line between production and not ("correct once, but not anymore")”

And, of course, the conclusion of our conversation:

  • Data Teams should act more like Engineering Teams while responding to Customer Requests


That’s it for now. A few words about the writer:

In my former life I was a Data Scientist. I studied Computer Science and Economics for undergrad and then Statistics in graduate school, eventually ending up at MiT, and, among other places, Looker, a business intelligence company, which Google acquired for its Google Cloud.

Yours truly, Segah A. Mir
1
Share
Share this post

Control Panels, Data Modeling, and Snowflake

mirdata.substack.com
Previous
Next
Comments
Top
New

No posts

Ready for more?

© 2023 Segah A. Mir
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing