🎤 Open Source data tools revisited – #1
For me data modelling is the core asset of your data setups. A lot of areas in data engineering can be covered by 3rd party service (eg. Integration, Storage) but the model is where creativity, skillsets and art is needed.
Here you define how you data-drive your company. How to calculate subscription values, churn rates, customer lifetime value,… Everything that will later be used as natural given.
dbt is a library/service that helps you to manage the modelling process. It gives structure to your sql models. It creates relations between them and it lets you write macros for ever returning tasks (marketing channel groupings).
dbt was open source from the beginning and is maintained by the folks at Fishtown Analytics and it took the data engineering world by storm.
I would not do a setup without dbt (or similar services) anymore . It would always feel that there is something missing.
Go, put it on your lists for tools to check out, if you haven’t done. If you have any questions, just write me here.
I post my analytics & data thoughts every morning. Don’t miss the good ones and just follow me. I promise I always follow back.