Sometimes, when changes are made in a datamodel, you need to reprocess historical data.
This can happen because
- a schema change
- a bug is fixed
- a definition of a metric has changed
Now, just pressing “rebuild all” for convenience is not always the most cheap option.
The beauty of using a query orchestration tool like Dataform is how you can manage what queries run, when they run, and in what order.
In this post, I’ll describe how to rebuild only a part of your models, namely the ga4_sessions table.
Some dataform internals: a quick primer
The GA4Dataform workflow knos two main modes of operation:
- you run a full refresh
- or an incremental build
If you manually run a workflow, you can toggle beteeen the two modes.
Full refresh will delete all tables (even the incremental ones) and replace them with new ones, built from scratch.
Non-full refresh will use the incremental logic, to append data to existing tables – which is usually cheaper.
But there is another level of granularity, it’s not “all or nothing”.
Actions and tags, dependencies and dependents
Selection of actions are basically queries, or assertions, or operations. Everything that can be executed. When you select one or more “actions”, you can choose to execute one, more or all actions.
Selection of tags are labels that are given to an action on the code level. In the config section of an action, you can apply tags. If you select a tag name, all actions labeled with that tag are executed.
After Selecting tags or Actions, you have 2 extra checkbox options to choose from:
Include dependencies will add all “upstream” actions of the selection. So anything “left” of the lineage. Or: all actions that will need to be executed to make the selected actions possible.
Include dependents will add all “downstream” actions of the selection. So anything “right” of the lineage. Or: all actions that use the output of the selected actions.
Let’s rebuild the ga4_sessions table
In the image below, you can see the lineage of a typical GA4Dataform install. The actions that are essential to rebuild the session table are
- int_ga4_sessions – the intermediate, incremental table
- ga4_sessions – the final table, which is also incremental
In the “Execute manual workflow” screen that you see after choosing “Start Execution” on the “Releases and scheduling” tab of your workspace, you will need to select those two actions.
You can use Selection of actions and select both tables – which is not very convenient
Or you can use Selection of tags and choose sessions as tag. This is easier.
After that, choose
- Include dependents – to make sure all your tables that use the rebuilt session table are also refreshed
- Run with full refresh – to make sure the sessions table is rebuilt from scratch
By leaving Include dependencies unchecked, we make sure the events table is NOT rebuilt – saving you time and possibly some costs.
After clicking Start Execution, your actions will run.
As you can see, the session table is rebuilt, and everything downstream (in this case, the “demo_daily_sessions_report” table and the assertions are automatically included.
That’s it! You’ve rebuilt the sessions table. Happy querying!
More mini-tutorials
This was a quick mini dataform tutorial, covering a fraction of what dataform can do. I hope it was useful.
If you want more of these mini-tutorials about dataform or GA4Dataform topics, drop a comment below, or let us know on our Linkedin page – we’re always happy to share knowledge!
