Trace data sources with AI
Follow a report field or warehouse table back through the Airflow tasks that supplied it, keeping calculations, joins and evidence gaps visible.
You can trace data sources with AI when a metric’s name conceals how it was calculated. Start with the refund figure in a monthly report, for example. The tracing-upstream-lineage skill looks for the job that produced the target, then follows its inputs through intermediate tables towards the original system.
For a single field, the distinction between copying, deriving and aggregating a value matters. The result can show which source columns enter a calculation and where a filtering decision affects the final number. A connection name can identify an upstream system even when its internals are outside your access.
What it does
- Identify the DAG that writes a named table.
- Trace a field through renaming, calculations, joins and aggregation.
- Describe external sources referenced by the producing jobs.
- Build an upstream tree that distinguishes tables from tasks.
- Inspect recent run status or follow up on the freshness of a source.
Install and use
- Install the skill and connect the af CLI to an authorised Airflow environment. Provide available DAG definitions and warehouse schema access. The companion freshness skill is useful for checking source dates.
- Ask, “Trace monthly net sales to its inputs. Show where discounts enter the calculation and mark any part you cannot verify.”
- Read the chain in order from source to report. Keep a gap open when a dynamic query or missing definition prevents confirmation.
FAQ
Source
Made by Astronomer. Original skill: tracing-upstream-lineage, published on GitHub under the Apache-2.0 licence.
View on GitHub