Skip to content

graph tool

Use graph after the agent adds, moves or renames a step of a pandas pipeline or a node of an Apache Hamilton module, as xpd graph does. It reads which step or node reads which, and reports what would fail, without running the code.

graph(paths)
  • Changes no file
  • A second call with the same arguments changes nothing more
  • Stays inside the project

Tags: dataflow.

The server gives the agent this description of the tool.

Read the dataflow of step code or Apache Hamilton modules without running them: which step or node reads which, and what does not connect. Changes no file.

Run it after you add, move or rename a step or a node. Fix every error, and fix a warning unless you can say why the code is right. Before you add a column to step code, read pipelines[].steps and reuse a column that exists. For Hamilton, pass all modules of one Driver together. inputs are what the caller passes.

Type: list of string. Required.

Files or folders inside the project root.

The dataflow that xpd graph read from step code and Hamilton modules.

The 38 fields of the result
Field Type Required Default Description
pipelines list of object yes
pipelines[].name string yes
pipelines[].file string yes
pipelines[].line integer yes
pipelines[].steps list of object yes
pipelines[].steps[].name string yes The function, or the call of a chain link: .assign(gross).
pipelines[].steps[].kind string no "function" function, or link for a chain link.
pipelines[].steps[].line integer yes
pipelines[].steps[].reads list of string yes Columns the step reads.
pipelines[].steps[].columns list of string yes Columns the step adds.
pipelines[].steps[].computed boolean yes True when the step also adds columns named at run time.
pipelines[].links list of object yes
pipelines[].links[].from string yes
pipelines[].links[].to string yes
pipelines[].links[].columns list of string yes
pipelines[].source list of string no Columns the steps read that no earlier step adds: columns= of the read.
pipelines[].source_complete boolean no true False when a step also reads columns named at run time.
pipelines[].free list of string yes Steps that share no column with another step.
nodes list of object yes
nodes[].name string yes
nodes[].file string yes
nodes[].line integer yes
nodes[].function string yes
nodes[].inputs list of object yes
nodes[].inputs[].name string yes
nodes[].inputs[].optional boolean yes
nodes[].kind string yes function, parameterized, extracted or saved.
nodes[].config string or null yes The @config.when decorator, when there is one.
inputs list of object yes
inputs[].name string yes
inputs[].optional boolean yes
inputs[].readers list of string yes
unread list of string yes Nodes that no other node reads.
problems list of object yes
problems[].file string yes
problems[].line integer yes
problems[].severity "error", "warning", "note" yes
problems[].message string yes
{"paths":["samples/graph/steps_wrong_order.py"]}

The result holds pipelines, nodes, inputs, unread and problems. These are its problems.

{"problems":[{"file":"samples/graph/steps_wrong_order.py","line":11,"severity":"error","message":"margin() reads the column total_cost, which costs() adds later in report(). The read raises KeyError. Move margin() after costs()."},{"file":"samples/graph/steps_wrong_order.py","line":27,"severity":"error","message":"costs_again() joins the column total_cost, which costs() added already. The join raises \"columns overlap\"."}]}

Problems in the code are a result, not an error. A path outside the project root fails before xpd runs.