graph tool
Use graph after the agent adds, moves or renames a step of a pandas pipeline or a
node of an Apache Hamilton module, as xpd graph does. It reads which step or
node reads which, and reports what would fail, without running the code.
graph(paths)- Changes no file
- A second call with the same arguments changes nothing more
- Stays inside the project
Tags: dataflow.
Description
Section titled “Description”The server gives the agent this description of the tool.
Read the dataflow of step code or Apache Hamilton modules without running them: which step or node reads which, and what does not connect. Changes no file.
Run it after you add, move or rename a step or a node. Fix every error, and fix a warning unless you can say why the code is right. Before you add a column to step code, read pipelines[].steps and reuse a column that exists. For Hamilton, pass all modules of one Driver together. inputs are what the caller passes.
Arguments
Section titled “Arguments”Type: list of string. Required.
Files or folders inside the project root.
Result
Section titled “Result”The dataflow that xpd graph read from step code and Hamilton modules.
The 38 fields of the result
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
pipelines |
list of object | yes | ||
pipelines[].name |
string | yes | ||
pipelines[].file |
string | yes | ||
pipelines[].line |
integer | yes | ||
pipelines[].steps |
list of object | yes | ||
pipelines[].steps[].name |
string | yes | The function, or the call of a chain link: .assign(gross). | |
pipelines[].steps[].kind |
string | no | "function" |
function, or link for a chain link. |
pipelines[].steps[].line |
integer | yes | ||
pipelines[].steps[].reads |
list of string | yes | Columns the step reads. | |
pipelines[].steps[].columns |
list of string | yes | Columns the step adds. | |
pipelines[].steps[].computed |
boolean | yes | True when the step also adds columns named at run time. | |
pipelines[].links |
list of object | yes | ||
pipelines[].links[].from |
string | yes | ||
pipelines[].links[].to |
string | yes | ||
pipelines[].links[].columns |
list of string | yes | ||
pipelines[].source |
list of string | no | Columns the steps read that no earlier step adds: columns= of the read. | |
pipelines[].source_complete |
boolean | no | true |
False when a step also reads columns named at run time. |
pipelines[].free |
list of string | yes | Steps that share no column with another step. | |
nodes |
list of object | yes | ||
nodes[].name |
string | yes | ||
nodes[].file |
string | yes | ||
nodes[].line |
integer | yes | ||
nodes[].function |
string | yes | ||
nodes[].inputs |
list of object | yes | ||
nodes[].inputs[].name |
string | yes | ||
nodes[].inputs[].optional |
boolean | yes | ||
nodes[].kind |
string | yes | function, parameterized, extracted or saved. | |
nodes[].config |
string or null | yes | The @config.when decorator, when there is one. | |
inputs |
list of object | yes | ||
inputs[].name |
string | yes | ||
inputs[].optional |
boolean | yes | ||
inputs[].readers |
list of string | yes | ||
unread |
list of string | yes | Nodes that no other node reads. | |
problems |
list of object | yes | ||
problems[].file |
string | yes | ||
problems[].line |
integer | yes | ||
problems[].severity |
"error", "warning", "note" |
yes | ||
problems[].message |
string | yes |
Example
Section titled “Example”{"paths":["samples/graph/steps_wrong_order.py"]}The result holds pipelines, nodes, inputs, unread and problems. These are its
problems.
{"problems":[{"file":"samples/graph/steps_wrong_order.py","line":11,"severity":"error","message":"margin() reads the column total_cost, which costs() adds later in report(). The read raises KeyError. Move margin() after costs()."},{"file":"samples/graph/steps_wrong_order.py","line":27,"severity":"error","message":"costs_again() joins the column total_cost, which costs() added already. The join raises \"columns overlap\"."}]}Errors
Section titled “Errors”Problems in the code are a result, not an error. A path outside the project root fails before xpd runs.