Skip to main content
`Ctrl`+`K`

# smallpond.logical.node.ProjectionNode

class smallpond.logical.node.ProjectionNode(ctx: Context, input_dep: Node, columns: List[str] | None = None, generated_columns: List[Literal['filename', 'file_row_number']] | None = None, union_by_name=None)

  Select columns from output of an input node.

  __init__(ctx: Context, input_dep: Node, columns: List[str] | None = None, generated_columns: List[Literal['filename', 'file_row_number']] | None = None, union_by_name=None) → None

    Construct a ProjectNode to select only the columns from output of input_dep.

    ## Parameters

    input_dep

      The input node whose output would be selected.

    columns, optional

      The columns to be selected or created. Select all columns if set to None.

    generated_columns

      Auto generated columns, supported values: filename, file_row_number.

    union_by_name, optional

      Unify the columns of different files by name (see https://duckdb.org/docs/data/multiple_files/combining_schemas#union-by-name).

    ## Examples

    First create an ArrowComputeNode to extract hosts from urls.

        class ParseUrl(ArrowComputeNode):
            def process(self, runtime_ctx: RuntimeContext, input_tables: List[arrow.Table]) -> arrow.Table:
                assert input_tables[0].column_names == ["url"] # check url is the only column in table
                urls, = input_tables[0].columns
                hosts = [url.as_py().split("/", maxsplit=2)[0] for url in urls]
                return arrow.Table.from_arrays([hosts, urls], names=["host", "url"])

    Suppose there are several columns in output of data_partitions, ProjectionNode(…, [“url”]) selects the url column. Then only this column would be loaded into arrow table when feeding data to ParseUrl.

        urls_with_host = ParseUrl(ctx, (ProjectionNode(ctx, data_partitions, ["url"]),))

  Methods

  `__init__`(ctx, input_dep[, columns, ...]) | Construct a ProjectNode to select only the columns from output of input_dep.
  `add_perf_metrics`(name, value) |
  `create_task`(*args, **kwargs) |
  `get_perf_stats`(name) |
  `slim_copy`() |
  `task_factory`(task_builder) |

  Attributes

  `enable_resource_boost` |
  `num_partitions` |