Skip to content

Add self.upstream property for pre-restricted ancestor access in make() #1424

@dimitri-yatsenko

Description

@dimitri-yatsenko

Summary

Inside make(), provide self.upstream — a pre-constructed Diagram.trace(self & key) — as a property on auto-populated tables. This gives make() convenient, provenance-safe access to pre-restricted ancestor table expressions.

Context

Discussion: #1232
Depends on: #1423 (Diagram.trace())
Related: #242 (farfetch)

Design

The make(self, key) signature stays unchanged. self.upstream is simply a new property available during make() execution:

def make(self, key):
    # Read from upstream — pre-restricted by key
    session_date = self.upstream[Session].fetch1("session_date")
    traces = self.upstream[ExtractTraces].to_arrays("trace")

    # All query operators work — upstream[Table] returns a QueryExpression
    self.upstream[Session].aggr(self.upstream[Scan], n='count(scan_id)')

    # Write to self
    self.insert1({**key, "result": compute(traces)})

Read from self.upstream, write to self. The read/write boundary is the provenance boundary.

  • self.upstream only exposes declared ancestors and their part tables — requesting anything else raises an error
  • key is passed as before — no signature changes, no redundancy
  • make_kwargs work unchanged
  • Existing code that doesn't use self.upstream is completely unaffected

Implementation

  • The framework sets self.upstream = Diagram.trace(self & key) before calling make()
  • self.upstream[TableClass] returns pre-restricted QueryExpression objects with full fetch capabilities
  • Requesting a table outside the declared ancestor graph raises DataJointError

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureIndicates new features

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions