Skip to content

scene_pipelines.ingestion.provenance

DuplicatePolicy

Bases: str, Enum

What to do when the tracked row's primary key is already stored.

DataJoint's skip_duplicates / replace flags only look at the primary key. This policy also compares the content_hash stored in the row-meta stamp, so callers can detect a re-registration with different content. A stamp of another HASH_VERSION is treated as missing: its hash is not comparable.

Attributes:

Name Type Description
REJECT

Raise ValueError.

SKIP

Leave the stored row and its stamp untouched.

VERIFY

Leave the stored row untouched when the stamp hash matches payload; raise ValueError when it differs or no stamp exists.

OVERWRITE

Overwrite the row and it's stamp. Warns when the hash changed. Omitted optional fields are reset to their default.

insert_tracked_row

insert_tracked_row(row_meta_table: type[RowMetaBase], row: DjRow, *, payload: dict[str, Any], deployment: DjRow[Deployment], if_exists: DuplicatePolicy, writer_version: str, parts: Mapping[type[Part], Sequence[DjRow]] | None = None) -> DjKey

Insert row into row_meta_table.tracked_table and stamp it.

A new primary key is inserted and stamped. An existing primary key is handled by if_exists. parts are written whenever the row is: inserted with a new row, replaced on OVERWRITE, and left untouched otherwise. The Deployment row is inserted when missing. All statements run atomically: inside the caller's open transaction when there is one, otherwise in a transaction opened here.

Parameters:

Name Type Description Default
row_meta_table type[RowMetaBase]

Row-meta table; its tracked_table receives row.

required
row DjRow

Full insert dict for the tracked table.

required
payload dict[str, Any]

Fields hashed into content_hash (caller-defined shape).

required
deployment DjRow[Deployment]

Deployment row to stamp with.

required
if_exists DuplicatePolicy

Policy when the tracked primary key already exists.

required
writer_version str

Stamped ingestion_version: the version constant of the calling ingestion package, e.g. SCENE_WRITER_VERSION.

required
parts Mapping[type[Part], Sequence[DjRow]] | None

Optional part rows per part table, without the master key (it is added here), e.g. {Session.Subject: [{"subject_id": ...}]}. Include them in payload so the hash covers them.

None

Returns:

Type Description
DjKey

Primary key of the tracked row.

Raises:

Type Description
ValueError

When if_exists rejects the existing row.

SchemaVersionError

When a written database's schema version does not match the installed code (checked once per database and process).