Skip to content

Session API

Session is the entry point: from isocenter import Session. It is the class isocenter.session.DicomSession, rendered below. Its constructor and every method on this page are frozen at 1.0 (API stability), and so are the attributes in its table except store_backend, which is tier 2. The methods are listed in pipeline order, ingest → examine → config → audit → anonymize → redact → verify → export → report, with compact, release_memory and close last. What the methods return and raise is on Results and errors.

Session(persistence_file=None)

The session: an indexed store of DICOM files and the de-identification pipeline over it.

Import it as from isocenter import Session. Use it in a with block, or call close(): an open session holds a worker pool and two threads.

Attributes:

Name Type Description
store DicomStore

The object graph. store.patients is the List[Patient] of Patient -> Study -> Series -> Instance.

configuration IsocenterConfiguration

The configuration audit(), anonymize(), redact() and export() apply.

store_backend SqliteStore

The session's SQLite store. Its get_audit_* methods read the audit trail.

persistence_file str

The store's path, or ":memory:".

Open the session store at persistence_file, creating it if needed.

The store is the SQLite file and a pixel file beside it (by default isocenter.db and isocenter_pixels.bin). It holds the original identifiers and pixels of everything ingested; keep it where you keep the source data. Until export(), the session writes only the store, isocenter.log and files you ask for (a configuration, a key).

When a file named isocenter.key exists in the current working directory, the session calls enable_reversible_anonymization() with it, resolved to an absolute path now. With no such file, reversible anonymization stays off and no key is created.

Parameters:

Name Type Description Default
persistence_file str

Path to the SQLite database file for session persistence. Defaults to ISOCENTER_DB_PATH, then "isocenter.db". ":memory:" is accepted and is part of the frozen surface: the index lives in memory and the pixel sidecar in a temporary file the store unlinks on close(). On such a store redact() runs in threads on every interpreter, because its worker writes to the store and a process cannot share an in-memory database; with ISOCENTER_MAX_TASKS_PER_CHILD set, recycling overrides that and the call fails -- see that variable on the Environment Variables page.

None

Raises:

Type Description
ValueError

isocenter.key in the working directory is empty or is not a Fernet key.

ingest(directory)

Ingest every DICOM file under a directory into the session store.

Walks directory recursively, reads each DICOM file into the Patient -> Study -> Series -> Instance hierarchy, and saves the session when it finishes.

A file that cannot be ingested does not raise. It is counted in the returned summary and gets an ERROR audit row naming the path and the reason, which bars a PASS grade. Check the return value: a run that rejected files completes normally.

A file that ends the worker process reading it (the out-of-memory killer, a decoder crash, SIGKILL) is read again alone on a fresh worker, and rejected only if it ends that worker too. Files already read are kept, and a death that does not recur costs no file and logs one WARNING line. If fresh workers cannot run at all, every file left is rejected as "Not read", with a reason naming the usual causes (a script without the if __name__ == "__main__": guard among them), and the call returns. Any other failure of the worker pool raises.

Duplicate SOP Instance UIDs. A file whose SOP Instance UID an instance in this session already holds (ingested earlier in this call, by an earlier call, or loaded from the store) is declined: the instance is kept, the file is not read into the store, it is counted in IngestSummary.declined, and a WARNING audit row names the UID, the file, and the file the instance was ingested from. An instance the session already holds is always kept over a new file. Among files new to this call, the one whose path sorts first is kept, whatever order the filesystem lists them in; the sort is on the path string as walked (os.path.join of the directory and the name), the one the WARNING row prints. A declined file is not recorded as imported, so ingesting the same folder again declines it again.

Byte order. A big-endian source's values in words wider than a byte (OW, OL, OF, OD, OV and the waveform samples) are stored little-endian, as its pixels are. What cannot be converted whole (a UN value, a length that is not a whole number of words, samples with no usable Waveform Bits Allocated) is kept as read and gets one WARNING audit row per element.

Environment. ISOCENTER_FORCE_THREADS and ISOCENTER_MAX_TASKS_PER_CHILD have no effect: ingest runs on the session's own process pool, which has no threads mode and never recycles a worker. Each call that has files to read logs one WARNING when either is set. ISOCENTER_MAX_WORKERS is read on every call, and the pool is rebuilt when the width has changed; an unchanged width rebuilds nothing. While another ingest() in this session is running on the pool, a changed width is reported in one WARNING and this call runs at the pool's current width.

Concurrency. compact() on any thread of this session raises while an ingest runs. While a compact() is saving or rewriting, this call waits (bounded, see Raises) and then proceeds. A file whose pixels cannot be written in that time is rejected like any other failed file, with an ERROR audit row naming the path and the reason.

Parameters:

Name Type Description Default
directory str

The path to the directory containing DICOM files.

required

Returns:

Type Description
IngestSummary

How many files reached the store, and (path, reason) for each one that did not.

Raises:

Type Description
RuntimeError

If the ingest cannot start within 180 s because a compact() is still saving or rewriting the sidecar.

save(sync=False)

Persist the current session state to the store.

Parameters:

Name Type Description Default
sync bool

If True, block until the save is complete. A synchronous save first drains the persistence manager, as audit() and redact() do, and never returns early: a background save that does not finish blocks it.

False

examine()

Prints a summary of the session contents and equipment.

create_config(output_path)

Generates a unified configuration file (scaffold) in YAML format.

Reads the session inventory, pre-fills redaction rules for any machine the shipped knowledge bases recognise, adds default PHI tag policy, and writes the result as commented YAML.

Parameters:

Name Type Description Default
output_path str

Where to write the generated YAML. A .yaml suffix is appended if missing.

required

load_config(config_file)

Load a configuration file as the session's configuration.

Loading changes no data. preview_config() shows which instances its redaction rules match; audit(), anonymize() and redact() apply it.

Parameters:

Name Type Description Default
config_file str

Path to the YAML configuration file.

required

Raises:

Type Description
FileNotFoundError

If config_file does not exist.

ValueError

If the file fails validation: not .yaml/.yml, YAML syntax, a root that is not a mapping, a version this library does not read, a key the schema does not have at any level, a value of the wrong type, an unknown privacy_profile (including a <profile>@<edition> this version does not ship), an unknown action, a phi_tags, date_jitter or machines of the wrong shape, an invalid machine rule, or a phi_tags rule the pipeline cannot honour. After either error the configuration is exactly what it was before the call.

preview_config()

Performs a dry-run of the currently loaded configuration.

Checks the active redaction rules against the current session inventory and prints a summary of which instances would be affected (matched) by the rules. Does not modify any data.

audit(config_path=None)

Scan every patient in the session for PHI under a tag policy.

The policy is the file at config_path when one is given, and otherwise session.configuration.phi_tags. The scan runs in parallel worker processes.

Before the scan, two Patient objects holding one Patient ID are merged into the one that was in the session first, so store.patients can get shorter, as after anonymize().

Every status the scan records is recorded with the policy it ran under: the configuration's, or, with config_path, that file's rules under this session's remove_private_tags. An entity whose status is unchanged but whose policy differs is re-recorded, so the next save() writes it.

Parameters:

Name Type Description Default
config_path str

Path to a configuration file whose PHI rules to scan with.

None

Returns:

Type Description
PhiReport

The findings: iterable, indexable, and convertible with to_dataframe().

Raises:

Type Description
ValueError

When the file at config_path fails any check load_config() makes, or the policy (that file's, or configuration.phi_tags) holds a rule the pipeline cannot honour. Raised before a project secret is created.

RuntimeError

When patients sharing a Patient ID were de-identified under different date-offset schemes, so they cannot be merged; raised after the policy is validated and before anything is scanned or a project secret is created. Also on a store holding dates shifted under a project secret it no longer has.

auto_remediate_config(report)

Analyzes the provided OCR report and automatically updates the session's configuration to fix detected leaks (by expanding zones or adding new ones).

Parameters:

Name Type Description Default
report PhiReport

The findings from .scan_pixel_content()

required

Returns:

Type Description
int

The number of rules updated.

anonymize(findings=None)

Apply the remediation each PHI finding proposes (tag anonymization).

With findings, only those findings are remediated. An empty list, tuple, PhiReport or iterator applies nothing and returns 0: a filtered report that matched nothing is not a request to remediate everything. Only findings=None, or no argument, runs a full audit() under the current configuration and remediates every finding it raises. An empty call still reads the store's project secret, so a store that has lost it refuses rather than returning 0.

Nothing outside session.store is written. A finding whose entity is itself in the graph is acted on as it is, except a REMOVE_TAG, whose "already gone" is read on the object at the finding's address, and which declines where the two differ. Any other finding is resolved against the live graph at its entity_uid and entity_path (an instance's UID from before redact(), and a patient's original Patient ID after the pseudonym this store minted for it, included) and acts on the object found there, so a report kept across close() and a reopen cleans the graph export() writes. A finding whose address names no single object declines; one inside a sequence a pass already removed or emptied is satisfied. A Patient ID is written, and a date shifted, only with a value that belongs to the live patient holding it (this store's pseudonym, and an offset seeded on that patient under its own scheme), and declines otherwise. The findings passed are not modified.

Two patients left holding one Patient ID (a study ingested under a patient's original ID after that patient was anonymized) are merged into whichever was in the session first, and the other is removed from store.patients.

Parameters:

Name Type Description Default
findings List[PhiFinding]

Specific findings to clean.

None

Returns:

Type Description
int

How many remediations were applied. Failures are logged and excluded, so a caller can tell a clean run from a partial one.

Raises:

Type Description
RuntimeError

When two patients left holding one Patient ID were de-identified under different date-offset schemes. Raised at the merge, after the remediations are applied. Unreachable on a graph the library built, and reachable on one built in user code.

enable_reversible_anonymization(key_path='isocenter.key')

Turn on reversible anonymization with the key file at key_path.

Loads the key when a file is there. This call never creates the key file, and neither does recovery: the first lock_identities() creates it when none exists. So a mistyped path before recover_patient_identity() fails there with FileNotFoundError rather than minting a key the data was never locked under.

Parameters:

Name Type Description Default
key_path str

Path to the key file.

'isocenter.key'

Raises:

Type Description
ValueError

The file at key_path is not a Fernet key, or is empty (the message names the path). Nothing is cached by a failed enable: fix the file and enable again.

lock_identities(patient_id, persist=False, *, verbose=True, tags_to_lock=None)

Encrypt each instance's original identifiers into an identity token carried in the instance, for reversible anonymization.

The token is written into the Encrypted Attributes Sequence (0400,0500) under the key enable_reversible_anonymization() names. Values are captured from each instance, and one token is written per distinct set of them: under the default tags a patient with several studies carries about one token per study, and each instance's token holds that instance's own values.

Call it before anonymize() if recovery is required: afterwards the identifying tags no longer hold their original values, and the lock refuses to capture what a remediation wrote.

Anything but a str (an iterable of Patient IDs, a PhiReport or a list of findings) is handed to lock_identities_batch(), with the same persist, verbose and tags_to_lock. auto_persist_chunk_size is that method's own argument.

Refusals. The lock raises RuntimeError and writes no token when the patient cannot be locked as asked:

  • a value the lock would capture was written by a remediation (ANONYMIZED, ANON_..., a rule's value:, a shifted date);
  • a tag it names was emptied or removed by anonymize(); the message names the tags_to_lock that works without it;
  • a re-lock would lose a value the existing token holds;
  • the patient carries an identity token this library wrote that the key at the path given to enable_reversible_anonymization() does not decrypt, or that opens to no identity record, or that is in the layout releases before 1.0 wrote, which 1.x does not read;
  • a re-lock over a token this store did not write (one that arrived inside a file, or one a release before 0.9.8 wrote) would change a value it holds, naming the tag, which need not be one tags_to_lock names; recover_patient_identity(..., restore=True) followed by a lock is the way through;
  • a value it would capture is one no token can hold (bytes), naming the tag;
  • Patient's Name is blank under a rule of EMPTY or REMOVE on it;
  • the patient has instances and any of them holds no value in any tag tags_to_lock names (or it names none), counted. A tag held blank is a value, and a patient with no instances locks as 0 instances.

Each of these is judged on every instance's own values: a value a pass wrote on any study refuses the lock. An existing token is judged against the first instance that carries it. Before any patient is planned, the lock also refuses when no key file exists at the path and an instance in the session carries a token this library wrote; no key is created. Given a list or a report, every patient is checked first and, if any is refused, one error lists each and no patient is locked.

No message carries a Patient ID: a message says "this patient", its advice spells the ID <its Patient ID>, and a replaced Patient ID is described, not quoted. When the lock creates the key file (the first lock under a path with none), the file is created already written, with mode 0600.

Parameters:

Name Type Description Default
patient_id str

The ID of the patient to lock; anything that is not a str (a list, set, frozenset or iterator of IDs, or a report) is handed to lock_identities_batch().

required
persist bool

If True, writes each instance's token into the row the store holds for it, immediately; an instance the store holds no row for raises (see RuntimeError). If False, returns the modified instances for a later save().

False
verbose bool

If True, logs debug information.

True
tags_to_lock List[str]

The tags whose original values are embedded. When omitted: PatientName, PatientID, PatientBirthDate, PatientSex and AccessionNumber.

None

Returns:

Type Description
Union[List[Instance], LockingResult]

A list of modified instances.

Raises:

Type Description
RuntimeError

When reversible anonymization is not enabled, or for a refusal above, before any token is written. Also, with persist=True, when some instance's current SOP Instance UID has no row in the store to write its token into: a patient built by hand and never saved, or a UID regenerate_uid() moved (as redact() does) since the last save. That one is raised after the tokens are embedded: they are in memory, marked modified, so a later save(sync=True) stores them; this write stored none of them, and one ERROR audit row gives the counts. ingest() writes the rows itself, and the lock drains a save queued by save() before it embeds anything, so neither ingest-then-lock nor save()-then-lock raises it.

TypeError

When patient_id is not a str and is not a selection lock_identities_batch() accepts: None, bytes-like, not iterable, or an iterable holding an item that is neither a str nor a finding. Raised before the key is loaded or created.

ValueError

The key file at the path is empty (the message names the path) or is not a Fernet key. Neither is cached: a later call reads the file again.

sqlite3.Error

With persist=True, the store refused the write. The instances already carry the new token in memory, marked modified, so a later save() writes them; this write stored none of them, and one ERROR audit row says so. The row speaks for the write, not the store, which keeps whatever an earlier write put there. Given a list or a report, the batch form's sqlite3.Error applies.

lock_identities_batch(patient_ids, auto_persist_chunk_size=0, tags_to_lock=None, *, persist=False, verbose=True)

Lock the identities of several patients; see lock_identities().

Parameters:

Name Type Description Default
patient_ids Union[Iterable[str], PhiReport]

The patients to lock: an iterable of Patient IDs (read once, so an iterator works), a PhiReport, or an iterable of findings, which may be mixed with IDs. Read as every other method reads patient_ids, except that None is refused: there is no spelling for "lock every patient", and the report audit() returns locks every patient its scan found.

required
auto_persist_chunk_size int

If > 0, persists changes and releases memory every N instances, and the call returns an empty list.

0
tags_to_lock List[str]

Passed to every patient's lock; lock_identities()'s five default tags when omitted.

None
persist bool

Passed to every patient's lock: each patient's instances are written as they are locked. With auto_persist_chunk_size > 0 as well, an instance is written twice (with its patient, then with its chunk), which is redundant, not wrong.

False
verbose bool

Passed to every patient's lock: one debug line per patient.

True

Returns:

Type Description
Union[List[Instance], LockingResult]

All modified instances, or an empty list when auto_persist_chunk_size > 0.

Raises:

Type Description
RuntimeError

When reversible anonymization is not enabled, or when any patient found cannot be locked as asked (the refusals lock_identities() lists). Every patient is checked before any is locked, so the one error lists each refused patient with its own message, in Patient ID order, and no patient is locked, whatever persist or auto_persist_chunk_size says. A Patient ID that matches no patient is logged, not raised. The promise is about refusals, not the store: see sqlite3.Error. No message names a patient: each refusal is prefixed [n of m], its place among the m patients found, in Patient ID order, so the refused patient is sorted(ids that matched a patient)[n - 1]. The refusal raised before any plan (no key file, and a token this library wrote somewhere in the session) is one message with no number, and creates no key.

TypeError

When patient_ids is None, a bare str (the single-ID spelling is lock_identities(patient_id)), bytes-like, not iterable, or holds an item that is neither a str nor a finding, named by its position and type, never its value. Raised before the key is loaded or created and before any patient is planned.

ValueError

The key file is empty or is not a Fernet key, as for lock_identities().

sqlite3.Error

A store write failed while tokens were being persisted (persist=True writes per patient, auto_persist_chunk_size per chunk), after one ERROR audit row; or, as RuntimeError, a store write found no row for an instance (see lock_identities()), with the same shape and timing: raised after the tokens are embedded, unlike the refusals above. Nothing is rolled back across writes: patients written before the failure stay locked in the store, the failed write stored none of its instances (they hold their new tokens in memory, marked modified, and a later save() writes them), and the patients after it in Patient ID order are not locked. One write is one transaction. With both persist=True and auto_persist_chunk_size, each instance is written with its patient and again with its chunk, so where a chunk write fails, its instances were already stored with their patients.

recover_patient_identity(patient_id, restore=True)

Decrypt a patient's identity tokens and return the identity they hold; with restore=True, also write it back onto the patient.

Which token speaks for the patient. The first token of ours in study, series and instance order: a study without a token, or with an Encrypted Attributes Sequence this library did not write, is walked past. For a patient with a Patient ID, a first token holding a blank Patient ID is passed over for the first token holding a non-blank one, when there is one. A subject with no Patient ID is spoken for by its first token always, and keeps its key.

Every distinct token is opened before anything is written, with restore=False too: a token of ours on any study that this key cannot open, or that holds no record, raises and writes nothing. So restore=False also checks that the patient is recoverable under this key. Every failure raises; nothing is printed, and no message names a Patient ID.

What restore=True writes. Every instance of the patient in memory takes the locked identity tags from the token it carries. The restore is recorded, so a later save() stores it, and a patient already holding the restored Patient ID is merged into whichever of the two was in the session first.

  • Only the locked tags are put back. Every other date stays shifted by the patient's offset, so intervals are intact and a later audit() does not shift it again. A date among the locked tags is put back like any locked tag, and a later audit() raises it again.
  • A restored Study Date is also put back on each Study, which is where export() reads it, from that study's own token, when the restored value reads as a date. A blank or unreadable one leaves the Study as it is, with one WARNING per such study that carries no date.
  • An instance carrying no token takes only the patient-level identifiers (group 0010, which include Patient's Age, Size and Weight, and so may be another study's) of the token that speaks for the patient, and keeps its other locked identifiers as the pass left them; one WARNING gives the count. For a patient with no Patient ID, such an instance keeps a non-blank Patient ID of its own rather than take the token's blank one, and a second WARNING counts those.
  • Where tokens disagree on Patient's Name or Patient ID, each instance keeps its own and the Patient takes the speaking token's, with a WARNING. A token whose Patient ID is blank does not disagree on it.

Parameters:

Name Type Description Default
patient_id str

The Patient ID the patient holds now (normally its pseudonym).

required
restore bool

If True, write the recovered identity back, as above. If False, write nothing.

True

Returns:

Type Description
Dict[str, Dict[str, Any]]

The identity recovered. Each key is the SOP Instance UID of an instance carrying an identity token of ours, as it was when the call began; each value is a deep copy of the values that instance's token holds, keyed "gggg,eeee", in study, series and instance order. An instance carrying no token is absent, and the dict is never empty (a patient with no token raises). Both modes return the same mapping, taken before restore=True writes anything, and it is what the tokens hold, not what the restore wrote: a tokenless instance a restore gives group 0010 of the first token is absent. The patient-level answer (the token whose name and ID a restore stamps on the Patient) is next(iter(result.values())). Two instances sharing one SOP Instance UID, which only a hand-built graph can hold, share one key, and the later one's token is the value. These are the original identifiers, handed to the holder of the key; nothing prints or logs them.

Raises:

Type Description
FileNotFoundError

No key file at the path given to enable_reversible_anonymization(). Checked first, before the patient is looked up, and no key is created.

ValueError

No patient in this session holds patient_id; or the key file is empty or is not a Fernet key, which is read before the patient is looked up and is not cached.

RuntimeError

When reversibility is not enabled; the patient has no instances, or no instance carrying an identity token (an Encrypted Attributes Sequence that did not come from this library counts as no token, not as the wrong key); the key does not decrypt the token, or the key opens it but it holds no identity record this library writes, or was written by a later release of this library; an instance's token is in the layout releases before 1.0 wrote, the token in (0400,0510), which 1.x does not read; or, with restore=True, a patient holding the restored Patient ID was de-identified under a different date-offset scheme (raised before anything is restored).

redact(show_progress=True, force=False)

Apply the configured pixel redaction zones to every matching image.

Uses session.configuration.rules: every image of a series whose Device Serial Number a rule matches has each of the rule's zones set to zero. This changes pixel data in memory (and the sidecar, for persistence); call save() afterwards to persist it. A redacted instance takes a new SOP Instance UID, derived from its source SOP Instance UID and the rule's zones.

Concurrency. compact() on any thread of this session raises while a pass runs. While a compact() is saving or rewriting, this call waits (bounded, see Raises) and then proceeds. A worker whose pixels cannot be written in time comes back as a failed redaction, with an ERROR audit row.

Parameters:

Name Type Description Default
show_progress bool

If True, displays a progress bar.

True
force bool

Redact again the instances already redacted under this configuration, instead of skipping them. Every instance the rules match is redacted again, and its SOP Instance UID is derived again from its source SOP Instance UID and the rule's zones: unchanged zones give the UID and exported filename it already has, and other zones give another. file_path becomes None either way. To repair a store redacted by 0.9.0 or earlier, see Upgrading from 0.9.x.

False

Returns:

Type Description
int

How many instances had at least one configured zone applied to their pixels. An instance a rule matched but whose every zone fell outside the image is not counted; a zone with no area fails its instance and the pass raises RedactionError, so it is never counted. Zero means nothing was redacted: no rules loaded, no image matched one, every match was already redacted under this configuration, or no zone landed.

Raises:

Type Description
RedactionError

If any instance's zone could not be applied. Raised at the end of the pass, not at the first failure: the instances that could be redacted are redacted, the failures are already ERROR rows in the audit log, and the console summary has been printed, so a caller that catches it still has a correct object graph and a compliance report that grades REVIEW_REQUIRED. .failures carries (sop_uid, detail) per failed instance. It subclasses RuntimeError, so except RuntimeError catches it and the RuntimeErrors below alike. A failed instance is left exactly as it was found: no DERIVED flag, no record of the redaction, nothing persisted, so a corrected configuration retries it.

Exception

Whatever the redaction backend raised, after logging it.

RuntimeError

If the pass cannot start within 180 s because a compact() is still saving or rewriting the sidecar. Raised before any worker is dispatched and before any UID is regenerated, so there is nothing to undo.

RuntimeError

On a ":memory:" store when the environment asks for worker recycling (ISOCENTER_MAX_TASKS_PER_CHILD): a recycled worker is a process, and a process cannot reach an in-memory database. The message names the store, the variable, the cause and two remedies. Raised after the persistence drain and before the pass-lock: no task prepared, no SOP Instance UID regenerated, no pixel touched, no audit row. Not a RedactionError, since nothing was attempted. ISOCENTER_FORCE_PROCESSES does not raise: this call asks for threads, which outranks the variable, and a WARNING names the variable instead.

redact_by_machine(serial_number, roi)

Redact one zone on one machine's images, without editing the configuration.

Replaces the configuration's rules with one rule for serial_number holding roi, runs redact(), and restores the original rules afterwards, also when redact() raises.

Parameters:

Name Type Description Default
serial_number str

The device serial number to target.

required
roi List[int]

The Region of Interest as [y1, y2, x1, x2].

required

Raises:

Type Description
RedactionError

Propagated from redact() when the zone could not be applied. The original rules are restored first.

RuntimeError

Propagated from redact(), which refuses a ":memory:" store whose environment asks for worker recycling and a pass-lock wait that expires. The original rules are restored first.

scan_pixel_content(serial_number=None)

Scan instances for burned-in text with OCR, and report the text no configured redaction zone covers.

Only instances of machines (by Device Serial Number) the current configuration has a rule for are scanned; other machines are skipped.

Parameters:

Name Type Description Default
serial_number str

Scan only the machine with this serial number.

None

Returns:

Type Description
PhiReport

Findings for burned-in text no zone covers. Each finding's entity is the live Instance in session.store, whether the scan ran in threads or in processes, or None when that instance cannot be found in the graph; never a worker's copy. Its failures lists (entity_uid, reason) for each instance whose pixels could not be loaded or whose OCR raised on any frame, and a WARNING log line gives the count. Each failure is also one WARNING audit row naming the instance and the reason, so the report grades REVIEW_REQUIRED. An instance with no pixel element is neither scanned nor a failure. A worker process runs the pytesseract.pytesseract.tesseract_cmd the caller set.

Raises:

Type Description
RuntimeError

pixel_analysis.OcrUnavailableError when the ocr extra is not installed or the tesseract binary does not answer in the calling process, before any worker is dispatched and before the graph is read. pixel_analysis.PixelScanError, carrying .failures and .attempted, after the pass, the audit rows and the warning, when at least one instance failed and none could be read.

discover_redaction_zones(serial_number, sample_size=50, min_confidence=80.0)

OCR a random sample of one machine's instances and collect where burned-in text appears.

Parameters:

Name Type Description Default
serial_number str

The Device Serial Number of the machine.

required
sample_size int

The most instances to read; a machine with more is sampled at random.

50
min_confidence float

The lowest OCR confidence, 0 to 100, a text candidate needs to be kept.

80.0

Returns:

Type Description
DiscoveryResult

Every text candidate found. Call to_zones() on it for grouped redaction zones. n_sources counts only the sampled instances that were read (at least one frame through OCR), so an instance that could not be read does not dilute a zone's occurrence rate. Each one that failed is logged at ERROR, counted in a WARNING, and written as one WARNING audit row naming the instance and the reason, which grades the run REVIEW_REQUIRED. A worker process runs the caller's tesseract_cmd.

Raises:

Type Description
RuntimeError

pixel_analysis.OcrUnavailableError when the ocr extra is not installed or the tesseract binary does not answer, before any worker is dispatched and before the graph is read. pixel_analysis.PixelScanError, carrying .failures and .attempted, after the pass, the audit rows and the warning, when at least one sampled instance failed and none could be read.

reconcile_private_tags()

Delete stored private-tag rows that the store's core attributes do not hold.

A repair for a store de-identified before 0.9.1, whose stripped private tags come back when it is opened; see Upgrading from 0.9.x.

It deletes every instance_attributes row whose tag is absent from its instance's stored core attributes, removes the same tags from the in-memory graph, and writes one RECONCILE_PRIVATE audit row per affected instance. The graph edit advances no revision: store and graph agree afterwards, nothing reads as unsaved, and stored PHI statuses are kept.

Call it only for a store you know was de-identified before 0.9.1. In a store that keeps its private tags (remove_private_tags: false, saved by 0.9.1 or later), those rows are the private data, and this deletes all of them. If unsure, run anonymize() and save() instead: a save removes the rows of tags the graph no longer holds. Nothing makes this choice automatically.

Returns:

Type Description
int

instance_attributes rows deleted, not tags: a value with multiplicity 3 is three rows, and a tag holding an empty value is one row. 0 means nothing changed.

export(folder, format='dicom', **options)

Export the session to a directory in the requested format.

Either format writes one WARNING audit row, and changes nothing it writes, when the instances it writes carry PHI statuses recorded under a policy that is neither the one in force nor one this session scanned under, or with no recorded policy (a store written before 1.0); the report then grades REVIEW_REQUIRED. check_burned_in=True re-audits first, so it never does.

Either format also writes one WARNING audit row, and logs one WARNING line, when patient_ids names an ID no patient in the session holds: counted by position, never named, and the report then grades REVIEW_REQUIRED. The patients that match are exported as asked; nothing raises. The dicom format does the same for a subset value that names nothing in the session at any level.

A format served by any exporter other than the two built-in classes (one registered through exporters.register, a subclass of a built-in, or another class registered as dicom) writes one WARNING audit row before it runs, saying its output is not attested by Isocenter, so the report grades REVIEW_REQUIRED. None of the export gates runs for it, and it writes no EXPORT row. The registry is provisional until 1.1; see Exporter registry.

Parameters:

Name Type Description Default
folder str

Output directory.

required
format str

Registered format name. "dicom" (default) writes cleaned DICOM files; "wfdb" writes PhysioNet WFDB records.

'dicom'
**options dict

Passed through to the selected exporter. The DICOM format's options are listed under "DICOM export options" on the Session API page.

{}

Returns:

Type Description
Any

The selected format's own result object. The DICOM exporter returns an ExportSummary, whose written counts the files that reached disk and whose failures names the instances that did not. written is counted over de-duplicated UIDs, because the UID names the output file: two instances sharing one are two successful write operations and one file, the second having overwritten the first. The WFDB exporter returns a List[str] of paths; an empty list means nothing was attempted, because an export that attempted records and wrote none raises instead.

Raises:

Type Description
ValueError

If format is not a registered export format. Also on dicom, before anything is written, for a subset query that does not run, or a subset DataFrame with none of SOPInstanceUID, SeriesInstanceUID, StudyInstanceUID and PatientID.

TypeError

For an option name the selected exporter does not recognise; nothing is written. The two formats do not accept the same options, so a caller forwarding one dict to both must split it. Also, on both formats and before anything is written, for a patient_ids that is a bare str (wrap one ID in a list), bytes-like, not iterable, or holds an element that is not a str; and on dicom for a subset that is bytes-like, not iterable, or holds an element that is not a str.

ExportError

From either exporter, when zero of N attempted instances reached disk and at least one failed. An empty plan (zero of zero) does not raise: a subset that matched nothing is a fact about the run, and the EXPORT audit row already carries it. Nor does a DICOM export whose every instance the pre-export scan withheld: nothing was attempted, and its WARNING rows grade the run.

export_dataframe(output_path='export_metadata.csv', expand_metadata=False, patient_ids=None)

Write the cohort report to CSV or Parquet, and return it.

The format is chosen from the extension: .parquet writes Parquet, anything else writes CSV.

It reports the session's in-memory graph and does not save() first: pending edits are not committed as a side effect.

Parameters:

Name Type Description Default
output_path str

The output file path (ends with .csv or .parquet).

'export_metadata.csv'
expand_metadata bool

If True, includes all DICOM attributes as columns.

False
patient_ids Iterable[str]

Restrict the export to these Patient IDs, read by get_cohort_report(), which this calls: None means every patient in the session, and an ID no patient holds is counted in a WARNING log line.

None

Returns:

Type Description
pd.DataFrame

The frame that was written.

Raises:

Type Description
ImportError

If pandas (or, for Parquet, a Parquet engine) is not installed.

TypeError

For a patient_ids get_cohort_report() refuses (a bare str, bytes-like, not iterable, or a non-str element), before the directory is created or any file is written.

Exception

Whatever the Parquet write raised, after logging it.

get_cohort_report(expand_metadata=False, patient_ids=None)

Return a pandas DataFrame of the cohort, one row per instance.

Parameters:

Name Type Description Default
expand_metadata bool

If True, add a column for every DICOM attribute.

False
patient_ids Iterable[str]

Restrict the report to these Patient IDs, read exactly as export() reads its patient_ids. None means every patient in the session. An empty iterable matches nobody: it is a filter that selected nothing, not an absent filter. An iterator is read once. An ID no patient holds selects nothing and is counted, never named, in one WARNING log line; a report is a read and writes no audit row. A subject whose files carried no Patient ID is selected by the key this report's PatientID column shows for it, not by "".

None

Returns:

Type Description
pd.DataFrame

One row per instance.

Raises:

Type Description
TypeError

If patient_ids is a bare str (wrap one ID in a list), bytes-like, not iterable, or holds an element that is not a str (named by its position and type, never its value).

phi_status_summary()

What the session currently knows about the PHI in each entity.

Counts PhiStatus per level, as each entity stands now rather than as the last scan left it: an entity edited since it was scanned counts as UNSCANNED.

Series are not counted: the scan records a status on patients, studies and instances only.

redact() is the one edit that keeps an instance's status: an instance REMEDIATED or CLEARED before the pass reads the same after it, provided nothing but redaction's own writes changed it. See PhiStatus.

A status is counted whatever policy it was recorded under; phi_status_policy names that policy, and an export() of statuses recorded under a policy other than the one in force says so.

Returns:

Type Description
Dict[str, Counter]

Keyed "patients", "studies", "instances"; each a Counter of PhiStatus to how many carry it.

generate_report(output_path, format='markdown')

Write the compliance report for the session's store.

The report holds the grade (PASS or REVIEW_REQUIRED), decided from the audit trail the store holds and the graph's PHI statuses ("How the grade is decided" in Analytics & Reporting), with the session's counts, the audit actions, data loss, exceptions, and the policy in force. Generate it after export(): an export writes rows of its own, and a report generated before any export carries a note saying so.

Parameters:

Name Type Description Default
output_path str

The file path where the report should be saved.

required
format str

'markdown', the one spelling accepted. Defaults to 'markdown'.

'markdown'

Raises:

Type Description
ValueError

For any other format, 'md' and case variants included; no file is written.

generate_manifest(output_path, format='html')

Write an HTML or JSON manifest of every instance in the session.

One entry per SOP Instance in the session, with its source file path and key metadata (Modality, Manufacturer, and so on).

Each JSON item's anonymized is True when the last tag-policy PHI scan left no identifier unremediated on that instance's patient, study or instance, and none of the three has been edited since. It does not mean "anonymize() ran" (a clean input reads True after audit() alone), and it says nothing about burned-in pixel text.

After anonymize() the UIDs listed are the replacements, beside each instance's source file path, so the manifest maps source files to exported UIDs: keep it with the store, not with an export.

Parameters:

Name Type Description Default
output_path str

The file path where the manifest should be saved.

required
format str

'html' or 'json', exactly. Defaults to 'html'.

'html'

Raises:

Type Description
ValueError

For any other format, a case variant included; no file is written.

save_analysis(report)

Persists the results of a PHI analysis to the database.

Parameters:

Name Type Description Default
report Union[PhiReport, List[PhiFinding]]

The PHI report object or list of findings to save.

required

compact()

Rewrite the pixel sidecar (_pixels.bin) to reclaim the space of frames no instance references, and point every loader at the new offsets.

An expensive I/O operation. It starts with save(sync=True), so it waits for a background save that is running.

Two behaviours are contract, observable from any thread of this session:

  1. It raises RuntimeError while a redact() or ingest() pass is open on the same store. The check comes first, before the leading save, so a refused call has done nothing.
  2. A redact() or ingest() that starts while it is saving or rewriting waits, bounded by 180 s, and then proceeds.

Every frame writer, and this method for the whole rewrite, holds a cross-process lock beside the sidecar, so a frame written while compaction runs lands in the compacted file. A writer that cannot take the lock within 180 s raises RuntimeError; a background save that expires this way is logged as Background save failed and its instances stay unsaved for the next save.

The rewrite holds the lock for its whole length, about 0.2 s per GB on local SSD. A close() whose persistence worker is queued behind a compaction longer than 30 s reports that worker as wedged.

Raises:

Type Description
RuntimeError

While a redact() or ingest() pass is open on this store; when a save is still queued on the persistence manager after the leading save; or when the sidecar lock cannot be taken within 180 s.

release_memory()

Release cached pixel and waveform data from every instance.

Each instance goes through Instance.unload_pixel_data(), with its precondition: an array replaced through set_pixel_data() and not since written is kept. An array mutated in place is not tracked, so it is dropped here, and the next get_pixel_data() returns the frame from before the mutation. Only an array a save has already written can be mutated in place: a frame read from a file or from the store is read-only.

Cached waveform samples are int16 of shape (num_samples, num_channels): about 80 KB for a 10-second 12-lead, about 104 MB for a 24-hour 3-channel Holter.

Its progress bar follows ISOCENTER_SHOW_PROGRESS.

close()

Shut the session down: the persistence-manager thread, the audit thread that owns the sqlite connection, and the process pool. A session that is never closed leaks its worker subprocesses for the life of the process.

All three steps run even if an earlier one raises. If more than one fails, the first failure is raised and the later ones are logged. A second call is safe; it repeats the shutdown messages and the unsaved-instances WARNING.

DICOM export options

export(folder, format="dicom", **options) takes the options below for the dicom format. Their names and defaults are frozen with export() (API stability). An option name the format does not take raises TypeError, and nothing is written. The table is read from the format's own method, which is private: pass the options to export(), never call that method directly.

Export the session as DICOM, one folder per patient, study and series.

Parameters:

Name Type Description Default
folder str

The output directory path.

required
use_compression bool

If True, compresses output images using JPEG 2000 (lossless). A 16-bit image with more than one sample is written this way too, and pydicom with only its Pillow plugin cannot decode it; the export names each such instance at INFO.

True
check_burned_in bool

If True, scans for PHI before exporting and withholds every instance that still carries an identifier, at any level of its hierarchy. Each withheld instance writes one WARNING audit row naming it and the level (patient, study, series or instance) that carried the identifier, never the value, so the report grades REVIEW_REQUIRED and lists them in section 4. Withheld instances count as requested and not written ("1 of 2 requested"), and the EXPORT row says how many were withheld. An export that withheld everything returns an empty summary and does not raise: nothing failed. An instance outside subset is not withheld; it was never asked for.

False
check_reversibility bool

If True (the default), warn when the files this export wrote still carry the encrypted originals that lock_identities() embeds, and record the disclosure in the audit log. The check runs after the write, against what reached disk. Those identities are recoverable by anyone holding the key, which a recipient of the cohort cannot see for themselves. Passing False silences the warning and skips the audit entry. The export itself is unchanged either way: this reports, it does not withhold.

True
patient_ids Iterable[str]

Limit export to specific Patient IDs. Only None, or the parameter omitted, means every patient: an empty list, tuple or set is a filter that selected nobody and nothing is written. An iterator is read once before the walk. A bare str (wrap one ID in a list), a bytes-like value, a non-iterable, or an element that is not a str is refused with TypeError before anything is flushed or written. An ID no patient in the session holds selects nothing and is counted, never named: one WARNING log line and one WARNING audit row, so the report grades REVIEW_REQUIRED, while the patients that do match are exported as asked. After anonymize() a patient is selected by its replacement ID. The wfdb format and get_cohort_report() read patient_ids the same way.

None
show_progress bool

If True, shows progress bar.

True
subset Union[str, pd.DataFrame, Iterable[str]]

Filter the export: a pandas query string run against get_cohort_report(expand_metadata=True), a DataFrame (read by the first of SOPInstanceUID, SeriesInstanceUID, StudyInstanceUID and PatientID it carries; one with none of them raises ValueError), or any other iterable of UIDs at any level (list, tuple, set, a generator), read as patient_ids is. Only None means no filter; an empty one selects nothing. A bytes-like value, a non-iterable, or an element that is not a str raises TypeError before anything is scanned, flushed or written. A value that names nothing in the session at any level (itself, the UID this store replaced it with, or, for a SOP Instance UID taken before redact(), the instance's redacted UID) is counted by position and never named: one WARNING log line and one WARNING audit row, so the report grades REVIEW_REQUIRED, while the rest is exported as asked. A query cannot name anything the session lacks, so it is never counted.

None
verify_readback bool

If True, each worker re-reads the file it just wrote before it is published under its real name, and holds it against what it meant to write: Rows, Columns, SamplesPerPixel, NumberOfFrames and BitsAllocated against the dataset it serialized; then the file's PhotometricInterpretation against the transfer syntax the file itself carries, which must admit it and be a single value; then every pixel sample, decoded the way ingest() decodes it and compared bit for bit with the samples written, after redaction; and a DICOM waveform's WaveformData bytes. The stored samples are compared, not a colour conversion of them; a file labelled YBR_FULL or YBR_FULL_422 over samples that are not unsigned 8-bit is also decoded with pydicom's colour conversion, as ingest() decodes it, and fails when that decode raises, which it does for 16-bit and int8 samples. A value outside the declared BitsStored fails an uncompressed file, because every conformant reader masks it (-3024 at BitsStored 12 reads as 1072).

Passing True can cost a file the default export delivers. By default a Photometric Interpretation the written syntax does not admit (YBR_ICT or YBR_RCT on an uncompressed file, YBR_PARTIAL_422/_420 on any this exporter writes) is written exactly as the instance declared it, with a WARNING audit row and a REVIEW_REQUIRED grade. With True that instance fails, gets an ERROR row, and no file for it reaches the output folder; the reason names the label, the syntax and a remedy. The check does not ask whether the samples are really in the colour space the label names: RGB over YBR samples passes, and so does a file under a transfer syntax the check has no row for.

An unreadable or undecodable file, or any mismatch, fails that instance's export: it is counted out of "Instances Written", gets an ERROR audit row and takes the grade to REVIEW_REQUIRED; when every instance fails the call raises ExportError. Off by default because it costs a second parse and a full decode per instance: about twice the time of a default (JPEG 2000) export, and little more for an uncompressed one. Each worker holds one more decoded array while it checks.

False

Returns:

Type Description
ExportSummary

What reached disk and what did not; empty when nothing was attempted.

Raises:

Type Description
TypeError

patient_ids or subset is of a type the selection does not take, before anything is flushed or written.

ValueError

subset is a DataFrame carrying none of SOPInstanceUID, SeriesInstanceUID, StudyInstanceUID and PatientID, or a query string that does not run against the cohort report.

ExportError

Every planned instance failed (nothing written and at least one failure), raised last, after the audit rows and the EXPORT row are written. A partial export and an empty plan return a summary instead.