create_dataset_from_directory
Scan folders and register what they hold as one dataset.
from plexora import create_dataset_from_directorycreate_dataset_from_directory(
name,
*roots,
overrides=None,
description='',
exist_ok=False,
node=None,
) -> Datasetplan_dataset, then create_dataset with the projects it planned: every
image becomes a project, with the mask, table and channel names matched
to it. Files are registered where they are, not copied. Files the plan
could not place (unassigned) are left out; pass overrides to place
them, or use plan_dataset first to see them. In a bucket (s3://,
gs://, az://) images and masks stream from where they are, and each
cell table is left there and named in the dataset's meta["tables"], to
be added from its sample when it is wanted; the folder each sample was
filed under is meta["groups"]. On a files-only server
(sftp://) images and masks stream from where they are and each cell
table is copied to this computer, since a CSV cannot be streamed.
When to use
Use this for a cohort folder whose files are named consistently --
registration/, segmentation/, quantification/, or a folder per
sample. Use create_dataset with explicit specs when the folder
layout does not say which file is which.
Parameters
namestrrequiredThe dataset's name.
*rootsOne or more folders to scan.
overridesdictkeyword-onlydefault NoneCorrections -- see plan_dataset.
descriptionstrkeyword-onlydefault ''A note about the dataset.
exist_okboolkeyword-onlydefault FalseAdd to an existing dataset of this name instead of raising.
nodestrkeyword-onlydefault NoneThe data node the folders are on.
Returns
DatasetThe dataset, holding every project registered or adopted.
Raises
ValueErrorIf a root is not a folder, or nothing to register was found.
DatasetCreateErrorIf registering one of the projects fails; what registered before it is kept.
Examples
import plexora
cohort = plexora.create_dataset_from_directory("Melanoma", "/data/cohort")
print(list(cohort))Related
plan_dataset
Work out which images, masks and tables under these folders belong together, without registering anything.
create_dataset
Make a dataset, registering its projects if they are not there yet.