feat(features): add BioSequence and BioStructure decodable features - #8542
Draft
behroozazarkhalili wants to merge 2 commits into
Draft
feat(features): add BioSequence and BioStructure decodable features#8542behroozazarkhalili wants to merge 2 commits into
behroozazarkhalili wants to merge 2 commits into
Conversation
Adds two feature types that store a biological file as bytes and decode it to
the biopython object the format describes, following the contract Audio, Image,
Pdf and Nifti already use: storage is struct<bytes, path>, decode=False returns
that struct unchanged, and decoding raises ImportError when the optional
dependency is missing.
BioSequence -> Bio.SeqRecord.SeqRecord FASTA, FASTQ, GenBank
BioStructure -> Bio.PDB.Structure.Structure PDB, mmCIF
Two types rather than one or five. One general type would return a different
class depending on which file it was handed, which pushes a type check onto
every caller. Five would be one class per file format, but FASTA, FASTQ and
GenBank decode to the same object and differ only in the parser, so the format
is a field rather than a type. The split follows the boundary biopython itself
draws between Bio.SeqIO and Bio.PDB, which is also where the optional imports
divide.
biopython stays optional. It is declared as the `bio` extra and probed through
config.BIOPYTHON_AVAILABLE, which checks the import name `Bio` rather than the
distribution name. Users who never touch biological data pay nothing: the
availability flag is a find_spec call, and the lazy isinstance dispatch in
_cast_to_python_objects is additionally guarded on `"Bio" in sys.modules` so
having the package installed never forces the import.
Both types register where every other feature does, so a dataset carrying one
survives a round trip through dataset_info.json: the FeatureType union, the
_FEATURE_TYPES registry, and the object dispatch that encodes a live SeqRecord
or Structure passed straight to Dataset.from_dict.
Tests cover storage type, encoding from path, bytes and dict, the decode=False
path, flatten, cast_storage, and the Features.to_dict round trip for both types
without biopython present, plus decoding to SeqRecord and Structure when it is.
11 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds two decodable feature types that store a biological file as bytes and decode it to the biopython object the format describes.
decode_examplereturnsBioSequencefeatures/bio_sequence.pyBio.SeqRecord.SeqRecordBioStructurefeatures/bio_structure.pyBio.PDB.Structure.StructureBoth follow the contract
Audio,Image,PdfandNiftialready use: storage isstruct<bytes, path>,decode=Falsereturns that struct unchanged, and decoding raisesImportErrorwhen biopython is absent.Why two types
One general type would return a different class depending on which file it was handed, which pushes an isinstance check onto every caller. Five would be one class per file format, but FASTA, FASTQ and GenBank all decode to
SeqRecordand differ only in the parser, so the format is a field rather than a type. The split follows the boundary biopython itself draws betweenBio.SeqIOandBio.PDB, which is also where the optional imports divide.Naming agreed with @lhoestq in #7923 (comment).
biopython stays optional
Declared as the
bioextra and probed throughconfig.BIOPYTHON_AVAILABLE, which checks the import nameBiorather than the distribution namebiopython. Users who never touch biological data pay nothing: the availability flag is afind_speccall, and the lazy isinstance dispatch in_cast_to_python_objectsis additionally guarded on"Bio" in sys.modules, so having the package installed never forces the import.Registration
Registered everywhere the other decodable features are, so a dataset carrying one survives a round trip through
dataset_info.json: theFeatureTypeunion, the_FEATURE_TYPESregistry,features/__init__.py, the object dispatch that encodes a liveSeqRecordorStructurepassed straight toDataset.from_dict, and the docs undermain_classes.mdx.Verification
tests/features/test_bio.py, covering storage type, encoding from path, bytes and dict, thedecode=Falsepath,flatten,cast_storage, and theFeatures.to_dictround trip for both types without biopython present, plus decoding toSeqRecordandStructurewhen it is.ImportError: cannot import name 'BioSequence') before making them pass.tests/features/test_bio.py+tests/features/test_features.py: 222 passed, 3 skipped, no regressions.ruff checkandruff formatclean.This is the foundation for the five bio-format loaders; wiring each loader to these types follows in their own PRs.