Skip to content

feat(features): add BioSequence and BioStructure decodable features - #8542

Draft
behroozazarkhalili wants to merge 2 commits into
huggingface:mainfrom
behroozazarkhalili:feat/bio-feature-types
Draft

feat(features): add BioSequence and BioStructure decodable features#8542
behroozazarkhalili wants to merge 2 commits into
huggingface:mainfrom
behroozazarkhalili:feat/bio-feature-types

Conversation

@behroozazarkhalili

Copy link
Copy Markdown

Adds two decodable feature types that store a biological file as bytes and decode it to the biopython object the format describes.

Type Module decode_example returns Serves
BioSequence features/bio_sequence.py Bio.SeqRecord.SeqRecord FASTA #7923, FASTQ #7924, GenBank #7951
BioStructure features/bio_structure.py Bio.PDB.Structure.Structure mmCIF #7925, PDB #7926

Both follow the contract Audio, Image, Pdf and Nifti already use: storage is struct<bytes, path>, decode=False returns that struct unchanged, and decoding raises ImportError when biopython is absent.

Why two types

One general type would return a different class depending on which file it was handed, which pushes an isinstance check onto every caller. Five would be one class per file format, but FASTA, FASTQ and GenBank all decode to SeqRecord and differ only in the parser, so the format is a field rather than a type. The split follows the boundary biopython itself draws between Bio.SeqIO and Bio.PDB, which is also where the optional imports divide.

Naming agreed with @lhoestq in #7923 (comment).

biopython stays optional

Declared as the bio extra and probed through config.BIOPYTHON_AVAILABLE, which checks the import name Bio rather than the distribution name biopython. Users who never touch biological data pay nothing: the availability flag is a find_spec call, and the lazy isinstance dispatch in _cast_to_python_objects is additionally guarded on "Bio" in sys.modules, so having the package installed never forces the import.

Registration

Registered everywhere the other decodable features are, so a dataset carrying one survives a round trip through dataset_info.json: the FeatureType union, the _FEATURE_TYPES registry, features/__init__.py, the object dispatch that encodes a live SeqRecord or Structure passed straight to Dataset.from_dict, and the docs under main_classes.mdx.

Verification

  • 22 tests in tests/features/test_bio.py, covering storage type, encoding from path, bytes and dict, the decode=False path, flatten, cast_storage, and the Features.to_dict round trip for both types without biopython present, plus decoding to SeqRecord and Structure when it is.
  • Confirmed the tests fail without the implementation (ImportError: cannot import name 'BioSequence') before making them pass.
  • tests/features/test_bio.py + tests/features/test_features.py: 222 passed, 3 skipped, no regressions.
  • ruff check and ruff format clean.

This is the foundation for the five bio-format loaders; wiring each loader to these types follows in their own PRs.

Adds two feature types that store a biological file as bytes and decode it to
the biopython object the format describes, following the contract Audio, Image,
Pdf and Nifti already use: storage is struct<bytes, path>, decode=False returns
that struct unchanged, and decoding raises ImportError when the optional
dependency is missing.

    BioSequence  -> Bio.SeqRecord.SeqRecord        FASTA, FASTQ, GenBank
    BioStructure -> Bio.PDB.Structure.Structure    PDB, mmCIF

Two types rather than one or five. One general type would return a different
class depending on which file it was handed, which pushes a type check onto
every caller. Five would be one class per file format, but FASTA, FASTQ and
GenBank decode to the same object and differ only in the parser, so the format
is a field rather than a type. The split follows the boundary biopython itself
draws between Bio.SeqIO and Bio.PDB, which is also where the optional imports
divide.

biopython stays optional. It is declared as the `bio` extra and probed through
config.BIOPYTHON_AVAILABLE, which checks the import name `Bio` rather than the
distribution name. Users who never touch biological data pay nothing: the
availability flag is a find_spec call, and the lazy isinstance dispatch in
_cast_to_python_objects is additionally guarded on `"Bio" in sys.modules` so
having the package installed never forces the import.

Both types register where every other feature does, so a dataset carrying one
survives a round trip through dataset_info.json: the FeatureType union, the
_FEATURE_TYPES registry, and the object dispatch that encodes a live SeqRecord
or Structure passed straight to Dataset.from_dict.

Tests cover storage type, encoding from path, bytes and dict, the decode=False
path, flatten, cast_storage, and the Features.to_dict round trip for both types
without biopython present, plus decoding to SeqRecord and Structure when it is.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant