Skip to content

Port tests/decode.js to Python - #24

Open
cjchanh wants to merge 1 commit into
executablebooks:masterfrom
cjchanh:port-decode-js-tests
Open

Port tests/decode.js to Python#24
cjchanh wants to merge 1 commit into
executablebooks:masterfrom
cjchanh:port-decode-js-tests

Conversation

@cjchanh

@cjchanh cjchanh commented Aug 19, 2026

Copy link
Copy Markdown

Port tests/decode.js to Python

Closes #2 — ports the last remaining JS test fixture in the repo into
tests/test_decode.py as native pytest tests. No library code changes.

What was ported

  • String-level decode behavior%xx decoding, passthrough of invalid
    sequences (%2g%z1%%), and the three reserved-set variants.
  • The full 36-case binary UTF-8 sample matrix — the SAMPLES dict is
    ported verbatim from the JS fixture (grouping comments included), and
    parametrized as test_decode_utf8_sample_matrix, with each binary sample
    as the test id, matching the JS it(k) names. A _percent_encode helper
    mirrors the JS encodeBinary helper.
  • The two pre-existing Python tests (test_decode_multi_byte,
    test_decode_invalid_utf8) are kept unchanged — the diff is purely
    additive. The latter also preserves the known 3 × � vs
    urllib.parse.unquote (2 × �) replacement-count difference.
  • tests/decode.js is left in place as a reference (can be dropped in a
    follow-up if desired).

reservedSet semantics

mdurl.decode(string, exclude=...)exclude is the JS reservedSet
(default ;/?:@&=+$,#). Percent-escapes of characters in exclude are left
untouched, while everything else is decoded. The ported tests exercise this
with exclude="%", " ", and " %".

The binary UTF-8 matrix

The 36 samples cover invalid leading/continuation bytes, truncated
sequences, overlong ("impossible") encodings, surrogate-range encodings, and
the U+10FFFF boundary. As in the JS test, the True/False marks are
documentation only — upstream iterates Object.keys(samples) and never
reads them, and the port follows suit. Each sample's validity is decided
empirically by an oracle: decodeURIComponent throws URIError on malformed
UTF-8, so the port uses
urllib.parse.unquote_to_bytes(...).decode("utf-8", "strict") (raising
UnicodeDecodeError) as its equivalent. Samples the oracle rejects must
contain in the decode result; samples it accepts must match
decode exactly (and contain no ).

Test commands

python -m venv .venv
. .venv/bin/activate
pip install . -r tests/requirements.txt
pytest                        # full suite
pytest tests/test_decode.py   # this port only

This mirrors the CI setup (pip install . -r tests/requirements.txt,
pytest --cov --cov-fail-under=75); tests/requirements.txt includes
pytest-randomly, so the suite is order-independent. The pre-commit hooks
(isort, black, flake8, mypy, docformatter, …) all pass.

Add the full contents of tests/decode.js to tests/test_decode.py:
%xx decoding, invalid-sequence passthrough, the reserved-set variants,
and the complete 36-case binary UTF-8 sample matrix (validity judged
by a strict UTF-8 oracle that mirrors decodeURIComponent). Keep the
pre-existing decode tests unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant