Skip to content

fix: encode user-controlled input in request query strings (batch 1/#574) - #602

Merged
kaifcodec merged 1 commit into
kaifcodec:mainfrom
A-S-Manoj:fix-unescaped-input
Aug 24, 2026
Merged

fix: encode user-controlled input in request query strings (batch 1/#574)#602
kaifcodec merged 1 commit into
kaifcodec:mainfrom
A-S-Manoj:fix-unescaped-input

Conversation

@A-S-Manoj

@A-S-Manoj A-S-Manoj commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Related to #574 (batch 1)

Problem
Several scan modules build request URLs by interpolating user-controlled values (user, username, email, target) directly into an f-string query parameter, without percent-encoding:

# user_scanner/core/hudson.py
url = f"{base_url}{endpoint}?{param}={target}"

# user_scanner/user_scan/social/instagram.py
api_url = f"https://www.instagram.com/api/v1/users/web_profile_info/?username={user}"

If the input contains characters such as &, #, %, +, whitespace, or non-ASCII characters (all valid in usernames/emails), the resulting request can be corrupted — truncated queries, unintended extra parameters, or requests silently hitting the wrong data.

Fix
Per the pattern discussed on #574, switched these to httpx's params= argument (threaded through generic_validate/make_request/status_validate/impersonate_request), which encodes correctly and is cleaner than wrapping every call site in urllib.parse.quote().

Changes
This is batch 1 of 15 affected files:

  • user_scanner/core/hudson.py
  • user_scanner/email_scan/crm/axonaut.py
  • user_scanner/email_scan/learning/duolingo.py
  • user_scanner/user_scan/shopping/vinted.py
  • user_scanner/user_scan/community/wikipedia.py
  • user_scanner/user_scan/community/hackernews.py
  • user_scanner/user_scan/community/lemmy.py
  • user_scanner/user_scan/community/archwiki.py
  • user_scanner/user_scan/community/fandom.py
  • user_scanner/user_scan/community/thefirearmsforum.py
  • user_scanner/user_scan/community/defensivecarry.py
  • user_scanner/user_scan/social/threads.py
  • user_scanner/user_scan/social/pr0gramm.py
  • user_scanner/user_scan/social/minds.py
  • tests/test_hudson.py — updated the mock client to accept/record params, and added a regression test asserting the target is passed via params rather than baked into the URL

Remaining ~24 affected files (including instagram.py, gitlab.py, mastodon.py, and others across dev/, social/, learning/, creator/, etc.) will follow in subsequent batches.

Testing
Verified each modified module's actual outgoing request (mocking the underlying transport) to confirm the target is passed correctly via params rather than the raw URL, using inputs like user+tag@example.com to prove special characters no longer get mangled. Full suite: 375 passed, 3 skipped (pre-existing, unrelated) against current main. ruff check . and mypy user_scanner both clean.

@kaifcodec kaifcodec left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@A-S-Manoj I've tested the modules and they work perfectly. MERGING IT!

However, when you open the PR for batch 2, please update the CONTRIBUTING.md guidelines to mention this new approach. This will ensure future contributors follow the same standard for URL encoding.

@kaifcodec kaifcodec added bug fix Fixed a bug in existing files cleaning Cleaning the codebase, abandon modules, refactoring labels Aug 24, 2026
@kaifcodec
kaifcodec merged commit c7d9415 into kaifcodec:main Aug 24, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug fix Fixed a bug in existing files cleaning Cleaning the codebase, abandon modules, refactoring

2 participants