Skip to content

feat: add Nexus Mods username module - #481

Merged
kaifcodec merged 3 commits into
kaifcodec:mainfrom
kristoisberg:feature/nexusmods-username-module
Jul 30, 2026
Merged

feat: add Nexus Mods username module#481
kaifcodec merged 3 commits into
kaifcodec:mainfrom
kristoisberg:feature/nexusmods-username-module

Conversation

@kristoisberg

Copy link
Copy Markdown
Collaborator

This PR adds an username module for the gaming site nexusmods.com. One of the simpler modules yet, since the site exposes E2E testing attributes for nearly everything of interest.

Comment thread user_scanner/user_scan/gaming/nexusmods.py Outdated
@kaifcodec kaifcodec added the user module addition PRs or issues related to new user_scan module additions label Jul 29, 2026

@brunolm brunolm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. generic_validate (httpx) is Cloudflare-blocked — every request returns 403 (Just a moment...), so the module always errors and never returns taken/available. Live check: same URL via impersonate_request(..., allow_redirects=True) returns 200 for real profiles (Gopher, Pickysaurus, Dark0ne) and 404 for fakes. Switch to impersonate_validate and use allow_redirects=True (curl_cffi kwarg, not follow_redirects). Homepage warmup_url is optional; fingerprint alone was enough here.
  2. joined / last_active scrape display text (07 Sept 2004) while the same <time> elements expose ISO dateTime (e.g. 2004-09-07T10:55:16.000Z). Prefer the dateTime attribute for locale-independent, machine-sortable values.
  3. Also extract data-e2eid="unique-download-count" (present on live profiles, e.g. 9.4m) into extra, same pattern as the other e2e fields.
  4. Avatar URLs are https://avatars.nexusmods.com/<user_id>/100 — optionally emit the numeric id into extra (and/or prefer og:image for the avatar).
Raw output:

 Checking username: brunolm
  [!] Nexusmods [https://www.nexusmods.com/profile/brunolm] (brunolm): Error (Unexpected status code: 403)


[i] Scan complete.
  Total hits: 0

Matches the PR review: httpx hits Cloudflare (403), so the module never gets a profile response. No JSON file was written (nothing to save with only an error and no -o).

@kristoisberg

Copy link
Copy Markdown
Collaborator Author

@brunolm Thank you for the suggestions, I applied them all. I also parsed some additional information from the page's hydration data.

@kaifcodec
kaifcodec requested a review from brunolm July 30, 2026 13:39

@kaifcodec kaifcodec left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kristoisberg This is really a high quality module as it reveals the targets' email.
@brunolm Thanks for review on this.

MERGING IT!

@kaifcodec
kaifcodec merged commit 2d67e1f into kaifcodec:main Jul 30, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

user module addition PRs or issues related to new user_scan module additions

3 participants