feat: add standby URL, change default standby port

[jirimoravcik]

This PR introduces a new environment variable with standby URL, it also changes the default port for the standby port env var.

[janbuchar]

Cool. What's the relationship between this and web_server_url or container_url?

[jirimoravcik]

Cool. What's the relationship between this and web_server_url or container_url?

web_server_url == container_url, these are unique for each run (they look something like https://someidhere.runs.apify.net/). Standby URL is the nice generic, e.g. https://super-scraper-api.apify.actor, which does automatic load balancing and scaling.

[janbuchar]

Cool. What's the relationship between this and web_server_url or container_url?

web_server_url == container_url, these are unique for each run (they look something like https://someidhere.runs.apify.net/). Standby URL is the nice generic, e.g. https://super-scraper-api.apify.actor, which does automatic load balancing and scaling.

Understood, thanks. Could you mention the different in the field description please? And maybe update https://docs.apify.com/platform/actors/development/programming-interface/environment-variables as well?

[jirimoravcik]

Cool. What's the relationship between this and web_server_url or container_url?

web_server_url == container_url, these are unique for each run (they look something like https://someidhere.runs.apify.net/). Standby URL is the nice generic, e.g. https://super-scraper-api.apify.actor, which does automatic load balancing and scaling.

Understood, thanks. Could you mention the different in the field description please? And maybe update docs.apify.com/platform/actors/development/programming-interface/environment-variables as well?

I tried to improve the descriptions here.

I'm already preparing a PR for docs, but I want to mention the configs in SDKs, so that's why I'm doing it first.

[fnesveda]

Cool, just one small note

[vdusek]

In Crawlee, we use a custom URL validator function:

    url: Annotated[str, BeforeValidator(validate_http_url), Field(...)]

https://github.com/apify/crawlee-python/blob/v0.3.8/src/crawlee/_request.py#L112

@jirimoravcik Could we use the same approach here as well?

[jirimoravcik]

sure