Fix: skip empty files in next_request instead of terminating the stre…

…am (#158)

FilesRequestsIterator.next_request treated any empty next_chunks() result
as the end of the current request; when that produced an empty request,
iteration ended and every remaining queued file was silently dropped.
A safetensors shard with zero tensors at the head of a fresh request
therefore caused all subsequent shards to never be read.

Distinguish the two cases: a finished file with no data is skipped
(continue), while an unfinished file whose next chunk exceeds the
remaining buffer budget still closes the request (break), preserving
the existing memory-cap semantics.

Fixes #157

feat: add SAS token authentication for Azure Blob Storage (#148)

s3 public buckets (#144)

Support public bucket with new environment variable RUNAI_STREAMER_S3_UNSIGNED

Share Azure default credentials across clients (#134)

Before this change, a new AzureDefaultCredential was created with
every new AzureClient. This becomes problematic for distributed
loading scenarios where you can have N processes x M worker threads
per process each with their own credential instance. Specifically,
this can lead to overwhelming IMDS when when all of these clients
attempt to retrieve a token, resulting fataling throttling errors
during model loading.

AzureDefaultCredentials are thread safe and are meant to be used
across threads. By statically creating a single AzureDefaultCredential
per process for AzureClients to share, the credential instance will
automatically guard against swarming IMDS per instance so you
only end up with N concurrent IMDS token calls and greatly reduce the
chance of being throttled by IMDS.

It was also considered to share Blob Service clients as well across
threads since they are thread safe as well. It was decided against
as it would have required a larger change to be able to handle
differences in client options. It may be a change we should consider
in the future though to even further improve resource sharing
(e.g. reusing HTTP connection pools)

Safe Context Manager for Pulling Files by Multiple Processes (#133)

Extend supported types to be safetensors compatible (#127)

Added support in F8_E8M0 and forward integration with torch for F4

Sf validations (#121)

* validate safetensors file

* tests

* CR fixes

* CR fixes

* CR fixes

* CR fixes

Fix list_safetensors

Making list_safetensors to be non recursive for gcs and s3 should solve #117
Supporting s3 compatible storage - setting uri config to be path style for boto3 client if RUNAI_STREAMER_S3_USE_VIRTUAL_ADDRESSING equals "0"

Fix mock list sf (#103)

* mock list_safetensors to return mock paths

* debug logs

add files_to_tensors_metadata (#102)