[ML] Adding timeout to request for creating inference endpoint

[jonathan-buttner]

This PR also adds the timeout query parameter to the PUT request so users can specify a longer timeout than the default 30 seconds. 30 seconds was used previously based on the ack timeout for the master node.

The timeout only applies to the start deployment request and the inference request during validation. The model download request ignores the timeout because it waits for the model to finish downloading before responding on the listener.

Testing

PUT http://localhost:9200/_inference/sparse_embedding/elser2?timeout=2nanos
{
    "service": "elasticsearch",
    "service_settings": {
        "model_id": ".elser_model_2",
        "num_threads": 1,
        "adaptive_allocations": {
            "enabled": true,
            "min_number_of_allocations": 1,
            "max_number_of_allocations": 4
        }
    }
}

This request should result in a failure like the following

{
    "error": {
        "root_cause": [
            {
                "type": "status_exception",
                "reason": "Timed out after [2nanos] waiting for model deployment to start. Use the trained model stats API to track the state of the deployment."
            }
        ],
        "type": "status_exception",
        "reason": "Timed out after [2nanos] waiting for model deployment to start. Use the trained model stats API to track the state of the deployment."
    },
    "status": 408
}

[elasticsearchmachine]

Hi @jonathan-buttner, I've created a changelog YAML for you.

[davidkyle]

LGTM

[elasticsearchmachine]

Pinging @elastic/ml-core (Team:ML)

[elasticsearchmachine]

💔 Backport failed

Status	Branch	Result
❌	8.19	Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 126805

[jonathan-buttner]

💚 All backports created successfully

Status	Branch	Result
✅	8.19

Questions ?

Please refer to the Backport tool documentation