The dataset files are used for conducting end-to-end (E2E) testing for fine-tuning. The Dockerfile builds an image incorporating the dolly-15k-oai-style and kubernetes-reformatted-remove-outliers datasets, which are then used within init containers specifically for fine-tuning.

• Dockerfile: Builds the Docker image for the E2E tests.

• dataset.parquet: The datasets themselves, downloaded from dolly-15k-oai-style and kubernetes-reformatted-remove-outliers

Build the Docker images with the following command:

make docker-build-dataset