Skip to content

Commit 3300797

Browse files
authored
[Example] ggml: Update llama README for llama3 support (second-state#132)
Signed-off-by: hydai <z54981220@gmail.com>
1 parent e8b8300 commit 3300797

File tree

2 files changed

+82
-2
lines changed

2 files changed

+82
-2
lines changed

‎wasmedge-ggml/llama-stream/README.md‎

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,46 @@
11
# `llama-stream`
22

3-
## Execute
3+
## Execute - llama 3
4+
5+
### Model Download Link
6+
7+
```console
8+
wget https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF/blob/main/Meta-Llama-3-8B-Instruct.Q5_K_M.gguf
9+
```
10+
11+
### Execution Command
12+
13+
Please make sure you have the `Meta-Llama-3-8B-Instruct.Q5_K_M.gguf` file in the current directory.
14+
Don't forget to set the `llama3` environment variable to `true` to enable the llama3 prompt template.
15+
If you want to enable GPU support, please set the `n_gpu_layers` environment variable.
16+
You can also change the `ctx_size` to have a larger context window via `--env ctx_size=8192`. The default value is 1024.
17+
18+
```console
19+
$ wasmedge --dir .:. \
20+
--env llama3=true \
21+
--env n_gpu_layers=100 \
22+
--nn-preload default:GGML:AUTO:Meta-Llama-3-8B-Instruct.Q5_K_M.gguf \
23+
wasmedge-ggml-llama-stream.wasm default
24+
25+
USER:
26+
What's WasmEdge?
27+
ASSISTANT:
28+
WasmEdge is an open-source WebAssembly runtime and compiler that can run WebAssembly code in various environments, including web browsers, mobile devices, and server-side applications.
29+
USER:
30+
Does it support in Docker?
31+
ASSISTANT:
32+
Yes, WasmEdge supports running in Docker containers.
33+
USER:
34+
Does it support in Podman?
35+
ASSISTANT:
36+
Yes, WasmEdge also supports running in Podman containers.
37+
USER:
38+
Does it work with crun?
39+
ASSISTANT:
40+
Yes, WasmEdge supports running in crun containers.
41+
```
42+
43+
## Execute - llama 2
444

545
```console
646
$ wasmedge --dir .:. \

‎wasmedge-ggml/llama/README.md‎

Lines changed: 41 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,46 @@
11
# `llama`
22

3-
## Execute
3+
## Execute - llama 3
4+
5+
### Model Download Link
6+
7+
```console
8+
wget https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF/blob/main/Meta-Llama-3-8B-Instruct.Q5_K_M.gguf
9+
```
10+
11+
### Execution Command
12+
13+
Please make sure you have the `Meta-Llama-3-8B-Instruct.Q5_K_M.gguf` file in the current directory.
14+
Don't forget to set the `llama3` environment variable to `true` to enable the llama3 prompt template.
15+
If you want to enable GPU support, please set the `n_gpu_layers` environment variable.
16+
You can also change the `ctx_size` to have a larger context window via `--env ctx_size=8192`. The default value is 1024.
17+
18+
```console
19+
$ wasmedge --dir .:. \
20+
--env llama3=true \
21+
--env n_gpu_layers=100 \
22+
--nn-preload default:GGML:AUTO:Meta-Llama-3-8B-Instruct.Q5_K_M.gguf \
23+
wasmedge-ggml-llama.wasm default
24+
25+
USER:
26+
What's WasmEdge?
27+
ASSISTANT:
28+
WasmEdge is an open-source WebAssembly runtime and compiler that can run WebAssembly code in various environments, including web browsers, mobile devices, and server-side applications.
29+
USER:
30+
Does it support in Docker?
31+
ASSISTANT:
32+
Yes, WasmEdge supports running in Docker containers.
33+
USER:
34+
Does it support in Podman?
35+
ASSISTANT:
36+
Yes, WasmEdge also supports running in Podman containers.
37+
USER:
38+
Does it work with crun?
39+
ASSISTANT:
40+
Yes, WasmEdge supports running in crun containers.
41+
```
42+
43+
## Execute - llama 2
444

545
```console
646
$ wasmedge --dir .:. \

0 commit comments

Comments
 (0)