Skip to content

Commit 8612e9f

Browse files
authored
[Docs] ggml: add README for the embedding example with CI job (second-state#114)
Signed-off-by: dm4 <dm4@secondstate.io>
1 parent 059bbd0 commit 8612e9f

File tree

2 files changed

+53
-5
lines changed

2 files changed

+53
-5
lines changed

‎.github/workflows/llama.yml‎

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,18 @@ jobs:
132132
target/wasm32-wasi/release/wasmedge-ggml-multimodel.wasm \
133133
'describe this picture please'
134134
135+
- name: Embedding Example
136+
run: |
137+
test -f ~/.wasmedge/env && source ~/.wasmedge/env
138+
cd wasmedge-ggml/embedding
139+
curl -LO https://huggingface.co/second-state/All-MiniLM-L6-v2-Embedding-GGUF/resolve/main/all-MiniLM-L6-v2-ggml-model-f16.gguf
140+
cargo build --target wasm32-wasi --release
141+
time wasmedge --dir .:. \
142+
--nn-preload default:GGML:AUTO:all-MiniLM-L6-v2-ggml-model-f16.gguf \
143+
target/wasm32-wasi/release/wasmedge-ggml-llama-embedding.wasm \
144+
default \
145+
'hello world'
146+
135147
- name: Build llama-stream
136148
run: |
137149
cd wasmedge-ggml/llama-stream
@@ -142,11 +154,6 @@ jobs:
142154
cd wasmedge-ggml/chatml
143155
cargo build --target wasm32-wasi --release
144156
145-
- name: Build embedding
146-
run: |
147-
cd wasmedge-ggml/embedding
148-
cargo build --target wasm32-wasi --release
149-
150157
- name: Build llava-base64-stream
151158
run: |
152159
cd wasmedge-ggml/llava-base64-stream

‎wasmedge-ggml/embedding/README.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Embedding Example For WASI-NN with GGML Backend
2+
3+
> [!NOTE]
4+
> Please refer to the [wasmedge-ggml/README.md](../README.md) for the general introduction and the setup of the WASI-NN plugin with GGML backend. This document will focus on the specific example of generating embeddings.
5+
6+
## Get the Model
7+
8+
In this example, we are going to use the pre-converted `all-MiniLM-L6-v2` model.
9+
10+
Download the model:
11+
12+
```bash
13+
curl -LO https://huggingface.co/second-state/All-MiniLM-L6-v2-Embedding-GGUF/resolve/main/all-MiniLM-L6-v2-ggml-model-f16.gguf
14+
```
15+
16+
## Parameters
17+
18+
> [!NOTE]
19+
> Please check the parameters section of [wasmedge-ggml/README.md](https://github.com/second-state/WasmEdge-WASINN-examples/tree/master/wasmedge-ggml#parameters) first.
20+
21+
## Execute
22+
23+
Execute the WASM with the `wasmedge` using the named model feature to preload a large model:
24+
25+
```console
26+
$ wasmedge --dir .:. \
27+
--nn-preload default:GGML:AUTO:all-MiniLM-L6-v2-ggml-model-f16.gguf \
28+
wasmedge-ggml-llama-embedding.wasm default
29+
30+
Prompt:
31+
What's the capital of the United States?
32+
Raw Embedding Output: {"n_embedding": 384, "embedding": [0.5426152349,-0.03840282559,-0.03644151986,0.3677068651,-0.115977712...(omitted)...,-0.003531290218]}
33+
Interact with Embedding:
34+
N_Embd: 384
35+
Show the first 5 elements:
36+
embd[0] = 0.5426152349
37+
embd[1] = -0.03840282559
38+
embd[2] = -0.03644151986
39+
embd[3] = 0.3677068651
40+
embd[4] = -0.115977712
41+
```

0 commit comments

Comments
 (0)