Skip to content

Commit c828a39

Browse files
authored
[Docs] ggml: add README for the llava-1.6 execution command (#117)
Signed-off-by: dm4 <dm4@secondstate.io>
1 parent 8612e9f commit c828a39

File tree

1 file changed

+30
-5
lines changed

1 file changed

+30
-5
lines changed

‎wasmedge-ggml/llava/README.md‎

Lines changed: 30 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ In llava inference, we recommend to use the `ctx-size` at least `2048` when usin
4646
options.insert("ctx-size", Value::from(4096));
4747
```
4848

49-
## Execute
49+
## Execute (llava-1.5)
5050

5151
Execute the WASM with the `wasmedge` using the named model feature to preload a large model:
5252

@@ -60,12 +60,37 @@ $ wasmedge --dir .:. \
6060
--nn-preload default:GGML:AUTO:ggml-model-q5_k.gguf \
6161
wasmedge-ggml-llava.wasm default
6262

63-
Question:
63+
USER:
6464
what is in this picture?
65-
Answer:
65+
ASSISTANT:
6666
The image features a painting or portrait of a woman with long hair, likely inspired by the famous Mona Lisa artwork. She appears to be smiling and is surrounded by an ocean view, giving the impression of being on a boat. The scene is painted in black and white, creating a classic and timeless look. Additionally, there are two birds present within this painting, one on the left side towards the top and another slightly higher up in the center.
67-
Question:
67+
USER:
6868
Do you know who drew this painting?
69-
Answer:
69+
ASSISTANT:
7070
As a visual AI, I don't have knowledge about the artist who created this painting. However, it is inspired by the famous Mona Lisa artwork, which was painted by Leonardo da Vinci.
7171
```
72+
73+
## Execute (llava-1.6)
74+
75+
Execute the WASM with the `wasmedge` using the named model feature to preload a large model:
76+
77+
> [!NOTE]
78+
> For the llava-1.6 model, we will need to set the context size to at least 4096 to work.
79+
80+
```console
81+
$ wasmedge --dir .:. \
82+
--env mmproj=mmproj-vicuna7b-f16.gguf \
83+
--env image=monalisa.jpg \
84+
--env ctx_size=4096 \
85+
--nn-preload default:GGML:AUTO:vicuna-7b-q5_k.gguf \
86+
wasmedge-ggml-llava.wasm default
87+
88+
USER:
89+
what is in this picture?
90+
ASSISTANT:
91+
The image you've provided appears to be a portrait of the famous artwork known as the Mona Lisa, one of Leonardo da Vinci's most recognizable paintings. It features a woman with a subtly enigmatic smile and her gaze directed slightly off-center towards the viewer. The background is filled with elements of Renaissance landscape painting, including rolling hills, trees, waterways, and a distant city skyline, creating a serene yet intriguing atmosphere that complements the subject's ethereal presence.
92+
USER:
93+
Do you know who drew this painting?
94+
ASSISTANT:
95+
Yes, the painting was created by Leonardo da Vinci. It is one of his most celebrated works and is known for its enigmatic expression and the artist's attention to detail in both the subject's features and the landscape elements surrounding her.
96+
```

0 commit comments

Comments
 (0)