Skip to content

gguf-py: add support for I8, I16 and I32 - #6045

Merged
ggerganov merged 4 commits into
ggml-org:masterfrom
certik:gguf_writer
Mar 14, 2024
Merged

gguf-py: add support for I8, I16 and I32#6045
ggerganov merged 4 commits into
ggml-org:masterfrom
certik:gguf_writer

Conversation

@certik

@certik certik commented Mar 13, 2024

Copy link
Copy Markdown
Contributor
@certik
Refactor dtype handling to be extensible
b7e9d5c 
This code is equivalent as before, but now it is prepared to easily add
more NumPy dtypes.
@certik certik mentioned this pull request Mar 13, 2024
Open
@certik certik mentioned this pull request Mar 14, 2024
Closed
@ggerganov

ggerganov commented Mar 14, 2024

Copy link
Copy Markdown
Member

Will merge after #6050
@ggerganov
ggerganov merged commit 3ca2348 into ggml-org:master Mar 14, 2024
ggerganov added a commit that referenced this pull request Mar 14, 2024
@ggerganov
gguf-py : fix dtype check (#6045)
77178ee
@certik
certik deleted the gguf_writer branch March 14, 2024 14:13
@certik

certik commented Mar 14, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

@ggerganov my apologies for the tensor_shape vs tensor_dtype mistake --- I just discovered it this morning as well. I thought I tested it carefully, but I missed this. Thanks for fixing it!
@ggerganov

ggerganov commented Mar 14, 2024

Copy link
Copy Markdown
Member

No problem. Btw, I think we need to update the gguf-py version
@certik certik mentioned this pull request Mar 14, 2024
Merged
@certik

certik commented Mar 14, 2024

Copy link
Copy Markdown
Contributor Author

Btw, I think we need to update the gguf-py version

I sent a PR to do so here: #6060.
NeoZhangJianyu pushed a commit to NeoZhangJianyu/llama.cpp that referenced this pull request Mar 15, 2024
@certik @NeoZhangJianyu
gguf-py : add support for I8, I16 and I32 (ggml-org#6045)
3d317f6 
* Refactor dtype handling to be extensible

This code is equivalent as before, but now it is prepared to easily add
more NumPy dtypes.

* Add support for I8, I16 and I32

These types are allowed in the GGUF specification.

* Add support for I8, I16 and I32 to gguf_writer

* Add support for I8, I16, I32 to gguf_reader
NeoZhangJianyu pushed a commit to NeoZhangJianyu/llama.cpp that referenced this pull request Mar 15, 2024
@ggerganov @NeoZhangJianyu
gguf-py : fix dtype check (ggml-org#6045)
42b03c4
@CISC CISC mentioned this pull request Jun 3, 2024
Merged
mishig25 pushed a commit to huggingface/huggingface.js that referenced this pull request Jun 3, 2024
@CISC
Update GGUF quantization types (#729)
d00b389 
Bring `GGMLQuantizationType` up to date; adds `I8`, `I16`, `I32`, `I64`,
`F64`, `IQ1_M` and `BF16`.

Added in:
* ggml-org/llama.cpp#6045
* ggml-org/llama.cpp#6062
* ggml-org/llama.cpp#6302
* ggml-org/llama.cpp#6412
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
@certik
gguf-py : add support for I8, I16 and I32 (ggml-org#6045)
58a3414 
* Refactor dtype handling to be extensible

This code is equivalent as before, but now it is prepared to easily add
more NumPy dtypes.

* Add support for I8, I16 and I32

These types are allowed in the GGUF specification.

* Add support for I8, I16 and I32 to gguf_writer

* Add support for I8, I16, I32 to gguf_reader
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
@ggerganov
gguf-py : fix dtype check (ggml-org#6045)
79886de
phuongncn pushed a commit to phuongncn/llama.cpp-gx10-dgx-sparks-deepseekv4 that referenced this pull request Apr 28, 2026
@certik
gguf-py : add support for I8, I16 and I32 (ggml-org#6045)
cb903ba 
* Refactor dtype handling to be extensible

This code is equivalent as before, but now it is prepared to easily add
more NumPy dtypes.

* Add support for I8, I16 and I32

These types are allowed in the GGUF specification.

* Add support for I8, I16 and I32 to gguf_writer

* Add support for I8, I16, I32 to gguf_reader
phuongncn pushed a commit to phuongncn/llama.cpp-gx10-dgx-sparks-deepseekv4 that referenced this pull request Apr 28, 2026
@ggerganov
gguf-py : fix dtype check (ggml-org#6045)
0a538d4
ljubomirj pushed a commit to ljubomirj/llama.cpp that referenced this pull request May 6, 2026
@certik
gguf-py : add support for I8, I16 and I32 (ggml-org#6045)
d4a7d1d 
* Refactor dtype handling to be extensible

This code is equivalent as before, but now it is prepared to easily add
more NumPy dtypes.

* Add support for I8, I16 and I32

These types are allowed in the GGUF specification.

* Add support for I8, I16 and I32 to gguf_writer

* Add support for I8, I16, I32 to gguf_reader
ljubomirj pushed a commit to ljubomirj/llama.cpp that referenced this pull request May 6, 2026
@ggerganov
gguf-py : fix dtype check (ggml-org#6045)
bd3a847
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
@certik
gguf-py : add support for I8, I16 and I32 (ggml-org#6045)
1f2b7b5 
* Refactor dtype handling to be extensible

This code is equivalent as before, but now it is prepared to easily add
more NumPy dtypes.

* Add support for I8, I16 and I32

These types are allowed in the GGUF specification.

* Add support for I8, I16 and I32 to gguf_writer

* Add support for I8, I16, I32 to gguf_reader
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
@ggerganov
gguf-py : fix dtype check (ggml-org#6045)
becc95d
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
@certik
gguf-py : add support for I8, I16 and I32 (ggml-org#6045)
9db5ff6 
* Refactor dtype handling to be extensible

This code is equivalent as before, but now it is prepared to easily add
more NumPy dtypes.

* Add support for I8, I16 and I32

These types are allowed in the GGUF specification.

* Add support for I8, I16 and I32 to gguf_writer

* Add support for I8, I16, I32 to gguf_reader
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
@ggerganov
gguf-py : fix dtype check (ggml-org#6045)
35f3736
AlexiAlp pushed a commit to minghaop/llama.cpp that referenced this pull request Jun 2, 2026
@certik
gguf-py : add support for I8, I16 and I32 (ggml-org#6045)
0dad4f8 
* Refactor dtype handling to be extensible

This code is equivalent as before, but now it is prepared to easily add
more NumPy dtypes.

* Add support for I8, I16 and I32

These types are allowed in the GGUF specification.

* Add support for I8, I16 and I32 to gguf_writer

* Add support for I8, I16, I32 to gguf_reader
AlexiAlp pushed a commit to minghaop/llama.cpp that referenced this pull request Jun 2, 2026
@ggerganov
gguf-py : fix dtype check (ggml-org#6045)
c323f9d
AlexiAlp pushed a commit to minghaop/llama.cpp that referenced this pull request Jun 2, 2026
@certik
gguf-py : add support for I8, I16 and I32 (ggml-org#6045)
c2df551 
* Refactor dtype handling to be extensible

This code is equivalent as before, but now it is prepared to easily add
more NumPy dtypes.

* Add support for I8, I16 and I32

These types are allowed in the GGUF specification.

* Add support for I8, I16 and I32 to gguf_writer

* Add support for I8, I16, I32 to gguf_reader
AlexiAlp pushed a commit to minghaop/llama.cpp that referenced this pull request Jun 2, 2026
@ggerganov
gguf-py : fix dtype check (ggml-org#6045)
08efcc5
fukuro-kun pushed a commit to fukuro-kun/fukuro-llama-cpp-turboquant that referenced this pull request Jul 5, 2026
@certik
gguf-py : add support for I8, I16 and I32 (ggml-org#6045)
d2501bd 
* Refactor dtype handling to be extensible

This code is equivalent as before, but now it is prepared to easily add
more NumPy dtypes.

* Add support for I8, I16 and I32

These types are allowed in the GGUF specification.

* Add support for I8, I16 and I32 to gguf_writer

* Add support for I8, I16, I32 to gguf_reader
fukuro-kun pushed a commit to fukuro-kun/fukuro-llama-cpp-turboquant that referenced this pull request Jul 5, 2026
@ggerganov
gguf-py : fix dtype check (ggml-org#6045)
37ff2f1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants

@certik @ggerganov