struct
Q6_KEncoding
The Q6_K quantization encoding.
Because this holds the quantized data in a special packing format, it currently does not print float values at runtime—it's just a bag of bits in uint8 format.
Implemented traits
AnyType
,
QuantizationEncoding
Methods
quantize
static quantize(_tensor: Tensor[float32]) -> Tensor[uint8]
Quantizes the full-precision tensor tensor
to Q6_K.
The quantize method is not yet implemented. However, since Q6_K quantized ops are supported, Q6_KEncoding is still provided to allow code to be generic over quantization encoding type.
id
static id() -> String
Identifier for the Q6_K quantized encoding.
Was this page helpful?
Thank you! We'll create more content like this.
Thank you for helping us improve!
If you'd like to share more information, please report an issue on GitHub
😔 What went wrong?