Set compress_pos_emb to 2 for max_seq_len = 4096, or 4 for max_seq_len = 8192.
if the value of compress_pos_emb is 0, it means that the model is used in standard mode, without compression of positional embedded representations.

Launching new experimental GPTQ models (offers up to 8K context size) via ExLlama.

Hugging Face - [ Ссылка ]
How to install Oobabooga WebUI - [ Ссылка ]
How to use Oobabooga textgen in SillyTavern - [ Ссылка ]
How to use Oobabooga textgen in RisuAI - [ Ссылка ]

Music - Bill Kiley - You Will Never Know (Katana ZERO OST)

свернуть