LMStudio show max context 4096

#1
by olegshulyakov - opened
MLX Community org

Why does LM Studio shows max context lenght 4096 for this model?

MLX Community org

LM Studio shows a maximum context length of 4096 for this model because it is likely the default value set in the configuration, which may not reflect the actual capabilities of the model. This limitation can affect how much data the model can process at one time.

MLX Community org

@pabslop Yeah, found out that config.json is missing

{
    "text_config": {
        "max_position_embeddings": 131072,
    },
}
olegshulyakov changed discussion status to closed

Sign up or log in to comment