llm_load_print_meta: format         = GGUF V1 (latest)    llm_load_print_meta: arch           = llama    llm_load_print_meta: vocab type     = SPM    llm_load_print_meta: n_vocab        = 32016    llm_load_print_meta: n_merges       = 0    llm_load_print_meta: n_ctx_train    = 16384    llm_load_print_meta: n_ctx          = 5000    llm_load_print_meta: n_embd         = 5120    llm_load_print_meta: n_head         = 40    llm_load_print_meta: n_head_kv      = 40    llm_load_print_meta: n_layer        = 40    llm_load_print_meta: n_rot          = 128    llm_load_print_meta: n_gqa          = 1    llm_load_print_meta: f_norm_eps     = 1.0e-05    llm_load_print_meta: f_norm_rms_eps = 1.0e-05    llm_load_print_meta: n_ff           = 13824    llm_load_print_meta: freq_base      = 1000000.0    llm_load_print_meta: freq_scale     = 1    llm_load_print_meta: model type     = 13B    llm_load_print_meta: model ftype    = mostly Q4_K - Medium    llm_load_print_meta: model size     = 13.02 B    llm_load_print_meta: general.name   = LLaMA    llm_load_print_meta: BOS token = 1 '<s>'    llm_load_print_meta: EOS token = 2 '</s>'    llm_load_print_meta: UNK token = 0 '<unk>'    llm_load_print_meta: LF token  = 13 '<0x0A>'    llm_load_tensors: ggml ctx size =    0.11 MB    llm_load_tensors: mem required  = 7685.49 MB (+ 3906.25 MB per state)    .................................................................................................    llama_new_context_with_model: kv self size  = 3906.25 MB    ggml_metal_init: allocating    ggml_metal_init: loading '/Users/rlm/miniforge3/envs/llama2/lib/python3.9/site-packages/llama_cpp/ggml-metal.metal'    ggml_metal_init: loaded kernel_add                            0x12126dd00 | th_max = 1024 | th_width =   32    ggml_metal_init: loaded kernel_add_row                        0x12126d610 | th_max = 1024 | th_width =   32    ggml_metal_init: loaded kernel_mul                            0x12126f2a0 | th_max = 1024 | th_width =   32
Open In Collab
Open In Collab ->: llm_load_print_meta: format         = GGUF V1 (latest)    llm_load_print_meta: arch           = llama    llm_load_print_meta: vocab type     = SPM    llm_load_print_meta: n_vocab        = 32016    llm_load_print_meta: n_merges       = 0    llm_load_print_meta: n_ctx_train    = 16384    llm_load_print_meta: n_ctx          = 5000    llm_load_print_meta: n_embd         = 5120    llm_load_print_meta: n_head         = 40    llm_load_print_meta: n_head_kv      = 40    llm_load_print_meta: n_layer        = 40    llm_load_print_meta: n_rot          = 128    llm_load_print_meta: n_gqa          = 1    llm_load_print_meta: f_norm_eps     = 1.0e-05    llm_load_print_meta: f_norm_rms_eps = 1.0e-05    llm_load_print_meta: n_ff           = 13824    llm_load_print_meta: freq_base      = 1000000.0    llm_load_print_meta: freq_scale     = 1    llm_load_print_meta: model type     = 13B    llm_load_print_meta: model ftype    = mostly Q4_K - Medium    llm_load_print_meta: model size     = 13.02 B    llm_load_print_meta: general.name   = LLaMA    llm_load_print_meta: BOS token = 1 '<s>'    llm_load_print_meta: EOS token = 2 '</s>'    llm_load_print_meta: UNK token = 0 '<unk>'    llm_load_print_meta: LF token  = 13 '<0x0A>'    llm_load_tensors: ggml ctx size =    0.11 MB    llm_load_tensors: mem required  = 7685.49 MB (+ 3906.25 MB per state)    .................................................................................................    llama_new_context_with_model: kv self size  = 3906.25 MB    ggml_metal_init: allocating    ggml_metal_init: loading '/Users/rlm/miniforge3/envs/llama2/lib/python3.9/site-packages/llama_cpp/ggml-metal.metal'    ggml_metal_init: loaded kernel_add                            0x12126dd00 | th_max = 1024 | th_width =   32    ggml_metal_init: loaded kernel_add_row                        0x12126d610 | th_max = 1024 | th_width =   32    ggml_metal_init: loaded kernel_mul                            0x12126f2a0 | th_max = 1024 | th_width =   32