-U llama-cpp-python --no-cache-dirCheck out the latest code-llama models here.from langchain.llms import LlamaCppfrom langchain.prompts import PromptTemplatefrom langchain.chains import LLMChainfrom langchain.callbacks.manager import CallbackManagerfrom langchain.memory import ConversationSummaryMemoryfrom langchain.chains import ConversationalRetrievalChain from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandlercallback_manager = CallbackManager([StreamingStdOutCallbackHandler()])llm = LlamaCpp(    model_path="/Users/rlm/Desktop/Code/llama/code-llama/codellama-13b-instruct.Q4_K_M.gguf",    n_ctx=5000,    n_gpu_layers=1,    n_batch=512,    f16_kv=True,  # MUST set to True, otherwise you will run into problem after a couple of calls    callback_manager=callback_manager,    verbose=True,)    llama_model_loader: loaded meta data with 17 key-value pairs and 363 tensors from /Users/rlm/Desktop/Code/llama/code-llama/codellama-13b-instruct.Q4_K_M.gguf (version GGUF V1 (latest))    llama_model_loader: - tensor    0:                token_embd.weight q4_0     [  5120, 32016,     1,     1 ]    llama_model_loader: - tensor    1:               output_norm.weight f32      [  5120,     1,     1,     1 ]    llama_model_loader: - tensor    2:                    output.weight f16      [  5120, 32016,     1,     1 ]    llama_model_loader: - tensor    3:              blk.0.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]    llama_model_loader: - tensor    4:              blk.0.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]    llama_model_loader: - tensor    5:              blk.0.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]    llama_model_loader: - tensor    6:         blk.0.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]    llama_model_loader: - tensor    7:            blk.0.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]    llama_model_loader: - tensor    8:            blk.0.ffn_down.weight q6_K     [ 13824,  5120,
Open In Collab
Open In Collab ->: -U llama-cpp-python --no-cache-dirCheck out the latest code-llama models here.from langchain.llms import LlamaCppfrom langchain.prompts import PromptTemplatefrom langchain.chains import LLMChainfrom langchain.callbacks.manager import CallbackManagerfrom langchain.memory import ConversationSummaryMemoryfrom langchain.chains import ConversationalRetrievalChain from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandlercallback_manager = CallbackManager([StreamingStdOutCallbackHandler()])llm = LlamaCpp(    model_path="/Users/rlm/Desktop/Code/llama/code-llama/codellama-13b-instruct.Q4_K_M.gguf",    n_ctx=5000,    n_gpu_layers=1,    n_batch=512,    f16_kv=True,  # MUST set to True, otherwise you will run into problem after a couple of calls    callback_manager=callback_manager,    verbose=True,)    llama_model_loader: loaded meta data with 17 key-value pairs and 363 tensors from /Users/rlm/Desktop/Code/llama/code-llama/codellama-13b-instruct.Q4_K_M.gguf (version GGUF V1 (latest))    llama_model_loader: - tensor    0:                token_embd.weight q4_0     [  5120, 32016,     1,     1 ]    llama_model_loader: - tensor    1:               output_norm.weight f32      [  5120,     1,     1,     1 ]    llama_model_loader: - tensor    2:                    output.weight f16      [  5120, 32016,     1,     1 ]    llama_model_loader: - tensor    3:              blk.0.attn_q.weight q4_K     [  5120,  5120,     1,     1 ]    llama_model_loader: - tensor    4:              blk.0.attn_k.weight q4_K     [  5120,  5120,     1,     1 ]    llama_model_loader: - tensor    5:              blk.0.attn_v.weight q6_K     [  5120,  5120,     1,     1 ]    llama_model_loader: - tensor    6:         blk.0.attn_output.weight q4_K     [  5120,  5120,     1,     1 ]    llama_model_loader: - tensor    7:            blk.0.ffn_gate.weight q4_K     [  5120, 13824,     1,     1 ]    llama_model_loader: - tensor    8:            blk.0.ffn_down.weight q6_K     [ 13824,  5120,