Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKProvidersAnthropicAWSGoogleMicrosoftOpenAIMoreComponentsLLMsLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleArceeAzure MLAzure OpenAIBaidu QianfanBananaBasetenBeamBedrockBittensorCerebriumAIChatGLMClarifaiCohereC TransformersCTranslate2DatabricksDeepInfraDeepSparseEden AIFireworksForefrontAIGCP Vertex AIGooseAIGPT4AllGradientHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJavelin AI Gateway TutorialJSONFormerKoboldAI APILlama.cppLLM Caching integrationsManifestMinimaxModalMosaicMLNLP CloudOctoAIOllamaOpaquePromptsOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAINebula (Symbl.ai)TextGenTitan TakeoffTogether AITongyi QwenvLLMWriterXorbits Inference (Xinference)YandexGPTChat modelsDocument loadersDocument transformersText embedding modelsVector storesRetrieversToolsAgents and toolkitsMemoryCallbacksChat loadersComponentsLLMsOllamaOn this pageOllamaOllama allows you to run open-source large language models, such as Llama 2, locally.Ollama bundles model weights, configuration, and data into a single package, defined by a Modelfile. It optimizes setup and configuration details, including GPU usage.For a complete list of supported models and model variants, see the Ollama model library.Setup‚ÄãFirst, follow these instructions to set up and run a local Ollama instance:DownloadFetch a model via ollama pull <model family>e.g., for Llama-7b: ollama pull llama2 (see full list here)This will download the most basic version of the model typically (e.g., smallest # parameters and q4_0)On Mac, it will download to ~/.ollama/models/manifests/registry.ollama.ai/library/<model family>/latestAnd we specify a particular version, e.g., for ollama pull vicuna:13b-v1.5-16k-q4_0The file is here with the model version in place of
Ollama allows you to run open-source large language models, such as Llama 2, locally.
Ollama allows you to run open-source large language models, such as Llama 2, locally. ->: Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKProvidersAnthropicAWSGoogleMicrosoftOpenAIMoreComponentsLLMsLLMsAI21Aleph AlphaAmazon API GatewayAnyscaleArceeAzure MLAzure OpenAIBaidu QianfanBananaBasetenBeamBedrockBittensorCerebriumAIChatGLMClarifaiCohereC TransformersCTranslate2DatabricksDeepInfraDeepSparseEden AIFireworksForefrontAIGCP Vertex AIGooseAIGPT4AllGradientHugging Face HubHugging Face Local PipelinesHuggingface TextGen InferenceJavelin AI Gateway TutorialJSONFormerKoboldAI APILlama.cppLLM Caching integrationsManifestMinimaxModalMosaicMLNLP CloudOctoAIOllamaOpaquePromptsOpenAIOpenLLMOpenLMPetalsPipelineAIPredibasePrediction GuardPromptLayer OpenAIRELLMReplicateRunhouseSageMakerEndpointStochasticAINebula (Symbl.ai)TextGenTitan TakeoffTogether AITongyi QwenvLLMWriterXorbits Inference (Xinference)YandexGPTChat modelsDocument loadersDocument transformersText embedding modelsVector storesRetrieversToolsAgents and toolkitsMemoryCallbacksChat loadersComponentsLLMsOllamaOn this pageOllamaOllama allows you to run open-source large language models, such as Llama 2, locally.Ollama bundles model weights, configuration, and data into a single package, defined by a Modelfile. It optimizes setup and configuration details, including GPU usage.For a complete list of supported models and model variants, see the Ollama model library.Setup‚ÄãFirst, follow these instructions to set up and run a local Ollama instance:DownloadFetch a model via ollama pull <model family>e.g., for Llama-7b: ollama pull llama2 (see full list here)This will download the most basic version of the model typically (e.g., smallest # parameters and q4_0)On Mac, it will download to ~/.ollama/models/manifests/registry.ollama.ai/library/<model family>/latestAnd we specify a particular version, e.g., for ollama pull vicuna:13b-v1.5-16k-q4_0The file is here with the model version in place of