It is broken into two parts: installation and setup, and then references to specific Hugging Face wrappers.Installation and Setup‚ÄãIf you want to work with the Hugging Face Hub:Install the Hub client library with pip install huggingface_hubCreate a Hugging Face account (it's free!)Create an access token and set it as an environment variable (HUGGINGFACEHUB_API_TOKEN)If you want work with the Hugging Face Python libraries:Install pip install transformers for working with models and tokenizersInstall pip install datasets for working with datasetsWrappers‚ÄãLLM‚ÄãThere exists two Hugging Face LLM wrappers, one for a local pipeline and one for a model hosted on Hugging Face Hub.
Note that these wrappers only work for models that support the following tasks: text2text-generation, text-generationTo use the local pipeline wrapper:from langchain.llms import HuggingFacePipelineTo use a the wrapper for a model hosted on Hugging Face Hub:from langchain.llms import HuggingFaceHubFor a more detailed walkthrough of the Hugging Face Hub wrapper, see this notebookEmbeddings‚ÄãThere exists two Hugging Face Embeddings wrappers, one for a local model and one for a model hosted on Hugging Face Hub.
Note that these wrappers only work for sentence-transformers models.To use the local pipeline wrapper:from langchain.embeddings import HuggingFaceEmbeddingsTo use a the wrapper for a model hosted on Hugging Face Hub:from langchain.embeddings import HuggingFaceHubEmbeddingsFor a more detailed walkthrough of this, see this notebookTokenizer‚ÄãThere are several places you can use tokenizers available through the transformers package.
This page covers how to use the Hugging Face ecosystem (including the Hugging Face Hub) within LangChain.
This page covers how to use the Hugging Face ecosystem (including the Hugging Face Hub) within LangChain. ->: It is broken into two parts: installation and setup, and then references to specific Hugging Face wrappers.Installation and Setup‚ÄãIf you want to work with the Hugging Face Hub:Install the Hub client library with pip install huggingface_hubCreate a Hugging Face account (it's free!)Create an access token and set it as an environment variable (HUGGINGFACEHUB_API_TOKEN)If you want work with the Hugging Face Python libraries:Install pip install transformers for working with models and tokenizersInstall pip install datasets for working with datasetsWrappers‚ÄãLLM‚ÄãThere exists two Hugging Face LLM wrappers, one for a local pipeline and one for a model hosted on Hugging Face Hub.
Note that these wrappers only work for models that support the following tasks: text2text-generation, text-generationTo use the local pipeline wrapper:from langchain.llms import HuggingFacePipelineTo use a the wrapper for a model hosted on Hugging Face Hub:from langchain.llms import HuggingFaceHubFor a more detailed walkthrough of the Hugging Face Hub wrapper, see this notebookEmbeddings‚ÄãThere exists two Hugging Face Embeddings wrappers, one for a local model and one for a model hosted on Hugging Face Hub.
Note that these wrappers only work for sentence-transformers models.To use the local pipeline wrapper:from langchain.embeddings import HuggingFaceEmbeddingsTo use a the wrapper for a model hosted on Hugging Face Hub:from langchain.embeddings import HuggingFaceHubEmbeddingsFor a more detailed walkthrough of this, see this notebookTokenizer‚ÄãThere are several places you can use tokenizers available through the transformers package.