OpenVINO IR model with int8 quantization

by fakezeta - opened May 2

May 2

Hi, I've converted you model to OpenVINO IR format.
Template to use it on LocalAI are added to the README.md file.

NousResearch org May 15

is there instructions on how to configure function calling for Heremes-2-Pro with LocalAI?

May 15

Support for function calling on transformers Is in this PR with example instruction.

I still haven't had the chance to test it: will report here as soon as possible.

Jun 24

Sorry for the late reply: function calling is working both for Hermes-2-Pro and Hermes-2-Theta.
I tested it with AnythingLLM agents.

The configuration needed has been added to the README.md

NousResearch org Jun 26

glad to hear that -- could you open a PR with an example notebook on Hermes-Function-Calling repo?
https://github.com/NousResearch/Hermes-Function-Calling/tree/main/examples

Jun 26

I'll do my best: I'm not a developer just a passionate guy.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment