ICE-GRT: Instruction Context Enhancement by Generative Reinforcement
  based Transformers
The emergence of Large Language Models (LLMs) such as ChatGPT and LLaMA
encounter limitations in domain-specific tasks, with these models often lacking
depth and accuracy in specialized areas, and exhibiting a decrease in general
capabilities when fine-tuned, particularly analysis ability in small sized
models. To address these gaps, we introduce ICE-GRT, utilizing Reinforcement
Learning from Human Feedback (RLHF) grounded in Proximal Policy Optimization
(PPO), demonstrating remarkable ability in in-domain scenarios without
compromising general task performance. Our exploration of ICE-GRT highlights
its understanding and reasoning ability to not only generate robust answers but
also to provide detailed analyses of the reasons behind the answer. This
capability marks a significant progression beyond the scope of Supervised
Fine-Tuning models. The success of ICE-GRT is dependent on several crucial
factors, including Appropriate Data, Reward Size Scaling, KL-Control, Advantage
Normalization, etc. The ICE-GRT model exhibits state-of-the-art performance in
domain-specific tasks and across 12 general Language tasks against equivalent
size and even larger size LLMs, highlighting the effectiveness of our approach.
We provide a comprehensive analysis of the ICE-GRT, underscoring the
significant advancements it brings to the field of LLM.
65976a9e2d23530ec073898c
Chen Zheng
claimed_verified
2024-06-18T07:21:06.000Z
63be66194a2beec6555d49d6
https://cdn-avatars.huggingface.co/v1/production/uploads/63be66194a2beec6555d49d6/Ck-VAuADj9BCDxU0NsdTF.jpeg
CHEN ZHENG
user
zhengchenphd
65976a9e2d23530ec073898d
Ke Sun
65976a9e2d23530ec073898e
Da Tang
65976a9e2d23530ec073898f
Yukun Ma
65976a9e2d23530ec0738990
Yuyu Zhang
65976a9e2d23530ec0738991
Chenguang Xi
65976a9e2d23530ec0738992
Xun Zhou
ICE-GRT: Instruction Context Enhancement by Generative Reinforcement
  based Transformers
The emergence of Large Language Models (LLMs) such as ChatGPT and LLaMA
encounter limitations in domain-specific tasks, with these models often lacking
depth and accuracy in specialized areas, and exhibiting a decrease in general
capabilities when fine-tuned, particularly analysis ability in small sized
models. To address these gaps, we introduce ICE-GRT, utilizing Reinforcement
Learning from Human Feedback (RLHF) grounded in Proximal Policy Optimization
(PPO), demonstrating remarkable ability in in-domain scenarios without
compromising general task performance. Our exploration of ICE-GRT highlights
its understanding and reasoning ability to not only generate robust answers but
also to provide detailed analyses of the reasons behind the answer. This
capability marks a significant progression beyond the scope of Supervised
Fine-Tuning models. The success of ICE-GRT is dependent on several crucial
factors, including Appropriate Data, Reward Size Scaling, KL-Control, Advantage
Normalization, etc. The ICE-GRT model exhibits state-of-the-art performance in
domain-specific tasks and across 12 general Language tasks against equivalent
size and even larger size LLMs, highlighting the effectiveness of our approach.
We provide a comprehensive analysis of the ICE-GRT, underscoring the
significant advancements it brings to the field of LLM.
ICE-GRT, a Reinforcement Learning from Human Feedback approach using Proximal Policy Optimization, enhances domain-specific performance while maintaining general task capabilities in Large Language Models.
Large Language Models LLMs ICE-GRT Reinforcement Learning from Human Feedback RLHF Proximal Policy Optimization PPO in-domain scenarios Supervised Fine-Tuning domain-specific tasks general Language tasks Appropriate Data Reward Size Scaling KL-Control Advantage Normalization