content.raw: Supercool Weekend Read🤖
content.type: text
content.value: Supercool Weekend Read🤖
content.type: new_line
content.raw: Nvidia researchers achieved SOTA LLM compression metrics using pruning and knowledge distillation techniques.
content.type: text
content.value: Nvidia researchers achieved SOTA LLM compression metrics using pruning and knowledge distillation techniques.
content.type: new_line
content.type: new_line
content.raw: Details on Techniques (Simplified):
content.type: text
content.value: Details on Techniques (Simplified):
content.type: new_line
content.raw: They started off with a large pre-trained language model (15B params), then:
content.type: text
content.value: They started off with a large pre-trained language model (15B params), then:
content.type: new_line
content.type: new_line
content.raw: 1. Estimated the importance of different parts of the model (neurons, attention heads, layers) using activation-based metrics on a small calibration dataset.
content.type: text
content.value: 1. Estimated the importance of different parts of the model (neurons, attention heads, layers) using activation-based metrics on a small calibration dataset.
content.type: new_line
content.type: new_line
content.raw: 2. Pruned (remove) less important parts of the model to reduce its size.
content.type: text
content.value: 2. Pruned (remove) less important parts of the model to reduce its size.
content.type: new_line
content.type: new_line
content.raw: 3. Retrained the pruned model using knowledge distillation, where the original large model acts as a teacher for the smaller pruned model.
content.type: text
content.value: 3. Retrained the pruned model using knowledge distillation, where the original large model acts as a teacher for the smaller pruned model.
content.type: new_line
content.type: new_line
content.raw: 4. Used a lightweight neural architecture search to find the best configuration for the pruned model.
content.type: text
content.value: 4. Used a lightweight neural architecture search to find the best configuration for the pruned model.
content.type: new_line
content.type: new_line
content.raw: 5. Repeated this process iteratively to create even smaller models.
content.type: text
content.value: 5. Repeated this process iteratively to create even smaller models.
content.type: new_line
content.type: new_line
content.raw: Cool, giving it a try this weekend 😎
content.type: text
content.value: Cool, giving it a try this weekend 😎
content.type: new_line
content.raw: Code: 
content.type: text
content.value: Code: 
content.href: https://github.com/NVlabs/Minitron
content.raw: https://github.com/NVlabs/Minitron
content.type: link
content.type: new_line
content.raw: Paper: 
content.type: text
content.value: Paper: 
content.href: https://arxiv.org/abs/2407.14679
content.raw: https://arxiv.org/abs/2407.14679
content.type: link
content.type: new_line
content.raw: Demo: 
content.type: text
content.value: Demo: 
content.raw: https://huggingface.co/spaces/nvidia/minitron
content.id: nvidia/minitron
content.type: space
content.type: resource
content.url: https://huggingface.co/spaces/nvidia/minitron
rawContent: Supercool Weekend Read🤖
  Nvidia researchers achieved SOTA LLM compression metrics using pruning and knowledge distillation techniques.
  
  Details on Techniques (Simplified):
  They started off with a large pre-trained language model (15B params), then:
  
  1. Estimated the importance of different parts of the model (neurons, attention heads, layers) using activation-based metrics on a small calibration dataset.
  
  2. Pruned (remove) less important parts of the model to reduce its size.
  
  3. Retrained the pruned model using knowledge distillation, where the original large model acts as a teacher for the smaller pruned model.
  
  4. Used a lightweight neural architecture search to find the best configuration for the pruned model.
  
  5. Repeated this process iteratively to create even smaller models.
  
  Cool, giving it a try this weekend 😎
  Code: https://github.com/NVlabs/Minitron
  Paper: https://arxiv.org/abs/2407.14679
  Demo: https://huggingface.co/spaces/nvidia/minitron
reactions.reaction: 👍
reactions.users: Edouard360 ajibawa-2023 adamelliotfields davidle7 KingNish
reactions.reaction: 👀
reactions.users: John6666 Dihelson tuanlda78202
reactions.reaction: ❤️
reactions.users: Dihelson