Update README.md

README.md

@@ -55,33 +55,37 @@
 Today, we’re introducing DeepSeek-V2, a strong Mixture-of-Experts (MoE) language model characterized by economical training and efficient inference. It comprises 236B total parameters, of which 21B are activated for each token. Compared with DeepSeek 67B, DeepSeek-V2 achieves stronger performance, and meanwhile saves 42.5% of training costs, reduces the KV cache by 93.3%, and boosts the maximum generation throughput to 5.76 times. 
 
 <p align="center">
-
 <div style="display: flex; justify-content: center;">
-    <img src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/activationparameters.png?raw=true" style="height:300px; width:auto; margin-right:10px">
-    <img src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/trainingcost.png?raw=true" style="height:300px; width:auto; margin-left:10px">
+    <img src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/activationparameters.png?raw=true" style="height:400px; width:auto; margin-right:10px">
+    <img src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/trainingcost.png?raw=true" style="height:400px; width:auto; margin-left:10px">
 </div>
 </p>
+
 We pretrained DeepSeek-V2 on a diverse and high-quality corpus comprising 8.1 trillion tokens. This comprehensive pretraining was followed by a process of Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) to fully unleash the model's capabilities. The evaluation results validate the effectiveness of our approach as DeepSeek-V2 achieves remarkable performance on both standard benchmarks and open-ended generation evaluation.
 
-## 2. Model Downloads
+## 2. News
 
-<div align="center">
+- 2024.05.16: We released DeepSeek-V2-Lite, a lite version of DeepSeek-V2.
+- 2024.05.06: We released the DeepSeek-V2.
 
-| **Model** | **Context Length** | **Download** |
-| :------------: | :------------: | :------------: |
-| DeepSeek-V2-Lite   | 128k   | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite)   |
-| DeepSeek-V2-Lite-Chat (SFT)   | 128k   | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat)   |
-| DeepSeek-V2   | 128k   | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V2)   |
-| DeepSeek-V2-Chat (RL)   | 128k   | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V2-Chat)   |
+## 3. Model Downloads
+
+<div align="center">
 
+| **Model** | **#Total Params** | **#Activated Params** | **Context Length** | **Download** |
+| :------------: | :------------: | :------------: | :------------: | :------------: |
+| DeepSeek-V2-Lite | 16B | 2.4B | 32k   | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite)   |
+| DeepSeek-V2-Lite-Chat (SFT)   | 16B | 2.4B | 32k   | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat)   |
+| DeepSeek-V2   | 236B | 21B |  128k   | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V2)   |
+| DeepSeek-V2-Chat (RL)   | 236B | 21B |  128k   | [🤗 HuggingFace](https://huggingface.co/deepseek-ai/DeepSeek-V2-Chat)   |
 
 </div>
 
 Due to the constraints of HuggingFace, the open-source code currently experiences slower performance than our internal codebase when running on GPUs with Huggingface. To facilitate the efficient execution of our model, we offer a dedicated vllm solution that optimizes performance for running our model effectively.
 
-## 3. Evaluation Results
+## 4. Evaluation Results
 ### Base Model
-#### Standard Benchmark 
+#### Standard Benchmark (Models larger than 67B)
 
 <div align="center">
 
@@ -91,11 +95,28 @@ Due to the constraints of HuggingFace, the open-source code currently experience
 | **BBH** | English | 81.0 | 78.9 | 68.7 | 78.9 |
 | **C-Eval** | Chinese | 67.5 | 58.6 | 66.1 | 81.7 |
 | **CMMLU** | Chinese | 69.3 | 60.0 | 70.8 | 84.0 |
-| **HumanEval** | Code | 48.2 | 53.1 | 45.1 | 48.8 |
+| **HumanEval** | Code | 48.2	| 53.1 | 45.1 | 48.8 |
 | **MBPP** | Code | 68.6 | 64.2 | 57.4 | 66.6 |
 | **GSM8K** | Math | 83.0 | 80.3 | 63.4 | 79.2 |
 | **Math** | Math | 42.2 | 42.5 | 18.7 | 43.6 |
 
+</div>
+
+#### Standard Benchmark (Models smaller than 16B)
+<div align="center">
+
+| **Benchmark** | **Domain** | **DeepSeek 7B (Dense)** | **DeepSeekMoE 16B** | **DeepSeek-V2-Lite (MoE-16B)** |
+|:-------------:|:----------:|:--------------:|:-----------------:|:--------------------------:|
+| **Architecture**      | -    | MHA+Dense           | MHA+MoE              | MLA+MoE                       |
+| **MMLU**      | English    | 48.2           | 45.0              | 58.3                       |
+| **BBH**       | English    | xxxx           | xxxx              | 44.1                       |
+| **C-Eval**    | Chinese    | 45.0           | 40.6              | 60.3                       |
+| **CMMLU**     | Chinese    | 47.2           | 42.5              | 64.3                       |
+| **HumanEval** | Code       | 26.2           | 26.8              | 29.9                       |
+| **MBPP**      | Code       | 39.0           | 39.2              | 43.2                       |
+| **GSM8K**     | Math       | 17.4           | 18.8              | 41.1                       |
+| **Math**      | Math       | 3.3           | 4.3             | 17.1                       |
+
 </div>
 For more evaluation details, such as few-shot settings and prompts, please check our paper. 
 
@@ -107,7 +128,7 @@ For more evaluation details, such as few-shot settings and prompts, please check
 Evaluation results on the ``Needle In A Haystack`` (NIAH) tests.  DeepSeek-V2 performs well across all context window lengths up to **128K**. 
 
 ### Chat Model
-#### Standard Benchmark 
+#### Standard Benchmark (Models larger than 67B)
 <div align="center">
 
 | Benchmark | Domain         | QWen1.5 72B Chat | Mixtral 8x22B | LLaMA3 70B Instruct | DeepSeek-V1 Chat (SFT) | DeepSeek-V2 Chat (SFT) | DeepSeek-V2 Chat (RL) |
@@ -124,6 +145,23 @@ Evaluation results on the ``Needle In A Haystack`` (NIAH) tests.  DeepSeek-V2 pe
 
 </div>
 
+#### Standard Benchmark (Models smaller than 16B)
+
+<div align="center">
+
+| Benchmark | Domain         | DeepSeek 7B Chat (SFT) | DeepSeekMoE 16B Chat (SFT) | DeepSeek-V2-Lite 16B Chat (SFT) |
+|:-----------:|:----------------:|:------------------:|:---------------:|:---------------------:|
+| **MMLU**      | English        | 49.7             | 47.2          | 55.7                |
+| **BBH**       | English        | 43.1             | 42.2          | 48.1                |
+| **C-Eval**    | Chinese        | 44.7             | 40.0          | 60.1                |
+| **CMMLU**     | Chinese        | 51.2             | 49.3          | 62.5                |
+| **HumanEval** | Code           | 45.1             | 45.7          | 57.3                |
+| **MBPP**      | Code           | 39.0             | 46.2          | 45.8                |
+| **GSM8K**     | Math           | 62.6             | 62.2          | 72.0                |
+| **Math**      | Math           | 14.7             | 15.2          | 27.9                |
+
+</div>
+
 #### English Open Ended Generation Evaluation
 We evaluate our model on AlpacaEval 2.0 and MTBench, showing the competitive performance of DeepSeek-V2-Chat-RL on English conversation generation. 
 <p align="center">
@@ -137,7 +175,7 @@ We evaluate our model on AlpacaEval 2.0 and MTBench, showing the competitive per
 | **模型** | **开源/闭源** | **总分** | **中文推理** | **中文语言** |
 | :---: | :---: | :---: | :---: | :---: |
 | gpt-4-1106-preview | 闭源 | 8.01 | 7.73 | 8.29 |
-| DeepSeek-V2 Chat (RL) | 开源 | 7.91 | 7.45 | 8.35 |
+| DeepSeek-V2 Chat (RL) | 开源 | 7.91 | 7.45 | 8.36 |
 | erniebot-4.0-202404 (文心一言) | 闭源 | 7.89 | 7.61 | 8.17 |
 | DeepSeek-V2 Chat (SFT) | 开源 | 7.74 | 7.30 | 8.17 |
 | gpt-4-0613 | 闭源 | 7.53 | 7.47 | 7.59 |
@@ -147,6 +185,7 @@ We evaluate our model on AlpacaEval 2.0 and MTBench, showing the competitive per
 | DeepSeek-67B-Chat | 开源 | 6.43 | 5.75 | 7.11 |
 | Yi-34B-Chat (零一万物) | 开源 | 6.12 | 4.86 | 7.38 |
 | gpt-3.5-turbo-0613 | 闭源 | 6.08 | 5.35 | 6.71 |
+| DeepSeek-V2-Lite 16B Chat | 开源 | 6.01 | 4.71 | 7.32 |
 
 </div>
 
@@ -157,7 +196,7 @@ We evaluate our model on LiveCodeBench (0901-0401), a benchmark designed for liv
   <img width="50%" src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/code_benchmarks.png?raw=true">
 </p>
 
-## 4. Model Architecture
+## 5. Model Architecture
 DeepSeek-V2 adopts innovative architectures to guarantee economical training and efficient inference： 
 - For attention, we design MLA (Multi-head Latent Attention), which utilizes low-rank key-value union compression to eliminate the bottleneck of inference-time key-value cache, thus supporting efficient inference. 
 - For Feed-Forward Networks (FFNs), we adopt DeepSeekMoE architecture, a high-performance MoE architecture that enables training stronger models at lower costs. 
@@ -165,11 +204,10 @@ DeepSeek-V2 adopts innovative architectures to guarantee economical training and
 <p align="center">
   <img width="90%" src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/architecture.png?raw=true" />
 </p>
-
-## 5. Chat Website
+## 6. Chat Website
 You can chat with the DeepSeek-V2 on DeepSeek's official website: [chat.deepseek.com](https://chat.deepseek.com/sign_in)
 
-## 6. API Platform
+## 7. API Platform
 We also provide OpenAI-Compatible API at DeepSeek Platform: [platform.deepseek.com](https://platform.deepseek.com/). Sign up for over millions of free tokens. And you can also pay-as-you-go at an unbeatable price.
 
 
@@ -177,8 +215,7 @@ We also provide OpenAI-Compatible API at DeepSeek Platform: [platform.deepseek.c
   <img width="40%" src="https://github.com/deepseek-ai/DeepSeek-V2/blob/main/figures/model_price.png?raw=true">
 </p>
 
-
-## 7. How to run locally
+## 8. How to run locally
 **To utilize DeepSeek-V2 in BF16 format for inference, 80GB*8 GPUs are required.**
 ### Inference with Huggingface's Transformers
 You can directly employ [Huggingface's Transformers](https://github.com/huggingface/transformers) for model inference.
@@ -280,10 +317,23 @@ generated_text = [output.outputs[0].text for output in outputs]
 print(generated_text)
 ```
 
-## 8. License
+### LangChain Support
+Since our API is compatible with OpenAI, you can easily use it in [langchain](https://www.langchain.com/).
+Here is an example:
+
+```
+from langchain_openai import ChatOpenAI
+llm = ChatOpenAI(
+    model='deepseek-chat',
+    openai_api_key=<your-deepseek-api-key>,
+    openai_api_base='https://api.deepseek.com/v1',
+    temperature=0.85,
+    max_tokens=8000)
+``` 
+## 9. License
 This code repository is licensed under [the MIT License](LICENSE-CODE). The use of DeepSeek-V2 Base/Chat models is subject to [the Model License](LICENSE-MODEL). DeepSeek-V2 series (including Base and Chat) supports commercial use.
 
-## 9. Citation
+## 10. Citation
 ```
 @misc{deepseekv2,
       title={DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model}, 
@@ -295,5 +345,5 @@ This code repository is licensed under [the MIT License](LICENSE-CODE). The use
 }
 ```
 
-## 10. Contact
+## 11. Contact
 If you have any questions, please raise an issue or contact us at [[email protected]]([email protected]).