Update README.md
Browse files
README.md
CHANGED
@@ -18,7 +18,7 @@ language:
|
|
18 |
---
|
19 |
|
20 |
<h1 align="center">
|
21 |
-
<img src="
|
22 |
</h1>
|
23 |
<div style="display: flex; justify-content: center; gap: 10px;">
|
24 |
<a href="https://github.com/IAAR-Shanghai/MARA">
|
@@ -32,7 +32,7 @@ language:
|
|
32 |
|
33 |
**MARA** (Micro token-level Accept-Reject Alignment) simplifies the alignment process by breaking down sentence-level preference learning into fine-grained token-level binary classification. The MARA agent—a lightweight multi-layer perceptron (MLP)—operates as an alignment model that evaluates and classifies each candidate token as either *Accepted* or *Rejected* during LLM text generation.
|
34 |
<figure>
|
35 |
-
<img src="
|
36 |
<figcaption style="text-align: center;">Architecture of MARA: The alignment model performs token selection through accept-reject decisions.</figcaption>
|
37 |
</figure>
|
38 |
|
@@ -64,7 +64,7 @@ The source code and implementation details are open-sourced at [MARA](https://gi
|
|
64 |
<table class="center">
|
65 |
<tr>
|
66 |
<td width=100% style="border: none">
|
67 |
-
<img src="
|
68 |
<div style="text-align: left; margin-top: 8px;">Performance improvements of MARA across PKUSafeRLHF, BeaverTails, and HarmfulQA datasets. Each entry shows the percentage improvement in preference rate achieved by applying MARA compared to using the original LLM alone.</div>
|
69 |
</td>
|
70 |
</tr>
|
@@ -72,7 +72,7 @@ The source code and implementation details are open-sourced at [MARA](https://gi
|
|
72 |
<table class="center">
|
73 |
<tr>
|
74 |
<td width=100% style="border: none">
|
75 |
-
<img src="
|
76 |
<div style="text-align: left; margin-top: 8px;">Compatibility analysis of MARA, an alignment model trained with a LLM to be aggregate with other inference LLM. The value of each cell represents the percentage improvement in preference rate of our algorithm over the upstream model, i.e., inference model.</div>
|
77 |
</td>
|
78 |
</tr>
|
@@ -81,7 +81,7 @@ The source code and implementation details are open-sourced at [MARA](https://gi
|
|
81 |
<table class="center">
|
82 |
<tr>
|
83 |
<td width=100% style="border: none">
|
84 |
-
<img src="
|
85 |
<div style="text-align: left; margin-top: 8px;">Performance comparison of MARA against RLHF, DPO, and Aligner measured by percentage improvements of preference rate.</div>
|
86 |
</td>
|
87 |
</tr>
|
|
|
18 |
---
|
19 |
|
20 |
<h1 align="center">
|
21 |
+
<img src="icons.png" alt="MARA Icon" width="45" height="45"/> MARA AGENTS
|
22 |
</h1>
|
23 |
<div style="display: flex; justify-content: center; gap: 10px;">
|
24 |
<a href="https://github.com/IAAR-Shanghai/MARA">
|
|
|
32 |
|
33 |
**MARA** (Micro token-level Accept-Reject Alignment) simplifies the alignment process by breaking down sentence-level preference learning into fine-grained token-level binary classification. The MARA agent—a lightweight multi-layer perceptron (MLP)—operates as an alignment model that evaluates and classifies each candidate token as either *Accepted* or *Rejected* during LLM text generation.
|
34 |
<figure>
|
35 |
+
<img src="mara_architecture.png" alt="mara_architecture" style="display: block; margin: 0 auto;" />
|
36 |
<figcaption style="text-align: center;">Architecture of MARA: The alignment model performs token selection through accept-reject decisions.</figcaption>
|
37 |
</figure>
|
38 |
|
|
|
64 |
<table class="center">
|
65 |
<tr>
|
66 |
<td width=100% style="border: none">
|
67 |
+
<img src="table1.png" style="width:50%; max-width:100%;">
|
68 |
<div style="text-align: left; margin-top: 8px;">Performance improvements of MARA across PKUSafeRLHF, BeaverTails, and HarmfulQA datasets. Each entry shows the percentage improvement in preference rate achieved by applying MARA compared to using the original LLM alone.</div>
|
69 |
</td>
|
70 |
</tr>
|
|
|
72 |
<table class="center">
|
73 |
<tr>
|
74 |
<td width=100% style="border: none">
|
75 |
+
<img src="table3.png" style="width:50%; max-width:100%;">
|
76 |
<div style="text-align: left; margin-top: 8px;">Compatibility analysis of MARA, an alignment model trained with a LLM to be aggregate with other inference LLM. The value of each cell represents the percentage improvement in preference rate of our algorithm over the upstream model, i.e., inference model.</div>
|
77 |
</td>
|
78 |
</tr>
|
|
|
81 |
<table class="center">
|
82 |
<tr>
|
83 |
<td width=100% style="border: none">
|
84 |
+
<img src="table2.png" style="width:100%">
|
85 |
<div style="text-align: left; margin-top: 8px;">Performance comparison of MARA against RLHF, DPO, and Aligner measured by percentage improvements of preference rate.</div>
|
86 |
</td>
|
87 |
</tr>
|