Aggressive deep layer compression of Mistral Large 2411
Detected 88 source layers.
Surgery Plan: 63 operations.
Merging Block 47-72 into 5 layers...
> Compressing 26 layers -> 5 layers
Copying Embeddings, Norms, and Head...
Saving Index...
Updating Config...
DONE! New Model Depth: 67 Layers
Loading Embeddings...
Layer | BI Score | Weight Sim | Verdict
--------------------------------------------------
0 | 0.5022 | N/A |
1 | 0.9008 | 0.5000 | Active Layer
2 | 0.8713 | 0.7422 | Active Layer
3 | 0.9583 | 0.7109 | Active Layer
4 | 0.9647 | 0.8984 | Active Layer
5 | 0.9697 | 0.9062 | Active Layer
6 | 0.9531 | 0.9102 | Active Layer
7 | 0.9682 | 0.9219 | Active Layer
8 | 0.9694 | 0.9219 | Active Layer
9 | 0.9449 | 0.9219 | Active Layer
10 | 0.9621 | 0.9141 | Active Layer
11 | 0.9635 | 0.9141 | Active Layer
12 | 0.9480 | 0.9336 | Active Layer
13 | 0.9689 | 0.9375 | Active Layer
14 | 0.9666 | 0.9375 | Active Layer
15 | 0.9658 | 0.9336 | Active Layer
16 | 0.9609 | 0.9336 | Active Layer
17 | 0.9651 | 0.9414 | Active Layer
18 | 0.9645 | 0.9414 | Active Layer
19 | 0.9633 | 0.9375 | Active Layer
20 | 0.9644 | 0.9414 | Active Layer
21 | 0.9571 | 0.9453 | Active Layer
22 | 0.9557 | 0.9492 | Active Layer
23 | 0.9326 | 0.9492 | Active Layer
24 | 0.9488 | 0.9492 | Active Layer
25 | 0.9479 | 0.9531 | Active Layer
26 | 0.9613 | 0.9492 | Active Layer
27 | 0.9586 | 0.9492 | Active Layer
28 | 0.9398 | 0.9492 | Active Layer
29 | 0.9459 | 0.9453 | Active Layer
30 | 0.9445 | 0.9492 | Active Layer
31 | 0.9378 | 0.9531 | Active Layer
32 | 0.9465 | 0.9531 | Active Layer
33 | 0.9487 | 0.9531 | Active Layer
34 | 0.9458 | 0.9570 | Active Layer
35 | 0.9502 | 0.9609 | Active Layer
36 | 0.9221 | 0.9609 | Active Layer
37 | 0.9402 | 0.9609 | Active Layer
38 | 0.9406 | 0.9609 | Active Layer
39 | 0.9464 | 0.9648 | Active Layer
40 | 0.9593 | 0.9648 | Active Layer
41 | 0.9596 | 0.9648 | Active Layer
42 | 0.9559 | 0.9648 | Active Layer
43 | 0.9710 | 0.9688 | Active Layer
44 | 0.9792 | 0.9688 | Active Layer
45 | 0.9820 | 0.9727 | Possible Merge
46 | 0.9580 | 0.9727 | Active Layer
47 | 0.9874 | 0.9727 | Possible Merge
48 | 0.9876 | 0.9688 | Possible Merge
49 | 0.9894 | 0.9688 | Possible Merge
50 | 0.9877 | 0.9648 | Possible Merge
51 | 0.9883 | 0.9688 | Possible Merge
52 | 0.9897 | 0.9727 | Possible Merge
53 | 0.9886 | 0.9727 | Possible Merge
54 | 0.9882 | 0.9688 | Possible Merge
55 | 0.9864 | 0.9688 | Possible Merge
56 | 0.9885 | 0.9727 | Possible Merge
57 | 0.9868 | 0.9688 | Possible Merge
58 | 0.9899 | 0.9688 | Possible Merge
59 | 0.9871 | 0.9688 | Possible Merge
60 | 0.9903 | 0.9727 | MERGE CANDIDATE
61 | 0.9888 | 0.9766 | Possible Merge
62 | 0.9878 | 0.9727 | Possible Merge
63 | 0.9899 | 0.9727 | Possible Merge
64 | 0.9885 | 0.9727 | Possible Merge
65 | 0.9854 | 0.9727 | Possible Merge
66 | 0.9886 | 0.9727 | Possible Merge
67 | 0.9837 | 0.9688 | Possible Merge
68 | 0.9894 | 0.9727 | Possible Merge
69 | 0.9879 | 0.9766 | Possible Merge
70 | 0.9872 | 0.9688 | Possible Merge
71 | 0.9809 | 0.9727 | Possible Merge
72 | 0.9827 | 0.9766 | Possible Merge
73 | 0.9789 | 0.9766 | Active Layer
74 | 0.9790 | 0.9766 | Active Layer
75 | 0.9692 | 0.9844 | Active Layer
76 | 0.9611 | 0.9766 | Active Layer
77 | 0.9762 | 0.9727 | Active Layer
78 | 0.9757 | 0.9727 | Active Layer
79 | 0.9729 | 0.9766 | Active Layer
80 | 0.9744 | 0.9766 | Active Layer
81 | 0.9677 | 0.9727 | Active Layer
82 | 0.9665 | 0.9766 | Active Layer
83 | 0.9643 | 0.9766 | Active Layer
84 | 0.9571 | 0.9727 | Active Layer
85 | 0.9287 | 0.9727 | Active Layer
86 | 0.8072 | 0.9766 | Active Layer
87 | 0.6603 | 0.9609 | Active Layer
- Downloads last month
- 27
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
Model tree for schonsense/fogstral_compressed
Base model
mistralai/Mistral-Large-Instruct-2411