30b and 32b version?
it's been 5 days, is it going to be releasing, or no? and also, really apreciate the work you are doing for the community. 💐
Yea definitely will be released, but this will take little time. The MoE is harder to abliterate, and so the code needs some significant changes. However the 32B will be uploaded on Friday.
Yea definitely will be released, but this will take little time. The MoE is harder to abliterate, and so the code needs some significant changes. However the 32B will be uploaded on Friday.
Great news. Looks like your work is on the front page of r/Locallama! Tried the 8B model when I saw it on reddit and I was extremely impressed.
Outstanding quality with the 8B. I generally test all the "uncensored" models - yours takes the cake.
14B version? Just wondering.
As per other comments: Really stoked for the 32B version too.
RE: MOE/30B - this one is an odd one . Almost impossible to imatrix. (an issue I have never had with over 500+ models).
Consulted with other builders:
-> need artificially generated data at high temp + high rep pen from the model -> hopefully this might help fine tuning it (?)
Thanks for all you do.
@DavidAU Thanks a loot!!! but yes the MoE is also hard to abliterate, I actually have to change the code to also get the correct experts to update. The 14B will come later this week.
RE: 30BA3B:
Increase the number of active experts => far easier to work with.
This was key for the "imatrixing" issues.