Will training code for dialogue model be released?
#1
by
dingdong234
- opened
Wanna try custom data on dialogue model
Hi,there is no plan to open-source the training code for the time being. Below is the code for calculating loss:”
# teacher_vectors is from infgrad/stella-large-zh-v3-1792d
cosine_loss = 1 - (student_vectors * teacher_vectors).sum(axis=1).mean()
cosine_loss = cosine_loss * 10.0
sim_value_loss = F.mse_loss(
input=torch.matmul(student_vectors, student_vectors.T),
target=torch.matmul(teacher_vectors, teacher_vectors.T),
) * 100.0
final_loss = cosine_loss + sim_value_loss
The whole process is very simple, good luck
thank u,it helps a lot~
dingdong234
changed discussion status to
closed