I was working on an interesting problem of generating inferences from the excel data. I wrote a python program to generate rules from the data in the form of RDF Triple and now training using T5-Base model. with some 10k training data of rdf rules and inferences I was able to get some 80% to 85% test accuracy. I’m using ADAMW optimizer with lr of 1e-5.
One issue I have seen is the model is not able to generalize well on new numbers. eg. if I pass a rule of “Critical” | priority_ticketshare | “23.09%” to the model, it returns inference as Critical priority tickets accounted for 22.09% of total tickets. While the statement is correct, the number it has taken is wrong. Any idea how to solve this