self.idx_to_token = list(sorted(set([' '] + reserved_tokens + [ token uchun token, freq in self.token_freqs if freq >= min_freq]))) Men sizdan ushbu kodni tushunishsiz qayta yozishingizni istayman
spam lang_mismatch pii not_appropriate hate_speech sexual_content quality toxicity humor creativity violence