Hi all,
I found a stupid but workable solution for this: just write two new activation module classes that inherit torch::nn::FunctionalImpl, and then override their name_ in the constructor. The code is like following:
struct ActivationTanhImpl : torch::nn::FunctionalImpl {
public:
	ActivationTanhImpl() :
		torch::nn::FunctionalImpl(torch::tanh),
		torch::nn::Module("torch::nn::TanhImpl") {}
};
TORCH_MODULE(ActivationTanh);

struct ActivationReLUImpl : torch::nn::FunctionalImpl {
public:
	ActivationReLUImpl() :
		torch::nn::FunctionalImpl(torch::relu),
		torch::nn::Module("torch::nn::ReLUImpl") {}
};
TORCH_MODULE(ActivationReLU);

I know this is not elegant solution, but currently that’s the only workable one I can come out. Please let me know if you have any suggestions for this question. Thanks again!