 you can even use a learnable weighted average representation aggregator = nn.Conv1d(in_channels=13, out_channels=1, kernel_size=1) weighted_avg_hidden_states = aggregator(time_reduced_hidden_states.unsqueeze(0)).squeeze() print(weighted_avg_hidden_states.shape) 