Thanks for your reply.
I haven’t thought too much about the implementation details of ValueChoice.  My idea is that ValueChoice receives a list of candidate values, and it can return a value selected from the candidates. Therefore, we can use it to search an optimal architecture.
Searchable means that ValueChoice([3,5,7])  can return a value of 3, 5, or 7 according to the predefined search strategy. For example, nn.conv2d(3, 16, ValueChoice([3,5,7])) can be decoded as nn.conv2d(3, 16, 3), nn.conv2d(3, 16, 5), or nn.conv2d(3, 16, 7).
I have two questions about ValueChoice implementation, and I wish you can give me some suggestions:

how to decode ValueChoice into a value
during the search process, ValueChoice  returns a different value at different epoch. My second question is how to change or inherit weights of modules. For example, at the 1-st epoch,  ValueChoice returns 3, then we obtain nn.conv2d(3, 16, 3); but if ValueChoice returns 5 at the 2-nd epoch, then we obtain nn.conv2d(3, 16, 5). Can these two convolutional operations share weights?