Andy_Markman:

transform_value


thanks! i never used the transform_value:
Another issue I spotted with my language model is that when in the training loop: I print model summary: I get:
LanguageModel (
(drop): Dropout (p = 0.2), weights=(), parameters=0
(encoder): Embedding(10000, 200), weights=((10000, 200),), parameters=2000000
(rnn): RecurrentHighwayNetwork (
(highway_layers): ModuleList (
(0): HighwayLayer (
(plain_layer): Linear (200 -> 200)
(transform_layer): Linear (200 -> 200)
)
(1): HighwayLayer (
(plain_layer): Linear (200 -> 200)
(transform_layer): Linear (200 -> 200)
)
)
), weights=((200, 200), (200,), (200, 200), (200,), (200, 200), (200,), (200, 200), (200,)), parameters=160800
(decoder): Linear (200 -> 10000), weights=((10000, 200), (10000,)), parameters=2010000
)
It is taking too long to get to even one epoch… does my model have too many parameters? I am on a i7 cpu.