__init__() got an unexpected keyword argument 's2t_train_config'
Getting this error when I use the demo on the HF model card page.
__init__() got an unexpected keyword argument 's2t_train_config'
Can you try the code snippets included in the README? The HF auto-generated demo code likely won't work since the model is not upstreamed to the main ESPnet repo yet.
Facing a similar issue with the README code snippet as well. The trace is below. I think this occurs because the instructions have us install espnet from the ssl branch ( pip install 'espnet @ git+https://github.com/wanchichen/espnet.git@ssl' ) which doesn't support the 'normalize_output' keyword argument for CNNFrontend ( https://github.com/wanchichen/espnet/blob/ssl/espnet2/asr/frontend/cnn.py ). The master branch has that argument, but it doesn't work, which branch do you suggest we use?
xeus_model, xeus_train_args = SSLTask.build_model_from_file(
... None,
... "....../XEUS/model/xeus_checkpoint_new.pth",
... device,
... )
Traceback (most recent call last):
File "", line 1, in
File "....../miniconda3/envs/test/lib/python3.11/site-packages/espnet2/tasks/abs_task.py", line 2137, in build_model_from_file
model = cls.build_model(args)
^^^^^^^^^^^^^^^^^^^^^
File "....../miniconda3/envs/test/lib/python3.11/site-packages/espnet2/tasks/ssl.py", line 406, in build_model
frontend = frontend_class(**args.frontend_conf)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: CNNFrontend.init() got an unexpected keyword argument 'normalize_output'
Tried using the main branch, but this error still remained:
RuntimeError: Error(s) in loading state_dict for ESPnetSSLModel:
Missing key(s) in state_dict: "encoder.embed.0.conv.bias", "encoder.embed.0.conv.weight_g", "encoder.embed.0.conv.weight_v", "losses.0.decoder.label_embeddings", "masker.mask_embedding".
Unexpected key(s) in state_dict: "global_step", "util_modules.mask.mask_embedding", "encoder.embed.0.convs.0.bias", "encoder.embed.0.convs.0.weight_g", "encoder.embed.0.convs.0.weight_v", "losses.0.decoder.label_embeddings.weight", "losses.0.decoder.label_embeddings.bias".
I also tried initializing by pointing to the config_file instead of the model_file (making that None), but when xeus_model.encode is called it just crashes in a cryptic fashion.
zsh: killed python
Environment: Mac running python 3.11, PyTorch 2.7.1 within a conda environment (lmk if you recommend a different environment)
Hey, sorry about the issues! We are in the middle of updating the codebase on master to support XEUS and not use my branch. It's still a bit WIP, haven't finished testing yet. Still pending 1 or 2 PRs.
In the mean time, the current instructions should work with my branch if you use xeus_checkpoint_old.pth
xeus_model, xeus_train_args = SSLTask.build_model_from_file(
None,
'/path/to/checkpoint/here/xeus_checkpoint_old.pth',
device,
)