MERaLiON

/

MERaLiON-AudioLLM-Whisper-SEA-LION

like 16

Follow

MERaLiON 57

On my m3 pro, I running the inference script gives the error. (Note I get same error without using the device_map param. )

transformers version :: 4.51.3.

AttributeError                            Traceback (most recent call last)
Cell In[4], line 6
      3 audio_array = [audio_array]*2
      4 inputs = processor(text=chat_prompt, audios=audio_array)
----> 6 outputs = model.generate(**inputs, max_new_tokens=256) # max_new_tokens=256
      7 generated_ids = outputs[:, inputs['input_ids'].size(1):]
      8 response = processor.batch_decode(generated_ids, skip_special_tokens=True)

File ~/projects/nepse-analytics-backend/.venv/lib/python3.9/site-packages/torch/utils/_contextlib.py:116, in context_decorator.<locals>.decorate_context(*args, **kwargs)
    113 @functools.wraps(func)
    114 def decorate_context(*args, **kwargs):
    115     with ctx_factory():
--> 116         return func(*args, **kwargs)

File ~/projects/nepse-analytics-backend/.venv/lib/python3.9/site-packages/transformers/generation/utils.py:2465, in GenerationMixin.generate(self, inputs, generation_config, logits_processor, stopping_criteria, prefix_allowed_tokens_fn, synced_gpus, assistant_model, streamer, negative_prompt_ids, negative_prompt_attention_mask, use_model_defaults, **kwargs)
   2457     input_ids, model_kwargs = self._expand_inputs_for_generation(
   2458         input_ids=input_ids,
   2459         expand_size=generation_config.num_return_sequences,
   2460         is_encoder_decoder=self.config.is_encoder_decoder,
   2461         **model_kwargs,
   2462     )
   2464     # 12. run sample (it degenerates to greedy search when `generation_config.do_sample=False`)
-> 2465     result = self._sample(
   2466         input_ids,
...
    509     }
    510 )
    512 # Input ids will only be used from the second step. 

AttributeError: 'HybridCache' object has no attribute 'get_max_length'