<h3 dir="auto">Feature request</h3>
<p dir="auto">##transformers.AutoModel.from_pretrained()##  module allows to load pretrained models from local directories as well. The local pickle files path is an argument of the above function. These files can have different encoding types.</p>
<p dir="auto">A parameter called 'encoding' can be added in the parameter list similar to</p>
<p dir="auto">pandas.read_csv('path/to/csv/file',encoding='utf-8')</p>
<p dir="auto">which takes encoding type as parameter. But this proposed encoding feature must be enabled only while loading local files and should not take parameters while loading models from other sources (like HF-Hub etc).</p>
<h3 dir="auto">Motivation</h3>
<p dir="auto">I face Unicode-Decode error while loading pickled model files from local directory. To overcome this issue, the above feature would help to avoid the unicode-decode error.</p>
<h3 dir="auto">Your contribution</h3>
<p dir="auto">Unfortunately, I can't contribute now.</p>
<p dir="auto">Dear HuggingFace team,</p>
<p dir="auto">There is unfortunately no option to reopen the bugs, the issue I reported here [1] still exists with testing the last version of transformers. I add my comments on the same ticket. Could you kindly reopen this bug?</p>
<p dir="auto">The variations are very high after resume, which makes the results not usable with resuming after checkpoint, I also tried to make things deterministic in torch, but it also could not solve the issue. I study in an institution where I only have access to GPUs for short hours, and I very much appreciate your help on this issue to make training reproducible after resume from the checkpoints in trainer.</p>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/sgugger/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/sgugger">@sgugger</a></p>
<p dir="auto">[1] <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="861744931" data-permission-text="Title is private" data-url="https://github.com/huggingface/transformers/issues/11323" data-hovercard-type="issue" data-hovercard-url="/huggingface/transformers/issues/11323/hovercard" href="https://github.com/huggingface/transformers/issues/11323">#11323</a></p>