A possible bug in the transformers logging file 
### System Info

In this file https://github.com/huggingface/transformers/blob/main/src/transformers/utils/logging.py there is a bug that I am not sure of so I wanted to discuss it here before sending a pull request in case I misunderstood anything.

In this function:

```
def remove_handler(handler: logging.Handler) -> None:
    """removes given handler from the HuggingFace Transformers's root logger."""

    _configure_library_root_logger()

    assert handler is not None and handler not in _get_library_root_logger().handlers
    _get_library_root_logger().removeHandler(handler)
```

Shouldn't it be

```
assert handler is not None and handle in _get_library_root_logger().handlers
```

Because if the handler is asserted to not be in the `_get_library_root_logger().handlers`, how will it be removed from the `_get_library_root_logger`?

### Who can help?

@Phi

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Use this test case:

```
def test_remove_handler():
    # Create a sample handler
    handler = logging.StreamHandler()
    library_logger = _get_library_root_logger()
    library_logger.addHandler(handler)

    # Ensure the handler is in the library logger's handlers list
    assert handler in library_logger.handlers

    # Call remove_handler and pass in the handler as an argument
    remove_handler(handler)

    # Check that the handler is no longer in the library logger's handlers list
    assert handler not in library_logger.handlers
```

### Expected behavior

I expect that there will be no `AssertionError` on testing this test case because I simply added a handler then removed it.
cc @LysandreJik <|||||>Oh yeah great catch @bahgat-ahmed ! I think it was introduced by mistake in https://github.com/huggingface/transformers/pull/10633. 

I think some changes need to be done in this module:
- This needs to be updated
- The assertions need to be updated to `ValueError`s. 

Would you like to try your hand at it @bahgat-ahmed? Happy to guide you if you'd like.<|||||>Thanks a lot @LysandreJik for this, and for offering the guide! Definitely, happy to try my hand and help the community.

So I think this is the update that you want, right?

```
def remove_handler(handler: logging.Handler) -> None:
    """Removes given handler from the HuggingFace Transformer's root logger."""

    _configure_library_root_logger()

    if handler is None:
        raise ValueError("Handler is None")

    if handler not in _get_library_root_logger().handlers:
        raise ValueError("Handler is not in the list of handlers")

    _get_library_root_logger().removeHandler(handler)
```

If that is okay, tell me and I will make a Pull Request for this modification.

But do you see that it is better to also convert all `assertions` to `ValueErrors` in all the functions in this same file?

By the way, sorry for my late response. It is my first issue to open on GitHub :), so I thought that your answer to my issue will be sent as a notification on GitHub, but I didn't receive any notification. So, I have just checked now my issue and found your comments :)<|||||>That looks correct to me! If you want to update all assertions in the file, that would be welcome, but then I would split the PR in two: one PR focusing on the errors, and another one focusing on the initial bug you reported.

Does that sound good to you? Please ping me on the PR(s) and I'll be happy to review. Thanks!<|||||>Yes, sure sounds good, @LysandreJik . I can even send separate pull requests from my side. The first one for the bug, and the second one for updating all assertions. If that sounds good to you, just confirm to me :) 

You're most welcome, @LysandreJik . And thank you for your support :)<|||||>> Yes, sure sounds good, @LysandreJik . I can even send separate pull requests from my side. The first one for the bug, and the second one for updating all assertions. If that sounds good to you, just confirm to me :)
> 
> You're most welcome, @LysandreJik . And thank you for your support :)

The aim is to disable log output to the console because I want to implement an exe wrapper that does not display cmd on the command line,what should I do?
![image](https://user-images.githubusercontent.com/48466610/219825555-4a67d66f-f87b-4dce-9047-6af68f362644.png)

https://github.com/huggingface/diffusers/issues/2394<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.