tensorboard.compat.tensorflow_stub.errors.AlreadyExistsError: Directory already exists
Hi
I am running finetune_trainer.py on cloud with TPUs, here is the error, I appreciate your help thanks
```json
{
  "textPayload": "Traceback (most recent call last):\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch_xla/distributed/xla_multiprocessing.py\", line 330, in _mp_start_fn\n    _start_fn(index, pf_cfg, fn, args)\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch_xla/distributed/xla_multiprocessing.py\", line 324, in _start_fn\n    fn(gindex, *args)\n  File \"/workdir/seq2seq/finetune_trainer.py\", line 303, in _mp_fn\n    app.run(main, flags_parser=parse_flags)\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/absl/app.py\", line 300, in run\n    _run_main(main, args)\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/absl/app.py\", line 251, in _run_main\n    sys.exit(main(argv))\n  File \"/workdir/seq2seq/finetune_trainer.py\", line 246, in main\n    data_args=data_args,\n  File \"/workdir/seq2seq/seq2seq_trainer.py\", line 37, in __init__\n    super().__init__(*args, **kwargs)\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/transformers/trainer.py\", line 318, in __init__\n    self.control = self.callback_handler.on_init_end(self.args, self.state, self.control)\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/transformers/trainer_callback.py\", line 325, in on_init_end\n    return self.call_event(\"on_init_end\", args, state, control)\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/transformers/trainer_callback.py\", line 376, in call_event\n    **kwargs,\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/transformers/integrations.py\", line 213, in on_init_end\n    self.tb_writer = SummaryWriter(log_dir=args.logging_dir)\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/utils/tensorboard/writer.py\", line 221, in __init__\n    self._get_file_writer()\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/utils/tensorboard/writer.py\", line 252, in _get_file_writer\n    self.flush_secs, self.filename_suffix)\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/utils/tensorboard/writer.py\", line 62, in __init__\n    log_dir, max_queue, flush_secs, filename_suffix)\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/tensorboard/summary/writer/event_file_writer.py\", line 77, in __init__\n    tf.io.gfile.makedirs(logdir)\n  File \"/root/anaconda3/envs/pytorch/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/io/gfile.py\", line 673, in makedirs\n    return get_filesystem(path).makedirs(path)\ntensorboard.compat.tensorflow_stub.errors.AlreadyExistsError: Directory already exists\n",
  "insertId": "5rl5rhwn8m5sobp9q",
  "resource": {
    "type": "k8s_container",
    "labels": {
      "container_name": "seq2seq",
      "location": "europe-west4-a",
      "project_id": "try-ideas-for-rmi",
      "namespace_name": "ruse-xgcp",
      "cluster_name": "xcloud-v3-donut-europe-west4-a",
      "pod_name": "20201105.df.e2753.0-7mxdc"
    }
  },
  "timestamp": "2020-11-05T11:28:19.975413564Z",
  "severity": "ERROR",
  "labels": {
    "k8s-pod/job-name": "20201105.df.e2753.0",
    "k8s-pod/controller-uid": "4974cfc1-b3df-4882-9fd1-9095f4a944d9",
    "k8s-pod/jobowner": "ruse-xgcp",
    "k8s-pod/app": "xcloud",
    "k8s-pod/serviceName": "xc-20201105-df-e2753-0"
  },
  "logName": "projects/try-ideas-for-rmi/logs/stderr",
  "receiveTimestamp": "2020-11-05T11:28:23.810886436Z"
}
```
Well it seems to tell you that your tensorboard directory already exists? Try with a different directory?<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
