I think a flag like this might be most useful for inference jobs, where we care exclusively about performance, without regard for training behavior. You’re right - adjusting batch size for users automatically will have side effects for training; but we can avoid this issue by at least narrowing the scope to inference.
Additionally, tuning num_workers would improve performance for both training and inference, especially for large scale (GB/TB) inference jobs.
Also, experimentally, it seems that large batch sizes tend to yield the best speedups, up until data can no longer fit in memory.