fp16 is still has the problem
hello, as mentioned in #868 and #871 ,fp16 is broken, and you have fixed in the master once, but i am afraid it also has problem, DP is need after the amp.initialize() too.  i review the code of apex, found that amp do not support the model of parallel type:
```
def check_models(models):
    for model in models:
        parallel_type = None
        if isinstance(model, torch.nn.parallel.DistributedDataParallel):
            parallel_type = "torch.nn.parallel.DistributedDataParallel"
        if isinstance(model, apex_DDP):
            parallel_type = "apex.parallel.DistributedDataParallel"
        if isinstance(model, torch.nn.parallel.DataParallel):
            parallel_type = "torch.nn.parallel.DataParallel"
        if parallel_type is not None:
            raise RuntimeError("Incoming model is an instance of {}. ".format(parallel_type) +
                "Parallel wrappers should only be applied to the model(s) AFTER \n"
                "the model(s) have been returned from amp.initialize.")
```

and the other question is that after i fixed the problem and can run with fp16, but i found it speed the same time and gpu memory, why?

yes, I fixed it in #896 and waiting for author to merge..
but I still can't figure out why fp16 didn't save memory and didn't speed up....<|||||>Merged<|||||>close