@Harsh_Thaker thanks for reporting. I was able to repro, looks like the the crash was in pthreadpool when invoking convolution via NNPACK
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x10dcbd3a4)
    frame #0: 0x00000001013a83b4 HelloWorld`compute_input_packing + 256
    frame #1: 0x000000010108b2cc HelloWorld`std::__1::function<void (int, unsigned long)>::operator()(int, unsigned long) const + 48
    frame #2: 0x000000010108b884 HelloWorld`caffe2::ThreadPool::run(std::__1::function<void (int, unsigned long)> const&, unsigned long)::FnTask::Run() + 44
    frame #3: 0x000000010108b530 HelloWorld`caffe2::WorkersPool::Execute(std::__1::vector<std::__1::shared_ptr<caffe2::Task>, std::__1::allocator<std::__1::shared_ptr<caffe2::Task> > > const&) + 304
    frame #4: 0x000000010108b11c HelloWorld`caffe2::ThreadPool::run(std::__1::function<void (int, unsigned long)> const&, unsigned long) + 740
    frame #5: 0x00000001010885e4 HelloWorld`pthreadpool_compute_1d + 84
    frame #6: 0x0000000101087f00 HelloWorld`pthreadpool_compute_2d_tiled + 148
    frame #7: 0x00000001013a68b0 HelloWorld`nnp_convolution_inference + 3168
    frame #8: 0x0000000100aca13c HelloWorld`at::native::_nnpack_spatial_convolution(at::Tensor const&, at::Tensor const&, at::Tensor const&, c10::ArrayRef<long long>, c10::ArrayRef<long long>)::$_0::operator()(unsigned long) const + 680
    frame #9: 0x0000000100ac9284 HelloWorld`at::native::_nnpack_spatial_convolution(at::Tensor const&, at::Tensor const&, at::Tensor const&, c10::ArrayRef<long long>, c10::ArrayRef<long long>) + 1700
    frame #10: 0x0000000100d075a4 HelloWorld`at::TypeDefault::_nnpack_spatial_convolution(at::Tensor const&, at::Tensor const&, at::Tensor const&, c10::ArrayRef<long long>, c10::ArrayRef<long long>) + 248
...

Can you confirm whether you were seeing the same stack trace? If so, I belive that issue has been resolved in

https://github.com/pytorch/pytorch/pull/39868 5
https://github.com/pytorch/pytorch/pull/40951 2

Both of them wiil be merged to our 1.6.0 release.