Thank you for your help
I printed the targets values for the first epoch
tensor([111, 695, 202, 720, 217, 434, 471, 625, 379, 201,   3, 273, 469, 147,
        180, 141, 378, 326, 418, 725,  32, 225, 176, 297, 452, 328, 264, 208,
        441, 689, 580, 155], device='cuda:0')
tensor([260, 129, 505, 586, 529, 662, 570, 289, 391, 702, 433, 228, 139,  66,
         52, 732, 678, 437, 619, 526, 369, 100, 709, 181, 402, 460, 231, 492,
        516, 374, 440, 299], device='cuda:0')
tensor([157, 412, 149, 651, 669,   5,  31, 632, 680, 145, 103, 478, 213, 115,
        576, 738, 253,  17, 182, 327, 464, 252, 590, 706,  51, 238,   6, 553,
        107, 302, 240, 721], device='cuda:0')
tensor([532, 491, 693, 664, 567, 530, 135, 150, 515, 436, 119, 300, 655, 628,
        604,  23, 683, 332, 204, 481, 387, 315, 153, 750, 518, 456, 222, 511,
        200, 218,  36, 271], device='cuda:0')
tensor([739, 631, 599, 110, 601, 746, 183, 400, 520,  91, 169,   2, 239, 422,
        286, 455, 639, 483, 280, 479, 741, 340,  59, 524, 510, 396, 211, 144,
        419, 508, 577, 288], device='cuda:0')
tensor([125,  39,  63,  96, 519, 366, 463, 417,  55,  81, 574, 350, 197, 684,
         18, 334, 428, 597, 185, 338, 295, 493, 215,  76, 159, 488, 188, 536,
        555, 710,  38, 408], device='cuda:0')
tensor([283, 151,   9, 548, 486,  82, 186, 236, 652, 623, 170,  45, 626, 457,
        504, 609, 514, 281, 163, 318, 591, 177, 560, 672, 588, 267, 700, 542,
        195, 444, 256, 635], device='cuda:0')
tensor([353, 311, 376, 371, 677,  14, 321, 320, 282,  72, 291, 665, 543, 259,
        551, 584, 733,  97, 301,  86, 679, 410, 600,  48,  19, 336, 189, 317,
        346, 716, 610, 735], device='cuda:0')
tensor([ 24, 223, 521, 257, 382, 569, 415, 587, 290,  69, 137, 312, 337, 734,
        232, 160, 717, 503, 713, 612,  21, 199, 158, 643, 575, 620, 594, 388,
        210,  64, 568, 556], device='cuda:0')
tensor([262,  79, 657,  92, 461,  54, 166, 187, 429, 308,   1, 485, 666,  42,
        496, 352,  37, 497, 675, 356, 203, 431, 482, 697, 243, 547, 305, 196,
        468, 314, 432,   8], device='cuda:0')
tensor([644,  11, 737, 322, 355, 383, 581,  95, 642, 335,  56, 743, 636, 686,
        659, 673, 397, 138, 292,  50,  98, 613, 279,  78, 358, 132, 549, 707,
        633, 681, 194, 390], device='cuda:0')
tensor([389, 454, 539, 540, 130, 480, 624, 426, 233, 167, 362, 459, 660, 123,
         88,  71, 699, 685, 435, 172, 219, 255, 127, 365, 656, 668, 269, 490,
        618, 310, 101,  44], device='cuda:0')
tensor([650, 152, 105, 506, 663, 168, 298, 270, 641, 142, 611, 446, 489, 647,
        407, 126, 607, 670,  70, 373, 557, 726, 598, 209,  87, 494, 227, 339,
        319, 714, 304, 272], device='cuda:0')
tensor([416, 525, 124, 722, 424, 671, 140,  35, 276, 667, 254, 622, 544, 742,
        582, 731,  58, 381,  74, 592, 438, 285, 749, 361, 206, 370, 487,  65,
        447, 134, 747, 414], device='cuda:0')
tensor([661, 349, 637, 698, 629, 676, 405,  10, 104, 143, 708, 694, 701, 146,
         89, 306, 303, 224, 534, 404, 451, 559, 207, 263,  43, 242, 538, 646,
        727, 386, 658, 385], device='cuda:0')
tensor([248, 357,  90, 564, 120, 164, 498, 608, 522, 472, 220, 736, 345, 509,
        293, 212, 234, 325, 616, 606, 589, 393, 528, 348, 102, 347,  61, 423,
        705, 718, 696, 596], device='cuda:0')
tensor([719, 343,  99, 192, 499, 674, 740, 723, 284, 122, 430, 501, 605, 109,
         77, 113, 579, 537, 112, 171, 333, 545, 561,  85,  34, 341,  84, 406,
        654,  20, 323, 316], device='cuda:0')
Time taken: 14.64 sec.

-----------------------------------
Epoch: [2/150]
tensor([ 57, 159, 486, 182,  69, 625, 607, 370, 742, 559, 406, 271, 174, 173,
        347, 338, 427, 743,  62,  74, 396, 185, 693, 603, 168, 731, 462, 266,
         32, 143,  70, 122], device='cuda:0')
tensor([ 68, 386, 648,  54, 456, 301, 366, 536, 204, 732, 523, 748, 481, 653,
        435, 468, 155, 272, 141,  50, 690, 516, 440, 264, 429, 639, 258,  41,
        539, 545, 205, 333], device='cuda:0')


No target value is less than 0 or greater than 750. I have 751 classes.
I don’t understand how criterion will know about n_classes. It’s neither defined during the initilization, criterion = nn.CrossEntropyLoss()  and the error happens during the first call of loss = criterion(outputs, targets)
On CPU, it gives this error
  File "train.py", line 142, in train
    loss_id = criterion(outputs, targets)
  File "/home/fstu1/miniconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/fstu1/miniconda3/lib/python3.6/site-packages/torch/nn/modules/loss.py", line 862, in forward
    ignore_index=self.ignore_index, reduction=self.reduction)
  File "/home/fstu1/miniconda3/lib/python3.6/site-packages/torch/nn/functional.py", line 1550, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/home/fstu1/miniconda3/lib/python3.6/site-packages/torch/nn/functional.py", line 1407, in nll_loss
    return torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
RuntimeError: Assertion `cur_target >= 0 && cur_target < n_classes' failed.  at /opt/conda/conda-bld/pytorch_1532581333611/work/aten/src/THNN/generic/ClassNLLCriterion.c:93