It took seven people to help lift the male Staffordshire bull terrier out of the waterway near Litherland Road in Bootle, Liverpool on Saturday.
A blue rope was tied around its neck and tests indicated it had been strangled, the RSPCA said.
An examination at Greater Manchester Animal Hospital found the dog had been fed and walked before it died.
He was in a good condition with short nails and had eaten recently, the animal welfare charity confirmed.
RSPCA Inspector Helen Smith said it was not clear if the dog was alive when it was thrown in the canal.
She said: "If he was dead when he was thrown in, it would have taken at least two people to get him in there as the dog was heavy, as was the rucksack.
"He could have also been walked down there alive, thrown into the canal and choked by the weight around his neck."
Anyone with information is urged to contact the RSPCA.
The body of a dog has been pulled out of a canal with a rucksack full of cement tied to its neck.