"It was only his back that was sticking up so we had to chop the ice to get him out," Janina Przybylski of Stockholm's Western District police told the Aftonbladet newspaper.
There are no suspicions of foul play in connection with the death, with police theorizing the man and his dog fell into the water and drowned prior to the winter freeze.
"It must have happened last autumn," Przybylski told Aftonbladet.
Police estimate the man to be between 50- and 70-years-old, but have yet to identify him.
A review of "countless" missing persons reports failed to turn up any likely matches.
Police hoped an identity chip found in the dog's ear would yield results, but a check of the database revealed the dog had no registered owner.
Investigators have since turned to the general public for help in identifying the man, who had a gray beard and was wearing a white hat with dark blue or black stripes.
He was also wearing a gray fleece sweater and green sweatpants, according to Przybylski.
The medium-sized dog found with the man has a light, yellow-hued coat.
Anyone with any information that may help police identify the man is encouraged to contact the police by calling 114 14.
Follow The Local on Twitter