As reading through Segur, there is this remark after Battle of Smolensk (Aug 1812):
"At Vilna we had been able to establish hospitals for only six thousand patients. Convents, churches, synagogues, and barns had been requisitioned to house this multitude of sufferers. In these wretched quarters, often unwholesome, always overcrowded, the sick commonly went without food, bed, blankets, straw and medicine. There were soon not enough surgeons, so that everything, even the hospitals themselves, tended to breed disease, and nothing to cure it."
From other accounts it seems that history has forgotten the enormous loss of life prior to the retreat. In any account of the grave site in Lithuania, was there an explanation of why people have concluded the men were lost in the retreat? Maybe they had died from disease (reported has being terrible) while the rest of the Grand Armee was advancing on Moscow.