Authorities recovered about $120,000 worth of ivory from 17 tusks, the Zimbabwe Chronicle reported.
The men stood accused of lacing watering holes frequented by the elephants.
“What they were doing is very cruel because it does not end with the death of the elephants,” Chief Inspector Muyambirwa Muzzah told the Chronicle.
“We have what we call the fourth generation effect due to the potency of cyanide as a poison. Animals that feed on the dead elephants will die, and those that feed on the dead animals will also die.”
The elephants were found in the Hwange National Park after a ranger heard gunshots.
The park's spokeswoman Caroline Washaya Moyo told BBC News that tests would confirm cyanide was in fact used to kill the animals.
That 41 elephants have died makes it Zimbabwe’s worst case of poaching, Moyo said.
The Mail and Guardian in South Africa said elephant poaching was on the rise.
Ivory trade has doubled since 2007, the Mail and Guardian said, citing a report from this year’sConvention on International Trade in Endangered Species.
The report said poachers killed 25,000 African elephants last year, while poachers kill 30 elephants in Tanzania every day.
It tied the rise in poaching to rising poverty levels.