Condensation can occur when air is moving through a duct into the fireplace through a heated space and can cause the duct to sweat or have condensation form on the duct which can cause issues such as creating frost on the pipe which will melt and can cause water damage. Its common practice to insulate the duct so there is no chance of issues.
