What effect does Varroa have on roof top colonies?

Two out of the three established colonies in my roof top apiary are big and vigorous. Meanwhile Hive B has stayed small since it was caught as a swarm a year ago. Why?

I've been told that some colonies are smaller than others. The bees like it that way. I'm willing to trust the bees to right-size their colony but I'd like to understand what causes the differences. I wonder whether Varroa and its diseases might be an explanation.

Since moving Hive B back home to ground level I have seen evidence of comb building in Hive B. This is co-incident with a nectar flow so I'd expect there to be some increase in storage space. I have also noticed an increase in the defensiveness and a high rate of Varroa mite drop on the hive floor, relative the colony size.

Varroa suck the bee's hemolyph whilst they are pupating in their brood cells and later while the mites cling on adult bees. The hemolyph is bee blood. Removing it weakens the bee, as well as transmitting viruses. My suspicion is that this colony in Hive B has a low resistance to the mite and is being progressively weakened by their action. Bees returning to roof top hives from ground level will have to ascend over 20 metres whilst carrying a full load of nectar. If the weaker bees were being lost this would limit the number of foragers. Younger bees would be promoted to foragers which limits the number of nurse bees available to care for brood. This would give a mechanism which explains why this colony, which superseded last year and are apparently happy with their Queen, haven't built up.

I will be watching whether Hive B builds up strongly at ground level and continues to shed significant numbers of mites. This would strengthen my suspicion that they lack the resistance to mite attack demonstrated in Hives C and D.