Indoor thermal comfort is essential for occupants' well-being, productivity, and efficiency. Global climate change is leading to extremely high temperatures and more intense solar radiation, especially in hot, humid areas. Passive cooling is considered to be one of the environmental design strategies by which to create indoor thermal comfort conditions and minimize buildings' energy consumption. However, little evidence has been found regarding the effect of passive cooling on the thermal comfort of historical buildings in hot-dry or hot-humid areas. Therefore, we explored the passive cooling features (north-south orientation, natural ventilation, window shading, and light color painted walls) applied in historic residential buildings in Zanzibar and evaluated the residents' thermal responses and comfort perception based on questionnaires and field surveys. The results showed that the average predicted mean votes (PMVs) were 1.23 and 0.85 for the two historical case study buildings; the average predicted percentages of dissatisfaction (PPD) were 37.35% and 20.56%, respectively. These results indicate that the thermal conditions were not within the acceptable range of ASHRAE Standard 55. Further techniques, such as the use of lime plaster, wash lime, and appropriate organization, are suggested for the improvement of indoor thermal comfort in historical buildings in Zanzibar. This study provides guidelines to assist architects in designing energy-efficient residential buildings, taking into account cultural heritage and thermal comfort in buildings.