Floods due to heavy rains or typhoons are frequent annual hazards in Japan. This study aims to reduce disaster fatalities and contribute to disaster risk reduction. This retrospective observational study analyzed fatalities caused by heavy rains or typhoons. In Japan, 578 fatalities, related to seven occurrences of heavy rains and 16 typhoons, occurred between 2016 and 2020. Moreover, 13,195 houses collapsed due to hazards. Furthermore, 334 (73.2%) of the 456 fatalities were > 60 years old. Heavy rains caused more local area destruction due to floods and landslides than typhoons although wind- and disaster-related mortalities were found to be caused by typhoons. Human damage was eminent in older people because of their vulnerabilities and possibly dangerous behavior. Many fatalities were due to floods (46.9%) and landslides (44.1%). Indoor and outdoor mortalities due to heavy rains or typhoons were 157 (55.9%) and 124 (44.1%), respectively, and 24 (21.8%) of 124 outdoor mortalities occurred in vehicles. The number of recent flood mortalities in Japan correlates with the number of destroyed houses. Analyzing the victim's locations in the 2020 Kumamoto Heavy Rain using hazard and inundation maps suggested the difficulty of ensuring the safety of people living in dangerous areas. This study showed the characteristics of flood damage by heavy rains and typhoons in Japan and reports that flood damage is increasing because of the hazard size and community aging. Disaster risk reduction, disaster education, and evacuation safety plans for the elderly using hazard maps were important for strengthening disaster resilience.