Climate may significantly affect human society. Few studies have focused on the temperature impact on residents' health, especially mental health status. This paper uses 98 423 observations in China to study the relationship between temperature and health, based on the China Family Panel Studies survey during 2010-16. We analyze the health effects of extreme hot and cold weather and compare the effects under different social demographic factors including gender, age, and income. We find that temperature and health status exhibit a nonlinear relationship. Women and low-income households are more likely to be impacted by extreme cold, whereas men, the elderly, and high-income households are more sensitive to extreme heat. Our results highlight the potential effects of extreme temperatures on physical and mental health and provide implications for future policy decisions to protect human health under a changing climate.