Social media and volunteer rescue requests prediction with random forest and algorithm bias detection: a case of Hurricane Harvey

AI fairness is tasked with evaluating and mitigating bias in algorithms that may discriminate towards protected groups. This paper examines if bias exists in AI algorithms used in disaster management and in what manner. We consider the 2017 Hurricane Harvey when flood victims in Houston resorted to social media to request for rescue. We evaluate a Random Forest regression model trained to predict Twitter rescue request rates from social-environmental data using three fairness criteria (independence, separation, and sufficiency). The Social Vulnerability Index (SVI), its four sub-indices, and four variables representing digital divide were considered sensitive attributes. The Random Forest regression model extracted seven significant predictors of rescue request rates, and from high to low importance they were percent of renter occupied housing units, percent of roads in flood zone, percent of flood zone area, percent of wetland cover, percent of herbaceous, forested and shrub cover, mean elevation, and percent of households with no computer or device. Partial Dependence plots of rescue request rates against each of the seven predictors show the non-linear nature of their relationships. Results of the fairness evaluation of the Random Forest model using the three criteria show no obvious biases for the nine sensitive attributes, except that a minor imperfect sufficiency was found with the SVI Housing and Transportation sub-index. Future AI modeling in disaster research could apply the same methodology used in this paper to evaluate fairness and help reduce unfair resource allocation and other social and geographical disparities.

Automated summary

This paper investigates the presence of bias in AI algorithms used in disaster management, focusing on the 2017 Hurricane Harvey and Twitter rescue requests. A Random Forest regression model is used to predict rescue request rates, and fairness evaluation is conducted based on three criteria. The results show no obvious biases for most sensitive attributes, except for a minor imperfection in sufficiency for SVI Housing and Transportation sub-index. The authors suggest applying the same methodology to future AI modeling in disaster research to reduce unfair resource allocation and disparities.