An essay argues that utilitarianism, widely held among AI company employees, poses risks to humanity's future. The philosophy's principle of species impartiality could theoretically recommend replacing humans with AIs if that maximized total wellbeing. The authors contend that utilitarian beliefs are sincerely held by influential AI developers and could jeopardize humanity.