The society and culture we know now was built and sustained by underclasses of underpaid, overworked people who were forced into non-ideal situations and taken advantage of. Slavery is the ultimate example, but this has occurred again and again with black people, immigrants, and other minorities.
There are all sorts of systems and institutional barriers that have worked for centuries to ensure that large segments of our society—regardless of talent, skill, or character—will never be allowed to rise out of poverty or powerlessness. This country’s wealth was built on exploitation and violence, and those who worked hardest to build it were not empowered or enriched by its successes—they were enslaved people, migrant laborers, and domestic workers. Much of this country’s early infrastructure, for example, was built with slave labor, and then with grotesquely underpaid immigrant labor and prison labor. — Mediocre, pg 14
See also the black labor used to build the Panama Canal, The Path Between the Seas.