Amazon scraps 'sexist AI' recruiting tool that showed bias against women

Amazon chief executive Jeff Bezos

Amazon chief executive Jeff Bezos

"Because the company has hired so many male engineers or software developers, data scientists and so forth, that clearly the unsuccessful candidates were ones that would have this word 'women's" in it".

The tech giant Amazon shutdown an AI recruitment system that it developed when it couldn't stop the tool from discriminating against women.

A large number - including Hilton and Goldman Sachs - are turning to machine learning options to automate their recruitment process, either in-house or through recruitment as a service solutions. Having machines and AI play some role in the recruitment process would seemingly ease the pressure on Amazon's human work force. But quite quickly, the team realized a big problem: the program had taught itself some seriously questionable hiring practices, prioritizing male candidates, and masculine language, over women. Their goal was to develop AI that could rapidly crawl the web and spot candidates worth recruiting, the people familiar with the matter said.

They created 500 computer models to trawl through past candidates' resumes and pick up on around 50,000 key terms. Well, thanks to it being trained on 10 years of application data that contains mostly male applicants, the system ended up favouring men over women, reported Reuters, which spoke to folks familiar with the AI. It penalized resumes that included the word "women's", as in "women's chess club captain". The AI is also said to have downgraded graduates of two all-women's colleges.

Amazon tried to edit the program to make it more gender neutral but not being able to guarantee non-discrimination, the company ultimately disbanded the team working on the system and limited its usage.

Some 55% of U.S. human resources managers said Artificial Intelligence, or AI, would be a regular part of their work within the next five years, according to a 2017 survey by talent software firm CareerBuilder. The American Civil Liberties Union is now challenging a law that allows criminal prosecution of researchers and journalists who test hiring websites' algorithms for discrimination.

The firm now uses a "much-watered down version" of the recruiting engine to carry out "rudimentary chores".

Amazon, a source said, has a new team assembled to give online screening another try, this time with a focus on diversity.