This month, Amazon shut down a project it had been developing for four years — a recruitment tool driven by machine learning. The concept was a simple and appealing one: at its core, the project aimed to develop an algorithm that would sort incoming job applications to isolate the short list for managers to use in making their final selections.
Anyone who has been involved in such a process knows that isolating the top five or ten resumés from dozens of applicants is a time-consuming job. Any process that brings logic and speed to this stage of the recruitment chain can only be a good thing.
But algorithms are only as good as the data used to drive them, and in the Amazon case the machine “learning” was based on patterns in applications submitted to the firm in the previous ten years. Since Amazon, like all tech firms, is male-dominated, the algorithm taught itself that success equated with male-oriented activities and language, so phrases like “women’s chess club captain” or the names of all-women colleges, were penalised by the algorithm in ways that technical fixes could not remedy.
This episode exemplifies many of the appealing and unappealing features of the data-driven era that is rapidly coming upon us. The appeal is intuitive. If tasks, even complex ones, can be completed rapidly, accurately, and effectively by a tool that improves itself over time, then a range of tedious and expensive chores currently handled by humans can be outsourced, not to cheaper humans far away, but to software embedded right in your office.
However, there are also three broad sets of issues that give pause for thought and where globally agreed norms and rules — in a word, governance — is called for.
Source : Our World