-
An important difference between machine learning and regression analysis is the way in which new techniques are developed. Inventing a new machine learning method involves proving that it works better in practice. In contrast, inventing a new regression method requires first proving it works in theory. The focus on working in practice gave machine learning innovators more room to experiment, even if their methods generated estimates that were incorrect on average, or biased. This freedom to experiment drove rapid improvements that take advantage of the rich data and fast computers that appeared over the last decade.
To achieve legal standards, humans had to help. In its testing phase, Chisel's machine suggested what to redact, and the human rejected or accepted the suggestion. In effect, working together meant saving a lot of time, while achieving an error rate lower than the humans had achieved on their own. This human-machine division of labor worked because it overcame human weaknesses in speed and attention, and machine weaknesses in interpreting text.
-
In contrast to machines, humans are sometimes extremely good at prediction with little data. We can recognize a face after seeing it only once or twice, even if we see if from a different angle. We can identify a 4th-grade classmate forty years later, despite numerous changes in appearance.
While computer scientists are working to reduce machines' data needs, developing techniques such as "one-shot learning" in which machines learn to predict an object well after seeing it just once, current prediction machines are not yet adequate.
-
In the case of known unknowns, humans understand the inaccuracy of the prediction. The prediction comes with a confidence range that reveals its impression. In the case of unknown unknowns, humans don't think they have any answers. In contrast, with unknown knowns, prediction machines appear to provide a very precise answer, but that answer can be very wrong.
A key point is that, while prediction is a key component of any decision, it is not the only component. The other elements of a decision - judgment, data, and action - remain, for now, firmly in the realm of humans. They are complements to prediction, meaning they increase in value as prediction becomes cheap.
-
Judgment involves determines what we call the "reward function," the relative rewards and penalties associated with taking particular actions that produce particular outcomes.
A little girl asked her father: "Daddy? Do all fairy tales begin with 'once upon a time'?" He replied: "No, there are a whole series of fairy tales that begin with 'If elected, I promise ...'"
-
Action: What are you trying to do? For Atomwise, it is to test molecules to help cure or prevent disease.
Prediction: What do you need to know to make the decision? Atomwise predicts building affinities of potential molecules and proteins.
Judgment: How do you value different outcomes and errors? Atomwise and its customers set the criterion regarding the relative importance of targeting the disease and the relative costs of potential side effects.
Outcome: What are your metrics for task success? For Atomwise, it's the results of the test. Ultimately, did the test lead to a new drug?
Input: What data do you need to run the predictive algorithm? Atomwise uses data on the characteristics of the disease proteins to predict.
Training: What data do you need to train the predictive algorithm? Atomwise employs data on the binding affinity of molecules and proteins, along with molecule and protein characteristics.
Feedback: How can you use the outcomes to improve the algorithm? Atomwise uses test outcomes, regardless of their success, to improve future predictions
-
Machine-learning techniques are increasingly good at predicting missing information, including identification and recognition of items in images. Given a new set of images, the techniques can efficiently compare millions of past examples with and without disease and predict whether the new image suggests the presence of a disease.
Automation that eliminates a human from a task does not necessarily eliminate them from a job.
-
This is the classic "innovator's dilemma," whereby established firms do not want to disrupt their existing customer relationships, even if doing so would be better in the long run. The innovator's dilemma occurs because, when they first appear, innovations might not be good enough to serve the customers of the established companies in an industry, but they may be good enough to provide a new startup with enough customers in some niche area to build a product. Over time, the startup gains experience. Eventually, the startup has learned enough to create a strong product that takes away hits larger rival's customers. By that point, the larger company is too far behind, and the startup eventually dominates. AI requires learning, and startups may be more willing to invest in this learning than their more established rivals.
To figure out if AI is discriminating, you have to look at the output. Do men get different results than women? Do Hispanics get different results than others? What about the elderly or the disabled? Do these different results limit their opportunities?
|
|