Tag Archives: artificial intelligence

WTF is AI?

The term “artificial intelligence” (AI) is popping up everywhere from e-discovery to beer brewing. What do you think of when you hear it? Is it sexy and cutting-edge? This article explores what AI really is.

Virtually every computer program can be viewed as taking some input, applying some procedure, and generating some output. A program might take a number as input and output the square root of that number. It might take the history of moves so far in a chess game and output the optimal next move to give the best chance of wining the game. It may take a question voiced by a person as input and give an audible answer as output. What is special about a program that makes it count as AI?

One of the most iconic books on AI surveyed various attempts to define AI and categorized them in the figure below.

Russell & Norvig (2003). Artificial Intelligence: A Modern Approach (2nd ed.). Prentice Hall, p. 2.

Let’s focus on the bottom left quadrant of the figure where a program is considered to be AI if it acts like a human. The famous Turing Test would be an example of that. Would a computer program that computes square roots be considered AI under Kurzweil’s definition from the figure above? Do you know any unintelligent people that could compute the square root of an arbitrary number to several digits quickly? It seems like the program for computing square roots would qualify, though I doubt many people would intuitively peg it as AI. On the other hand, the definition by Rich and Knight from the same quadrant would not consider the square root calculator to be AI because the task is not one where humans currently beat computers. There isn’t agreement about what AI is within the same quadrant, let alone between different quadrants.

The definition by Rich and Knight touches on an important idea known as the AI effect, which says that once computers are good at doing a particular task, it’s not really considered AI anymore — it is then seen as “just a computation.” That means the things that are considered to be AI are always changing. Facial recognition is generally recognized as being AI today. It involves analyzing an image to determine which person is in it. The superficially similar process of analyzing an image to determine what text is in it, known as optical character recognition (OCR) and used heavily in e-discovery when scanned paper documents are encountered, was declared to no longer be considered to be AI by Roger Schank in a 1991 report because OCR worked too well to be interesting at that point.

With all of this ambiguity about what AI is, why are people so eager to slap the term on everything these days? Well, there have been some impressive and highly publicized accomplishments with cutting edge technologies recently and it can feel good to have a little of the hype rub off on you. Many of those accomplishments have been achieved with something called deep learning. The artificial neural network, which is a classifier that performs computations using a network structure that vaguely resembles the human brain, has been around for about 50 years but it didn’t work very well until some algorithms for effectively using a large number of layers, known as deep learning, were invented in 2006. Deep learning is able to accurately classify things even when the important relationships between the inputs are very complex as long as there is enough training data to nail down the parameters that describe all of that complexity.

In e-discovery we want to accurately classify documents as either responsive or non-responsive with as little human document review effort as possible. We often use a supervised machine learning process that is referred to as technology-assisted review (TAR) in this context. Little human effort means little training data, which is not optimal for deep learning. Additionally, it’s not clear that the task of classifying text documents involves the type of complexity where deep learning has a real advantage, so it is doubtful that deep learning would be beneficial compared to other classification algorithms for analyzing text documents in TAR (identifying objects in photos may be a more appropriate use). Technologies used for classification in TAR are typically simpler, older, and less computationally demanding than deep learning. For example, SVM (invented in 1963), logistic regression (roughly 1951), and kNN (1951) are often used. Of course, people in the e-discovery world have put a lot of effort into tuning the classifiers and their inputs to give good results for e-discovery in recent years, but the core classification algorithms are pretty old.

Labeling everything as AI gives the impression that it is similar to the highly successful (and complicated) deep learning stuff that is in the headlines, whereas in reality the algorithms used in many industries and contexts are much older and simpler.

Highlights from Ipro Tech Show 2019

Ipro renamed their conference from Ipro Innovations to the Ipro Tech Show this year.  As always, it was held at the Talking Stick Resortipro2019_ceo in Arizona and it was very well organized.  It started with a reception on April 29th that was followed by two days of talks.  There were also training days bookending the conference on April 29th and May 2nd.  After the keynote on Tuesday morning, there were five simultaneous tracks for the remainder of the conference, including a lot of hands-on work in computer labs.  I was only able to attend a few of the talks, but I’ve included my notes below. You can find my full set of photos here.  Videos and slides from the presentations are available here.

Dean Brown, who has been Ipro’s CEO for eight months, opened the conference with some information about himself and where the company is headed.  He mentioned that the largest case in a single Ipro database so far was 11 petabytes from 400 million documents.  Q1 2019 was the best quarter in the company’s history, and they had a 98% retention rate.  They’ve doubled spending on development and other departments.

Next, there was a panel where three industry experts discussed artificial intelligence.   AI can be used to analyze legal bills to determine which charges are reasonable.  Google uses AI to monitor and prohibit behaviors within the company, such as stopping your account from being used to do things when you are supposed to be away.  Only about 5% of the audience said they were using TAR.  It was hypothesized that this is due to FRCP 26(g)’s requirement to certify the production as complete and correct.  Many people use Slack instead of e-mail, and dealing with that is an issue for e-discovery.  CLOC was mentioned as an organization helping corporations get a handle on legal spending.ipro2019_lunch

The keynote was given by Kevin Surace, and mostly focused on AI.  You need good data and have to be careful about spurious correlations in the data (he showed various examples that were similar to what you find here).  An AI can watch a video and supplement it with text explaining what the person in the video is doing.  One must be careful about fast changing patterns and black swan events where there is no data available to model.  Doctors are being replaced by software that is better informed about the most recent medical research.  AI can review an NDA faster and more accurately than an attorney.  There is now a news channel in China using an AI news anchor instead of a human to deliver the news.  With autonomous vehicles, transportation will become free (supported by ads in the vehicle).  AI will have an impact 100 times larger than the Internet.ipro2019_juggle

I gave a talk titled “Technology: The Cutting Edge and Where We’re Headed” that focused on AI.  I started by showing the audience five pairs of images from WhichFaceIsReal.com and challenged them to determine which face was real and which was generated by an AI.  When I asked if anyone got all five right, I only saw one person raise their hand.  When I asked if anyone got all five wrong, I saw three hands go up.  Admittedly, I picked image pairs that I thought were particularly difficult, but the result is still a little scary.ipro2019_hotel

I also gave a talk titled “TAR Versus Keyword Challenge” where I challenged the audience to construct a keyword search that worked better than technology-assisted review.  The format of this exercise was very different from previous iterations, making it easy for participants to test and hone their queries.  We had 1,924 queries submitted by 42 participants.  They achieved the highest recall levels seen so far, but still couldn’t beat TAR.  A detailed analysis is available here.