The audience was challenged to construct a keyword search query that is more effective than technology-assisted review (TAR) at IG3 West 2018. The procedure was the same as round 4, so I won’t repeat the details here. The audience was small this time and we only got one query submission for each topic. The submission for the law topic used AND to join the keywords together and matched no articles, so I changed the ANDs to ORs before evaluating it. The results and queries are below. TAR beat the keyword searches by a huge margin this time.
| Biology | Recall | |
| Query | Top 3,000 | Top 6,000 |
| Search | 20.1% | 20.1% |
| TAR 3.0 SAL | 72.5% | 91.0% |
| TAR 3.0 CAL | 75.5% | 93.0% |
| Medical | Recall | |
| Query | Top 3,000 | Top 6,000 |
| Search | 28.5% | 38.1% |
| TAR 3.0 SAL | 67.3% | 83.7% |
| TAR 3.0 CAL | 80.7% | 88.5% |
| Law | Recall | |
| Query | Top 3,000 | Top 6,000 |
| Search | 5.5% | 9.4% |
| TAR 3.0 SAL | 63.5% | 82.3% |
| TAR 3.0 CAL | 77.8% | 87.8% |
biology query: (Evolution OR develop) AND (Darwin OR bird OR cell)
medical query: Human OR body OR medicine OR insurance OR license OR doctor OR patient
law query: securities OR conspiracy OR RICO OR insider