Published in AI

AI can be fooled by playing with words

by on26 February 2020


Boffins come up with text fooler

Changing a single word in your resume can fool AI screening software into classifying you as a person with deep convictions instead of a person with a long list of convictions.

Boffins from the University of Hong Kong and Singapore’s Agency for Science, Technology, and Research devised an algorithm called TextFooler capable of deceiving an AI system without changing the meaning of a piece of text. The algorithm uses AI to suggest which words should be converted into synonyms to fool a machine.

To trick an algorithm designed to judge movie reviews, for example, TextFooler altered the sentence:

“The characters, cast in impossibly contrived situations, are totally estranged from reality”, to read: “The characters, cast in impossibly engineered circumstances, are fully estranged from reality.”

This caused the algorithm to classify the review as “positive”, instead of “negative”. The demonstration highlights an uncomfortable truth about AI—that it can be both remarkably clever and surprisingly dumb.

Researchers tested their approach using several popular algorithms and data sets, and they were able to reduce an algorithm’s accuracy from above 90 percent to below 10 percent. People generally judged the altered phrases to have the same meaning.

Last modified on 26 February 2020
Rate this item
(0 votes)

Read more about: