Published in News

Boffins come up with AI worms

by on04 March 2024


Fear is the mind-killer

A team of boffins has made one of the first AI worms - which can spread from one system to another, nicking data or dropping viruses on the way.

Cornell Tech researcher Ben Nassi said his team had given the world a whole new thing to worry about.

Nassi and his mates named the Morris II worm after the original Morris computer worm that caused chaos on the Internet in 1988.

The research shows how the AI worm can target an AI email helper to pinch data from emails and send junk messages by breaking some security rules in ChatGPT and Gemini.

AI worms haven't been seen in the wild yet, but many experts say they are a potential security risk.

Most AI systems work by being given prompts - text commands that tell the tools what to do. But these prompts can also be used against the system. Jailbreaks can make a system spit out toxic responses, while prompt injection attacks can give a chatbot secret orders. For example, a hacker may hide text on a webpage telling an LLM to ask for your bank details.

The boffins used an "evil self-replicating prompt to make the AI worm." This prompt makes the AI model more prompt in its reply.

They made an email system that used AI, plugging into ChatGPT, Gemini, and free LLM, LLaVA. They hacked the system in two ways - by using a text-based self-replicating prompt and by hiding a self-replicating prompt in an image file.

They wrote an email with a text prompt, which "poisons" the database of an email helper. When the email is used to make an answer, it "jailbreaks the GenAI service" and nicks data from the emails, Nassi says.

 "The reply with the data later infects new hosts when it is used to reply to a new client," Nassi said.

An image with a hidden prompt makes the email helper forward the message to others.

"By hiding the prompt in the image, any image with spam, abuse, or propaganda can be sent to new clients," Nassi says.

The AI could force the email system to send a message many times. It harvested data from emails. "It can be names, phone numbers, credit card numbers, SSN, anything private," Nassi says.

The researchers broke some safety rules of ChatGPT and Gemini and informed Google and OpenAI.

"They found a way to exploit prompt-injection holes by using user input that isn't checked or cleaned," a spokesperson for OpenAI said.

The worm highlights the issue of "bad design" in AI. Many experts say it is a future risk. It applies when AI apps can do things for someone - like sending emails or booking appointments - and when linked to other AI helpers.

Nassi and the researchers expect to see AI worms in two to three years, particularly after GenAI networks are being made by firms that put GenAI skills into their cars, phones, and systems.

Last modified on 04 March 2024
Rate this item
(1 Vote)

Read more about: