In case you weren’t already feeling paranoid about how AI is shaping our future, scientists for the first time have used an AI model to create new and viable viruses that have never been seen in nature.

Before you panic, the work, detailed in a study published in the journal Science, involved creating viruses that only target bacteria and are harmless to humans. Bacteriophages, as they’re called, are an important medical intervention for targeting drug-resistant pathogens.

“This is an important milestone,” Patrick Cai, a synthetic biologist at the University of Manchester who was not involved in the study, told The New York Times.

Still, it’s an alarming testament to AI’s power and trajectory. The researchers at Stanford University and the Arc Institute trained an AI model to recognize patterns of genetic structure in nature to create new DNA molecules. These molecules were inserted into bacteria that, over time, turned into wholly new viruses, at the expense of killing their hosts.

In their research, which first gained attention last year but has now been peer-reviewed, the scientists used an AI model called Evo. Unlike a general purpose large language model, Evo is trained on billions of lines of genetic sequences, and for this work in particular, millions of bacteriophage genomes. But it still functions like an LLM, predicting what DNA building blocks — nucleotides — should follow each other like a chatbot predicts what words to spit out to finish a sentence.

The researchers chose to focus on a E. coli-killing virus called Phi X-174, a variant that’s so well studied that it serves as the equivalent of a lab rat in virology. Viruses in general are simple compared to complex cellular life, making them an ideal platform to test AI’s capabilities.

They used Evo to create new viruses based on Phi X-174, and it generated 700,000 candidates. They then narrowed it down to just 300 that they would recreate in the lab. Of those 300, 16 proved to be viable, infecting and killing their bacterial hosts.

There’s clear medical merit to using AI to create bacteria-slaying viruses, but there’s already fears that the technology could be abused to crank out pathogens that are deadly to humans.

“Although this is promising for life sciences applications, it also raises urgent biosafety and biosecurity questions. The ability to compose viral genomes using generative AI now exists; the governance to safely steer it does not,” warned Tom Inglesby and Moritz Hanke at the Center for Health Security at Johns Hopkins University in Baltimore in a perspective article.

Tom Ellis, a professor of synthetic genome engineering at Imperial College London who was not involved in the research, acknowledged this threat, but told The Guardian that it would be much harder to create more complex genomes and that controlling access to genetic data could stymy the threat.

And if someone were to deliberately engineer deadly viruses, he added, doing it by hand would probably be more practical.

“But honestly, the threat from full AI design and writing of a genome of a virus or bacteria is very overblown when we consider that just taking existing pathogens and making gain-of-function changes to their genomes is so much easier and much more likely to be a real pathogenic threat,” he said.

More on pathogens: America Getting Lesson in Consequences of Trump’s War on Public Health as Rare Disease Outbreaks Ravage the Country