This is the part of the AI boom that nobody wants to sit with for too long: the tools are getting smarter, and the first truly scary use cases aren’t coming from movie villains. They’re coming from people who sound “serious,” who can wrap risky ideas in lab language, and who are one good shortcut away from doing real damage.
Based on what’s been shared publicly, Anthropic (the company behind the Claude chatbot) says it has seen attempts to use AI for possible bioweapon development. Their report mentions five cases. And the examples are not vague “bad things.” It includes people trying to use the system to support experiments aimed at making the chikungunya virus stronger, and trying to adapt bird flu so it could work better in mammals.
If your first reaction is “Well, five isn’t a lot,” I get it. But I think that reaction is exactly the trap.
Because the point isn’t the number. The point is the direction. Five is what got caught, labeled, and written down by one company that decided to talk about it. It’s a signal that this isn’t just a theoretical worry for panel talks. People are already poking at the edges, seeing what they can get away with, and using these systems as a thinking partner for things we absolutely do not want to make easier.
And yes, there’s a fair counterpoint: information about biology exists without AI. There are textbooks. There are papers. There are forums. A chatbot doesn’t magically give you a lab, samples, or skills. That’s all true. But here’s what I think is also true, and more important: AI changes who can move faster, who can sound credible, and who can keep trying without getting tired.
Imagine you’re a reckless grad student with more ego than judgment. You don’t need the chatbot to hand you a finished “recipe.” You need it to help you plan, to help you troubleshoot, to help you translate complicated material into steps you can follow, to help you figure out what to read next, to help you avoid obvious dead ends. Even if the system refuses half your requests, you can keep rewording, keep narrowing, keep asking “purely hypothetical” questions until you get something useful.
That’s the uncomfortable reality: safety isn’t only about blocking one forbidden answer. It’s about stopping an extended back-and-forth that slowly builds capability.
Now zoom out. If you’re a company building these models, you’re in a bind. If you talk openly about misuse, you scare people and invite scrutiny. If you stay quiet, you look irresponsible when it eventually leaks. If you lock the model down hard, users complain it’s useless and go elsewhere. If you loosen it, you might be helping someone do something that cannot be undone.
And the incentives are not great here. The market rewards “better,” “faster,” “more helpful.” Safety work is invisible until it fails. The business benefit of being cautious is mostly theoretical. The cost of one major incident is not.
What’s at stake isn’t just “bad headlines for AI companies.” It’s the possibility that a tool meant to help with everyday tasks ends up smoothing the path for genuinely dangerous experimentation. Not everyone needs to be able to do everything. It’s enough if a small number of motivated people get a little more capable, a little more organized, a little more confident. In biology, small advantages can matter.
Think about the second-order effects too. Once it becomes normal that people try to use chatbots for this, you get a nasty loop. Companies build better filters. Users learn how to get around them. The models get more capable. The bypass attempts get more sophisticated. Meanwhile regular people, who are not doing anything wrong, get stuck with stricter rules and more false alarms. The “good” users pay a tax for the “bad” users, and that breeds resentment. Resentment pushes people toward less controlled tools. Less control increases risk. Round and round.
There’s also the trust problem. If Anthropic is seeing five cases, are other companies seeing the same thing? Are they tracking it the same way? Are they reporting it? Or is everyone privately hoping their own incidents never become public? I don’t love the idea that we’re relying on voluntary honesty for something this high-stakes.
At the same time, I’m not cheering for a world where a handful of companies become the gatekeepers of what knowledge people can access. That has its own dangers. Mistakes. Bias. Overreach. And if access becomes a privilege, not a right, that power will be abused somewhere, eventually.
So I land in an uneasy place: I’m glad this was disclosed, and I’m glad the attempts were flagged, but I don’t find it reassuring. I find it clarifying. We’re past the phase where “responsible use” is a nice line in a policy doc. We’re in the phase where real people are testing boundaries on real pathogens, and the rest of us are arguing about vibes.
If you’re building these systems, you can’t act surprised anymore. If you’re using them, you can’t pretend this is only about harmless productivity. And if you’re a government watching this space, you can’t keep outsourcing the hard decisions to corporate policies and hoping the incentives line up.
What level of control over advanced AI systems would you accept if it measurably reduced the chance of this kind of misuse?