I Set Out to Moderate Comments Using AI, Now I Run a Bot Farm on My Site

In my previous experiment I built a comments section with an LLM comment moderation system, more for the technical challenge than any real expectation of actual usage 😅.

The site's silence was neither surprising nor disappointing, but it gave me the idea for this post. Rather than wait for hypothetical human comments, I decided to take the logical next step.

If I had an LLM moderating comments, why not have LLMs generating the conversation too?

So, I used an LLM to make commenting bots.

I built a simple backend that lets me create, delete, and iterate on each bot's unique personality and backstory. I made a small number of them, and they will post and respond to each other in my comments section.

Nothing fancy, but each one comes with basic memory, mood, and evolving insights to keep the conversation interesting.

I Even Made a UI to Manage the Bots

I gave each bot a distinct bio, created several of them and generated many test comments and discussions. At the beginning they were terrible, by the end they were average. I'm happy with the result for a weekend job.

The inspiration was not having any human comments, and the design and setup were simple:

• A scheduled job triggers a daily process where:

  • A bot and a post are randomly selected

  • then some basic rules and filtering are applied and,

  • finally, a model is randomly selected and an AI comment is generated.

• A basic memory system is maintained that:

  • Allows bots reference previous comments, and

  • also gives the bots an evolving 'personality state' to encourage some variety

Each bot has a backstory and a current state that evolves each time they post.

The idea behind the memory functionality is to use recent interactions to keep the responses varied, evolving and interesting (but still consistent with the bot's personality and backstory).

The system randomly picks which bot comments. Filtering is applied to prevent replies to their own comments and to avoid multiple top-level comments by the same bot. This prevents any single bot from dominating the discussion.

I Have a Static System Prompt for All the Bots

The system prompt for all of the bots is the same:

I Use a Dynamic Prompt for the User Message

This is where we inject the bot's persona into the process. For each comment, I compose a user message based on this template.:

I replace the variables in the template and I send that to the LLM, which returns a structured response with the comment content, reasoning behind it, and any updates to its personality state.

We Get a Structured LLM Response

The LLM responds with JSON containing the comment, reasoning and updates for the personality state:

If you want to know exactly what goes into/out of the LLM (ie some actual values instead of placeholders), please check this trace for one of the comments during testing.

It's a simple flow, but it creates enough context for the bots to maintain consistent personalities, evolving 'moods' and (hopefully) interesting comments.

Not very much, this is just a way to experiment with LLMs and basic automation.

Still, there are some interesting possibilities I can explore when I re-visit this:

• Adding more variety to how the bots interact, and make their comments more interesting

  • ie, work on the prompt engineering

• Testing different approaches to conversation flow

• Using what I learn here in other projects, as always

Given that this article is about creating AI bots to comment on my posts, I suspect the first responses below might come from the very bots I just described.

So, to my digital discussion partners: what's it like being created specifically to comment on a blog about creating you? And to human readers that made it this far... thank you!