Product update: introducing bot filtering for server-side personalization

As generative AI and large language models (LLMs) continue to advance, bots have become a more significant part of the online ecosystem.

While bots play legitimate roles in SEO and content indexing, their activity can unintentionally interfere with analytics and experimentation and mess up all growth metrics.

You might be asking yourself why you didn't notice that when looking at your analytics tool. Most bots' requests to crawl the web happen on the server side, while most analytics tools operate only on the client side. Because of that, tools like Google Analytics only show a small share of bots that run on the client side.

To ensure our customers' data remains meaningful and actionable, we're introducing a new feature to exclude bot sessions from server-side rendering (SSR) personalization.

Why does bot filtering matter?

Server-side personalization offers significant advantages for companies looking to optimize their website performance.

Since most of our customers implement their personalization and AB testing strategies rendering the content on the server side, the rise of bot requests started affecting the numbers they saw in their experience analytics.

Bot requests don't represent actual user behavior, which could lead to:

• Data distortion

  Bots inflate session counts, making it harder to interpret user behavior.

• Experimentation challenges

  Bot activity skews AB test results, as one variant may attract more bot sessions than another, reducing results reliability.

• Quota usage

  Sessions triggered by bots consume part of our customers' plan's quota, which is better reserved for real users.

How to filter bots from Croct's account

To address this, we've now added a workspace-level flag that excludes SSR bots from receiving personalized or AB testing content. This feature is enabled by default in all accounts and works by identifying bots before applying any experience content.

When this flag is active:

• Bots receive the slot's default content.
• Their sessions are excluded from analytics and experience metrics.

For scenarios where serving specific content to bots is intended, such as SEO-focused content for bots, our customers can turn off the flag and manually exclude bots from specific experiences by adding this rule to their audience criteria:

1
... and device’s category is not 'bot'

This update helps them:

• Streamline analytics and ensure their data reflects user behavior, free from bot interference.
• Improve AB testing accuracy by generating trustworthy results for better decision-making.
• Optimize plan usage by saving quota for meaningful interactions with actual users.

Looking ahead

We're excited to bring this feature to your server-side workflows as part of our commitment to providing accurate, actionable data.

Bot filtering is just one step in our ongoing efforts to help our customers deliver the most relevant content while maintaining control over their personalization strategy. If you have any questions or suggestions, please contact us.