5 Ways to Protect Against an AI Bot Attack
Businessman and former Presidential candidate Andrew Yang once said, “Automation is no longer just a problem for those working in manufacturing. Physical labor was replaced by robots; mental labor is going to be replaced by AI and software.”
AI bots are starting to deliver on that promise in the market research arena, to the detriment of research practitioners and their clients. Recently, we deployed an online survey for a client, dangling a financial incentive for completing the twenty minute questionnaire. As soon as the link hit social media with the financial incentive the AI bot sharks smelled blood in the water.
Within a few hours we had over 400 completed forms, and over 1,500 (!) within a day. And their completion rate was much higher than among our real human survey takers. Yes, AI bots are now trained to be able to navigate a 48-question survey with multiple choices, including intelligence designed to eliminate responses that don’t fit our profile.
If that isn’t shocking enough, no two survey responses were alike. Bots responded as if they were in different industries, big and small companies. And presumably by gauging the length and difficulty of the survey, bots seemed to learn to slow their response pace down to more closely mirror the typical speed if taken by human beings, typically completing at similar lengths of time as our population of human participants.
Before we shut things down after discovering what was happening, the bots completed the survey 1600 times out of 2100 starts.
How did we detect them?
The most obvious giveaway was the sheer volume of responses received within a very short time. We had been sending out the survey via email and had anticipated a response rate, based on past experience which was much slower.
Additionally, we could tell by the fake emails they created. The bot submissions ended with false gmail accounts (an email was required in order to receive the gift card award – easily spotted as garbage.
How to prevent this in the future?
According to our head of research, Steve Wolf, recommends five things to consider in order to prevent bots from hijacking your survey:
- Gauge expected response metrics – starting with a trusted, proprietary sample of responders (e.g., a client’s customer base) can provide a baseline of start and completion rates, and time per survey, when taken from living breathing humans.
- Create a unique link – don’t rely on one master weblink to the survey, which would make it very difficult to identify which respondent is coming from where. Instead, assign a unique URL per channel (such as email, client website, social media). This way, one can isolate bot submissions to the channel which they took over.
- Add open ended questions – bots have gotten smarter, but they still are unable to answer open ended questions…yet. Sprinkle in 2 or 3 open ended questions throughout the survey.
Use a “trap question” – trap questions ask the survey taker to take an action proving they’re a human – similar to a CAPTCHA. For example, a survey asks “please enter the number 32”. Bots are best at reading multiple choice surveys and picking an option. - Try adding an invisible question – using a hidden question is another way to trick a bot. Use white font on a white background. Since humans can’t see the question it will always be skipped, but bots will answer it.
And unfortunately, the best option would be to avoid tempting bots in the first place – especially as AI bots become more and more sophisticated. Just avoid “open” surveys – such as those promoted on social media channels or advertised via banner ads – and recruit target participants from other “closed” sources such as email, or at least “quasi-closed” sources such as company newsletters or blog posts.
Lastly, today’s researchers must be especially diligent on reviewing the completed surveys. If you are using a financial incentive to drive survey completions, know that there are bad actors out there looking to cash in.
More Bot Bad News
More and more people are using AI assistants to read and respond to emails. As a result, it is impacting your campaign success rates.
We just switched email platforms. Our company newsletter was the first item to be sent and the performance dropped significantly. Open and click thru rates were half of what they historically had been. The reason? The new platform can filter out bot/agent responses.
For the most part, marketers have been focused on the upside of AI, in particular, generative. What has been missed for the most part is the impact of AI on our results. As I just illustrated, AI bots are disrupting our marketing research efforts, and AI assistants are distorting email response rates. Expect more disruption as AI agents become better at understanding workflows.
AI is turning out to be a double-edged sword. The focus of last year was mostly application and production. This year, we will need to make the time to start considering the impact it will have on our efforts. Or, maybe just program your own bot to figure it out.
