The MMSMOA Retweet Bot in Action

Hot on the heels of last week's triumphant return of the "MOA" edition of MMS, I'm writing up a post-conference review of my somewhat-accidental creation: the MMSMOA Retweet Bot.

I shared the bot's story with a number of folks during the conference, often while peddling the bot's stickers, but several folks also encouraged me to share the details via blog post as well so here it is!

The Bot's Origin Story

I'd built a couple Twitter bots in the past (@holidayprogress and @dadjoke_genbot), both using the v1 Twitter API. With the v2 API functions in place (and the fact I'd not made a new bot in over a year), I wanted to waste some time create something new with Twitter's v2 API. I really like the more well-defined search functionality in the v2 API as it's cleaner and provides more obvious argument handling/parameters than in v1. Added bonus: the Postman collection available from Twitter makes initial tinkering much more simple.

About a month ago at suppertime I asked the family what I should do next for a bot...but they couldn't come up with a tangible suggestion at the time. Since MMS was coming up I decided to create a "retweet bot" to help filter out some noise—specifically the noise of threads, replies, and retweets of retweets of tweets—my own way of giving a little bit back to a larger tech community that has been incredible! Over a few evenings I built out the framework of the bot and tested it out with my development bot/account.

So, How Does It Work?

My idea behind the bot was to create something that didn't just retweet everything and add to the noise. The intent was to select and highlight "original" content from folks in the community. It can be super overwhelming to follow the conference hashtag so cutting through the noise means folks, specifically newer folks or those who couldn't attend, could follow the bot to help get (or keep) their bearings...and hopefully be useful at the same time.

Bot Search Criteria

The search criteria for the bot includes:

• Look at activity over the past 1-2 hours;
• Match tweets with the #MMSMOA hashtag OR having tagged the official @mmsmoa account;
• Exclude anything that is a retweet (or quote tweet);
• Exclude anything that is a reply; and
• Exclude anything from the official @mmsmoa account.

That last bullet is the most important. This is an auxiliary bot; not including tweets from the official account was deliberate. Most folks following the bot account would also be following the official account, so there is no "value" including or retweeting official content in the search scope.

Random Tweet Selection and Activity

Once we find original tweets, one is randomly selected from the results and, if it hasn't already been used (or the user's account isn't set to "private" or otherwise restricted), we do three things:

1. Like the selected tweet;
2. Retweet the selected tweet; and
3. Follow the original user (if not already following).

Self-Metering Capability

I wanted the bot to self-meter its activity to not over-tweet during periods of light activity. It's obvious that we'll have a lot of content during the conference itself, but to keep things interesting (and less spammy) I wanted the bot to remain a little random. Out of this was born the metering device, which is semi-mathematically represented here:

Next Trigger = 3 * (Search Window / Search Result Count)

The design target was to retweet 1 of every 3-4 tweets found—the above determines the average time between original tweets over the recent search window and multiplies by 3. This trigger interval "predicts" three matches at the next run based on recent activity.

The bot code is invoked every five minutes regardless of activity and if we have reached the next trigger we select and retweet something; if not we just look at recent activity and shorten the trigger if we're seeing more activity than originally predicted. The latter bit is what keeps the bot from under-retweeting as we enter higher volume periods.

Releasing the Bot

After I had the basics together and working I quietly let the bot go live to see what happened. I cloned the bot code to one of my Linux (Ubuntu) VPS boxen, set the config details, and on the evening of April 19 it found and retweeted its first match!

I let the bot do its thing for the next day or two. In that time it got 10-15 followers—pretty rad given I'd not done anything than let it naturally retweet content. It was at that point I decided to get some bot stickers created because it was likely to be more of a cool thing for folks to monitor than I'd expected or anticipated. There was some buzz behind the magic!

As of the time I write this, the bot is being followed by 98 folks...which is about 70 more than I ever anticipated it might be!

I made a few pre-MMS changes to the bot source including a switch to an object-based model for the Twitter API bits, some additional monitoring/logging, and installed an SSH client on my phone (with key auth of course) so I could monitor or bugfix/change anything that might go sideways during the conference.

Post-Conference Reflection and Evaluation

Conference Cadence Changes

Much to my elation, the bot worked perfectly throughout the conference. I did, however, make some "just in time" changes:

• Changed the bot search interval to 1 hour during the conference to better represent activity during the highest volume; and
• Temporarily changed the cadence formula (multiplier of 3 to 2) to retweet more content (1/2-1/3 of tweets instead of 1/3-1/4 as designed).

What About the Data?

First off, apologies for the gnarly large image that's difficult to visually parse—I wanted to explain a few fun bits out of this graphic. Important to note is the chart's x axis isn't adjusted for time, but bot activity. That means a unit of time does not have the same "width" across the chart. Busier times are wider (and taller); low activity is narrower (and shorter/devoid).

The chart represents bot activity from Sunday, May 1 through Thursday, May 5 (inclusive). It's interesting to see that each day throughout MMS there's a distinct set of two peaks in activity, plus the third (anomaly) peak during the May 2 evening Patch My PC party. Further interesting is the overall trend of fewer tweets as the week progressed, and the temporary but dramatic drop in activity on May 3.

The blue lines represent search results at the time a retweet was made; the yellow lines represent moments a matching tweet had already been used. Something else to note: more yellow lines pop up after peak activity (or micro-peaks) as there are fewer new tweets being generated—but the bot predicted a slower decay due to activity. This means odds are higher something will match an already-retweeted item because the "bucket" is smaller than predicted. When building the bot I figured this might happen, and it's cool to see it play out.

There's a one-time-event (yellow line) matching SEVEN previous tweets. Looking at the raw data at the time when this interval was reached, only nine tweets were found but most of them had already been retweeted. It took seven randomized attempts to find an unused tweet. During the hour preceding about 12-15 tweets were in scope per run with few retries and a 10 minute cadence; folks must've bailed for lunch leaving the activity pretty low (the hour following averaged about 6 tweets in scope per run).

The Real Bot Cadence

On the whole the bot retweeted just over 500 unique and original tweets throughout the conference (and those periods before and after). It automatically followed just over 190 folks during that same time, and wound up with just under 100 followers.

When adjusted for the 1-hour search window during the conference period the bot would generally retweet based on volume:

• < 5 tweets: every 30-90 minutes;
• 5-10 tweets: every 15-20 minutes;
• 10-20 tweets: every 10 minutes; and
• 20+ tweets: every 5-10 minutes.

What's Next?

Given the bot's success and community interest, I intend to leave it running indefinitely at this time. I've changed the bot search window to 6 hours for another week or so. Twitter's v2 API only allows for search up to 7 days prior; once we're a week or two out from the conference I'll probably change the window to 1-2 days. At that time I'll also adjust the cron to run every 6-12 hours (instead of every five minutes)...and we'll see where it goes.

I'm certain I'll start it all up again (and crank it back into higher-volume production) for future editions of MMS, so keep an eye out for that!

Share and Share Alike!

I wrote the bot totally as a novelty project for myself with hopes that folks in the larger community might also find it useful. As such I put the bot code out in GitHub and the bot has its own landing page. You are more than welcome to clone the bot repo and make your own similar bot or do something wildly different with it!

Headline image via the Retweet Bot website; graphically a mashup of Jamie Sale’s free clip art and the MMS logo.