The Anatomy of a Hot Question

Here is the traffic since The Workplace was created:

These are the three questions that caused those huge spikes in traffic:

1. Is it rude to leave an interview early if you have already made your decision?
2. How should I deal with an employee who has slept with my wife?
3. What is a 'friendly' way to let managers know that having good developers is a privilege?

These questions alone account for some of the highest viewed questions on our site (#3, #2, and #5 respectively).

So What's the Issue?

More visits should be a good thing. More exposure should be a good thing. But they aren't right now for three reasons:

1. New Users drown out community moderation
2. As a result these questions make bad examples
3. Community moderation burdens increase

These are all quantifiable harms to growing communities. And they should be avoided.

Voting Volume Increases 233%

The week prior to one of these hot questions, on average (all three cases) we have an average of 128 votes/day. During the week of a hot questions, that more than doubles to almost 300 votes/day. That is a lot of additional votes.

Most of those are upvotes. On TWP in general, our upvote/downvote ratio is 13.6. For these hot questions, the upvote/downvote ratio is a whopping 30.4.

People come in, make tons of upvotes, and any direction the community wants to give is lost in the noise.

Question Volume Increases 190%

The question volume also increases by 190%, from 17.2 posts/day to almost 33 posts per day. The downvote ratio on these posts is much higher -- the week prior to a hot question we have an average of 0.39 downvotes per post made. The week during a hot question it raises to 0.47. The week after it rises to 0.67 downvotes/post.

The data explorer does not include deleted questions, which would further skew the numbers.

People click through the hot questions, see mediocre questions and answers with tons of upvotes, and take that as carte blanche to start posting their own

12 Reserve Active Members Required

Peak voting during one of the hot questions is over 600 votes per day. On the week before we peak around 250. That means we have to make up 350 additional votes per day. Users have a limit of 30 votes per day. That means we need 12 additional active users casting max votes per day to try to make a dent in the votes from new users flooding in.

That requires time. Especially when the volume of closed and deleted questions and answers increases due to the drop in quality from the new users.

Even if we tried, the flood is overwhelming compared to the ability of a community to self-moderate.

There are Unquantifiable Issues Too

A quick check of our meta will find times where regular users wonder why we close so many questions. When the community is stretched thin voting, protecting, closing, deleting, and commenting on posts by new users, making aggressive edits and working with users to teach them the rules and help them improve their posts is a luxury that feels like it can wait.

Seeing one question bring in so many users often spawns successive hot questions which end up compounding the problem because even poor answers increase hotness score. The constant flood of new users and watching the usually clean review queue shoot up as well as the number of flags takes away moderator resources as well as contributes to community moderation burnout.

A Small Request

This last storm of traffic was mitigated (slightly) by me popping in the Tavern of the Meta and getting a dev to protect the question before it got out of hand. And that worked great. Only...

1. Is that really the best use of the dev's time?
2. Should we really expect your average user to pop over to meta.so chat to try to solve a problem on a beta site?

Community moderators should be aware of this issue, and what it does to community moderation. If a hot question pops up on a smaller site, I plead with whoever is awake and around to just plant yourself in that site's chat and be available to take action if the community moderators need it. Beta sites have only a few mods who aren't generally awake all the time (though I don't think jmort sleeps). Just the gesture to say, "Hey, we are here to help" would be a morale booster to people who are overwhelmed by janitorial duty.

Of course, this could be helped by nipping the problem in the bud, but in the meantime, community mods, please realize that us unpaid community volunteers could use your help sometimes, and would really appreciate it.

Ben's Query

Ben says in the comments:

What would be interesting to know is how many users, on average, stuck around and continued to be a contributing member of the community after arriving in this fashion. It could be that the number of new users you gain is more than worth the extra bit of moderation for a few days.

I took a look at the new users whose first post was an answer on one of those first two hot questions. With the help of many users, I was able to create a query that looks at new users who joined via those questions. Unfortunately, data.se doesn't look at deleted posts. There are 14 additional users with deleted questions, 3 of which are unregistered accounts. So altogether we have 24 new users who joined through these posts directly.

Here is what those users look like:

　　　　　　　　　　　　　　　　　　　　　　^Reputation

^{The numbers at the bottom of the chart on the X-axis represent the number of users who have reached that reputation level. 500 is the cutoff for closing on beta sites}

only 33% of all users who joined through these questions are active. There are only 2 active 500+ members who can help with community moderation (only one of them has helped). 42 reviews is not comparable to the increase in moderation required for these questions.

These questions do not seem to bring long-term users with good contribution to the community.