Counting Posts per User per Month in WordPress

Had an interesting problem land on my plate today. A corner-office type wanted to know how many posts per month each of our authors had written. Sounds like a round of layoffs are coming, eh?

I hadn’t written any SQL in a LONG time but I managed to finally figure it out. Certainly it was faster than trying to add things up by hand (we have 17 brands we need this data for). Figured I’d save the queries in case I ever need them again. I am not saying this is the most efficient way to do this — I’m no DBA — but for a 1 time thing, it worked.

SELECT t.year_month, users.user_nicename, users.user_email, count(post_author)
FROM
(
SELECT DISTINCT(EXTRACT(YEAR_MONTH FROM `post_date`)) as `year_month`
FROM wp_posts
WHERE post_date > "2021-01-01 00:00:01"
ORDER BY `year_month` ASC
) as t,
wp_users users,
wp_posts posts
WHERE
posts.post_author = users.ID AND
EXTRACT(YEAR_MONTH FROM posts.post_date) = t.year_month AND
posts.post_status = 'publish' AND
posts.post_type IN ('post', 'company', 'download', 'event', 'page', 'podcast', 'product', 'webcast')

GROUP BY posts.post_author, t.year_month
ORDER BY posts.user_nicename, t.year_month ASC

In my case the request was for the past 2 years, so we’re starting at 2021-01-01.

The EXTRACT(YEAR_MONTH FROM `post_date`) was a new syntax for me.

So the posts.post_type IN line will have to be customized for your site; it depends on what content types you want to count. For this site we have a few custom content types. For a “vanilla” wordpress site IN (‘post’, ‘page’) should work.

This query sorts by author first, so you have 24 (since I’m grabbing 24 months of data) rows for the first user, then 24 for the next, and so on.

SELECT t.year_month, users.user_nicename, users.user_email, count(post_author)
FROM
(
SELECT DISTINCT(EXTRACT(YEAR_MONTH FROM `post_date`)) as `year_month`
FROM wp_posts
WHERE post_date > "2021-01-01 00:00:01"
ORDER BY `year_month` ASC
) as t,
wp_users users,
wp_posts posts
WHERE
posts.post_author = users.ID AND
EXTRACT(YEAR_MONTH FROM posts.post_date) = t.year_month AND
posts.post_status = 'publish' AND
posts.post_type IN ('post', 'company', 'download', 'event', 'page', 'podcast', 'product', 'webcast')

GROUP BY t.year_month, posts.post_author
ORDER BY t.year_month ASC

Same data, different format. This one orders by date, so you get a row for each author for Jan 2021, then a row for each author for Jan 2022 and so on.

One thing to try when Wordfence breaks your server

I opened up my local copy of one of our sites today and… white screen. WTF, it was working fine the last time I used it. I hit reload and now I’m getting a server not found error. That’s weird.

So I check the log files and see this:

[Mon May 07 15:09:14 2018] [notice] Child 167276: Starting thread to listen on port 80.
[Mon May 07 15:09:16 2018] [notice] Parent: child process exited with status 255 — Restarting.
[Mon May 07 15:09:17 2018] [notice] Digest: generating secret for digest authentication …
[Mon May 07 15:09:17 2018] [notice] Digest: done
[Mon May 07 15:09:19 2018] [notice] Apache/2.2.31 (Win32) DAV/2 mod_ssl/2.2.31 OpenSSL/1.0.2e mod_fcgid/2.3.9 mod_wsgi/3.4 Python/2.7.6 PHP/7.1.7 mod_perl/2.0.8 Perl/v5.16.3 configured — resuming normal operations
[Mon May 07 15:09:19 2018] [notice] Server built: May 6 2016 10:19:53
[Mon May 07 15:09:19 2018] [notice] Parent: Created child process 50416
[Mon May 07 15:09:20 2018] [notice] Digest: generating secret for digest authentication …
[Mon May 07 15:09:20 2018] [notice] Digest: done
[Mon May 07 15:09:21 2018] [notice] Child 50416: Child process is running
[Mon May 07 15:09:21 2018] [notice] Child 50416: Acquired the start mutex.
[Mon May 07 15:09:21 2018] [notice] Child 50416: Starting 64 worker threads.
[Mon May 07 15:09:21 2018] [notice] Child 50416: Starting thread to listen on port 80.
[Mon May 07 15:09:23 2018] [notice] Parent: child process exited with status 255 — Restarting.

This repeated over and over. Apache was starting but when I attempted to load the site, it would crash. If I tried to load something simple that didn’t involve wordpress, everything worked fine.

Long story short, something had become foobar’d with Wordfence. Not blaming the plugin itself, I think I munged something up when I imported a fresh SQL dump. I usually don’t pull over wordfence directories but maybe I screwed up, who knows?

The fix is simple enough.

Step 1:
Delete everything in \public_html\wp-content\wflogs

Step 2:
Truncate all the wp_wf* tables in your database.

Bam, problem hopefully solved. Wordfence will recreate the files you delete and re-fill that tables as it needs to. And your local will start to work again. Huzzah!

Now if it happens again, you might have a bigger issue. What turned out to be the problem in my case was that Wordfence was exhausting Apache’s memory, essentially. This only happened for me in a MAMP environment; I never ran into the issue with my Docker-based locals.

I fixed it by tweaking the Apache ThreadStackSize by inserting this into http.conf:

ThreadStackSize 8388608

Since I was on MAMP I had to edit a template. See this post for more on dealing with MAMP templates.

Apparently (I read this on the Internet so it must be true) MAMP sets a much smaller default stack size (1 MB) as compared to a typical Linux installation (8 MB). So the above ThreadStackSize pushes the size to the same as you’d find on linux.

MAMP noobs: tweaking server settings and MySQL import problems

When I set up local sites for development I generally use Docker, but the folks back at the main office use MAMP Pro. Since recently I’ve had a greater hand in supporting these people I figured I’d better use MAMP Pro for something, so the next time I had a local to set up I used it.

Couple gotchas hit me right away. When I tried to import a bunch of data to MySQL the script would fail. I kept getting “Lost Connection to the Server” errors. Very small scripts worked, but I’m dealing with long-established sites that have a lot of data.

The solution for me was tweaking a my.ini (in Windows MAMP anyway, it’s an .ini not a .cnf file) setting:

max_allowed_packet=1M

I bumped that up to 32M

max_allowed_packet=32M

But whoops, let’s back up. You can’t just go in and change settings on the MAMP server like you normally would, so this was learning moment #2.

In MAMP you have to edit a template:

This will first pop up a warning that you could potentially screw everything up by editing these files (but we knew that, didn’t we?) and then open an editor where you can make your changes. When you save them MAMP will warn you that you have to restart your servers in order for the changes to take effect (and it will offer to do that for you).

Easy enough, once you know where to look.

Cleaning up a WordPress wp_options table bloated with transients

I’ve got a pair of wordpress sites under my care that have an issue with the wp_options table getting bloated. I mean, really big. Millions of rows for a modest sized site. I suspect it has something to do with the Yoast plugin but I’m not 100% sure. At some point I need to figure out WHY this happens but in the meantime I’ve just been treating the symptoms, so to speak. Every month or so I go in and clean up the tables.

Basically what is happening is that a bunch of transient records aren’t expiring or getting deleted or whatever they are supposed to be doing.

Live example from one of these sites (which, I should add, is running on MySQL)

select COUNT(*) from wp_options
2284298

I’ve actually seen it bigger.

So how many of these records are transients?
select COUNT(*) FROM `wp_options` WHERE `option_name` LIKE ('%\_transient\_%')
2274527

So all but 9,771 rows are transients. Even 10K rows seems pretty big for wp_options but for now I’ll pick my battles.

So time to clean up. Step 1 is always to backup the database first, or AT LEAST backup the wp_options table. I prefer to wait to do this late at night or very early in the morning, just in case it impacts performance.

Once you have a backup, do the cleanup
DELETE FROM `wp_options` WHERE `option_name` LIKE ('%\_transient\_%')

Then recheck the count:
select COUNT(*) from wp_options
10054

Much better. And now we’re set for another month or two.

As I said, I’ve been having to do this for a couple sites for a while now, and so far haven’t seen any negative impact from nuking the transients, but as always, I don’t assume responsibility for any damage you cause to your site!