Adding GD support to a Docker image

I had to create a new docker container to emulate a site we have working on older versions of MySQL and PHP (don’t ask, long story).

The CMS was failing spectacularly. After suspecting memory issues and other esoteric settings I realized it was much more basic: there was no GD support in this container and the plugin that manipulated images was dying without generating any error messages (thanks for that, plugin developer).

Long story short, here’s my Dockerfile to get GD support working. This was one of those “look at a bunch of stack exchange answers and kind of merge them together” solutions so I’m not 100% sure how universally it will work. The bits in red are what I added for GD support. Frankly I’m not sure about that RUN NUMPROC line… I need to research that more because I honestly don’t understand what it is doing.

EDIT: OK the -j parameter passed to docker-php-ext-install is the number of make jobs that can run concurrently (in order to speed up compilation). The RUN NUMPROC line is looking at how many CPUs you have and telling make it can run that many jobs at the same time.

Edit 2: As of April 2, 2019 I started running into problems where my docker build who throw up errors like this:
Failed to fetch http://deb.debian.org/debian/dists/jessie-updates/main/binary-amd64/Packages

Apparently Debian “Jessie” updates have been moved to the archives. The solution is to tweak the sources list, new line added in green
Solution comes from https://github.com/debuerreotype/docker-debian-artifacts/issues/66#issuecomment-476616579

FROM php:5.6.38-apache

COPY 000-default.conf /etc/apache2/sites-available/000-default.conf
COPY php.ini /usr/local/etc/php/php.ini
RUN mkdir -p /etc/apache2/ssl/
COPY ./ssl.crt /etc/apache2/ssl/ssl.crt
COPY ./ssl.key /etc/apache2/ssl/ssl.key
RUN mkdir -p /var/run/apache2/

RUN sed -i '/jessie-updates/d' /etc/apt/sources.list # Now archived

RUN apt-get update -y && apt-get install -y zlib1g-dev libjpeg62-turbo libpng-dev libjpeg-dev

RUN docker-php-ext-configure gd \
--with-png-dir=/usr/lib/ \
--with-jpeg-dir=/usr/lib/ \
--with-gd

RUN NUMPROC=$(grep -c ^processor /proc/cpuinfo 2>/dev/null || 1) \
&& docker-php-ext-install -j${NUMPROC} gd

RUN docker-php-ext-install pdo_mysql
RUN docker-php-ext-install mysqli
RUN a2enmod rewrite
RUN a2enmod ssl

EXPOSE 80
EXPOSE 443

Moving old images/files

Here’s a simple one for you. I had a directory with tens of thousands of files (images) and I needed to clean it up by moving all the images with a time stamp of 2016 or earlier into another directory.

Step 1 was figuring out how many days it has been between today and 12/31/2016 using this tool:
https://www.timeanddate.com/date/durationresult.html

I came up with 618

Step 2 was running this command:

find . -maxdepth 1 -mtime +618 -type f -exec mv "{}" ~/www/images_bak/uploads \;

Where
. = current directory
-maxdepth 1 = don’t go into subdirectories
-mtime +618 = files more than 618 days old
-type f = files only, no directories
-exec mv “{}” = we want to move the files, not remove them
~/www/images_bak/uploads = directory we want to move them to

Credit for this solution goes to user Jenny D at serverfault.com

I re-write these things cuz I can never find them twice… LOL

One thing to try when Wordfence breaks your server

I opened up my local copy of one of our sites today and… white screen. WTF, it was working fine the last time I used it. I hit reload and now I’m getting a server not found error. That’s weird.

So I check the log files and see this:

[Mon May 07 15:09:14 2018] [notice] Child 167276: Starting thread to listen on port 80.
[Mon May 07 15:09:16 2018] [notice] Parent: child process exited with status 255 — Restarting.
[Mon May 07 15:09:17 2018] [notice] Digest: generating secret for digest authentication …
[Mon May 07 15:09:17 2018] [notice] Digest: done
[Mon May 07 15:09:19 2018] [notice] Apache/2.2.31 (Win32) DAV/2 mod_ssl/2.2.31 OpenSSL/1.0.2e mod_fcgid/2.3.9 mod_wsgi/3.4 Python/2.7.6 PHP/7.1.7 mod_perl/2.0.8 Perl/v5.16.3 configured — resuming normal operations
[Mon May 07 15:09:19 2018] [notice] Server built: May 6 2016 10:19:53
[Mon May 07 15:09:19 2018] [notice] Parent: Created child process 50416
[Mon May 07 15:09:20 2018] [notice] Digest: generating secret for digest authentication …
[Mon May 07 15:09:20 2018] [notice] Digest: done
[Mon May 07 15:09:21 2018] [notice] Child 50416: Child process is running
[Mon May 07 15:09:21 2018] [notice] Child 50416: Acquired the start mutex.
[Mon May 07 15:09:21 2018] [notice] Child 50416: Starting 64 worker threads.
[Mon May 07 15:09:21 2018] [notice] Child 50416: Starting thread to listen on port 80.
[Mon May 07 15:09:23 2018] [notice] Parent: child process exited with status 255 — Restarting.

This repeated over and over. Apache was starting but when I attempted to load the site, it would crash. If I tried to load something simple that didn’t involve wordpress, everything worked fine.

Long story short, something had become foobar’d with Wordfence. Not blaming the plugin itself, I think I munged something up when I imported a fresh SQL dump. I usually don’t pull over wordfence directories but maybe I screwed up, who knows?

The fix is simple enough.

Step 1:
Delete everything in \public_html\wp-content\wflogs

Step 2:
Truncate all the wp_wf* tables in your database.

Bam, problem hopefully solved. Wordfence will recreate the files you delete and re-fill that tables as it needs to. And your local will start to work again. Huzzah!

Now if it happens again, you might have a bigger issue. What turned out to be the problem in my case was that Wordfence was exhausting Apache’s memory, essentially. This only happened for me in a MAMP environment; I never ran into the issue with my Docker-based locals.

I fixed it by tweaking the Apache ThreadStackSize by inserting this into http.conf:

ThreadStackSize 8388608

Since I was on MAMP I had to edit a template. See this post for more on dealing with MAMP templates.

Apparently (I read this on the Internet so it must be true) MAMP sets a much smaller default stack size (1 MB) as compared to a typical Linux installation (8 MB). So the above ThreadStackSize pushes the size to the same as you’d find on linux.

Creating a self-signed SSL certificate for local Docker development

Usually I don’t bother setting up SSL for local development but sometimes you’ll be using a service that requires it. Plus, more and more browsers are pushing you towards SSL all the time. [Side note, I wish someone would make a browser just for developers that gets out of the way and stops trying to protect us from ourselves when we’re working on local sites.]

A lot of folks are moving towards using Let’s Encrypt for free SSL certs but I’m either stubborn or dumb. Basically I didn’t want to figure out how to work it into a Docker container system, so I went old school and created a self-signed certificate. Please note you WILL need to add a “security exception” to your browser when you’re done with this process.

First off, you need openssl. I ~think~ I installed OpenSSL via Chocolatey install openssl on Windows, but honestly it was a while ago. You could also use the Openssl that comes in Ubuntu for Windows.

The command to create a self-signed cert is:

openssl req -new -newkey rsa:4096 -days 3650 -nodes -x509 -subj "/C=US/ST=NC/L=Local/O=Dev/CN=mysite.local" -keyout ./ssl.key -out ./ssl.crt

First off, credit for this goes to StackExchange user THelper and in all honesty I don’t know exactly what all the parameters mean. (I’m so trusting!)

Here’s what you need to know to get this to work. ssl.key and ssl.crt are the filenames you’ll be generating. This string “/C=US/ST=NC/L=Local/O=Dev/CN=mysite.local” has to be customized:

/C = country code (I’m in the US)
/ST = state (North Carolina for me)
/L = locale… as you can see I fudged this
/O = Organization… again, fudged. All these matter if you’re creating a real cert to use online but for local.. meh
/CN = domain name. This one is important. I’m working on https://mysite.local

So edit that string and run the command and it should generate ssl.key and ssl.crt files.

I use docker-compose so I need to tweak my docker-compose.yml file to expose port 443 (the ssl port):

ports:
- 80:80
- 443:443

And in your dockerfile you need to copy the keys to your container. To do this, first drop them in the same directory as your docker file (I tried keeping them outside the directory but would get “Forbidden path outside the build context” errors).

Next, add this to your docker file:

RUN mkdir -p /etc/apache2/ssl/
COPY ./ssl.crt /etc/apache2/ssl/ssl.crt
COPY ./ssl.key /etc/apache2/ssl/ssl.key

So we’re creating a directory and copying the files into it.

Also remember to expose port 443 in your docker file:

EXPOSE 80
EXPOSE 443

Now you need to update your apache virtual hosts to support ssl.

Here’s part of my Virtual Host file, the first 3 lines are what we’re focused on, notice they match the directory we created in our docker file:

<VirtualHost *:443>
 SSLEngine On
 SSLCertificateFile /etc/apache2/ssl/ssl.crt
 SSLCertificateKeyFile /etc/apache2/ssl/ssl.key

ServerName mysite.local

ServerAdmin webmaster@localhost
 DocumentRoot /var/www/html

 ErrorLog ${APACHE_LOG_DIR}/error.log
 CustomLog ${APACHE_LOG_DIR}/access.log combined

<Directory "/var/www/html">
  Require all granted
  AllowOverride All
  Options Indexes FollowSymLinks
 </Directory>
 </VirtualHost>

And that should do it!

MAMP noobs: tweaking server settings and MySQL import problems

When I set up local sites for development I generally use Docker, but the folks back at the main office use MAMP Pro. Since recently I’ve had a greater hand in supporting these people I figured I’d better use MAMP Pro for something, so the next time I had a local to set up I used it.

Couple gotchas hit me right away. When I tried to import a bunch of data to MySQL the script would fail. I kept getting “Lost Connection to the Server” errors. Very small scripts worked, but I’m dealing with long-established sites that have a lot of data.

The solution for me was tweaking a my.ini (in Windows MAMP anyway, it’s an .ini not a .cnf file) setting:

max_allowed_packet=1M

I bumped that up to 32M

max_allowed_packet=32M

But whoops, let’s back up. You can’t just go in and change settings on the MAMP server like you normally would, so this was learning moment #2.

In MAMP you have to edit a template:

This will first pop up a warning that you could potentially screw everything up by editing these files (but we knew that, didn’t we?) and then open an editor where you can make your changes. When you save them MAMP will warn you that you have to restart your servers in order for the changes to take effect (and it will offer to do that for you).

Easy enough, once you know where to look.

Excluding a directory when creating a tar file

So this is a dumb one but it took me longer than expected to find an answer.

So say I want to make a local copy of a wordpress site. I want to tar up all the files to download because downloading the files one by one will take hours and hours.

Assuming a directory structure like

/www/public_html/index.php

I would normally go to /www and do

tar -zcvf mysite.tar.gz public_html

That gets everything…much faster to ftp.

But on some of our sites, we wind up with a BIG cache and I really don’t need all that stuff. In our case the cache is at:

/www/public_html/wp-content/cache

So how to exclude the cache files? There’s an –exclude flag for tar but it seems finicky. Turns out it has to be the first parameter, at least for me and some others.

So this worked for me:

tar --exclude='public_html/wp-content/cache' -zcvf mysite.tar.gz public_html

This did NOT work for me
tar -zcvf mysite.tar.gz public_html --exclude='public_html/wp-content/cache'
Nor did this:
tar -zcvf --exclude='public_html/wp-content/cache' mysite.tar.gz public_html

–exclude seemed to only work when it came first after the tar command.

HP Spectre x360 mouse cursor freakout bug SOLVED

A couple years ago our IT Director asked me to pick out a new laptop for myself. Given that I am a Windows guy and several of my colleagues use expensive Macbooks, and given that I am a remote worker and so act as my own helpdesk/hardware support system, I had a lot of leeway in what I picked.

I wound up going with the HP Spectre x360, a sexy-slim laptop with a 15″ touch screen and a hinge that lets you flip it backwards into pseudo-tablet mode. I think it was about $1600 at the time.

I LOVED this laptop for a couple of months, then the mouse cursor started acting up. It would jump to the dead center of the screen and them streak upwards. When it reached the top of the screen it would reappear at the bottom and continue moving upwards. At first this happened now and again but over the course of the next few weeks it got worse and worse.

I tried all sorts of things to fix it. I started by just tweaking the trackpad settings, then updated the trackpad driver, then disabled it completely, then went through the same routine with the touchscreen driver. Since the problem was somewhat intermittent I thought I’d fixed it many times, but the problem always came back.

Finally I went for the nuclear option and reinstalled Windows 10. That seemed to work for a while but eventually the problem returned, at which point I started thinking that maybe it was a hardware problem.

I decided I’d have to wipe the machine and talk to our IT Director about getting it sent in for service and then never did that because a) it sounded like a pain and b) I hated to admit to IT I’d encountered a problem I couldn’t fix. I set the laptop aside for… well.. 8 or 9 months? I had my own Surface to use when I needed to travel and most of my work I do on my personally owned desktop system with 2 big monitors…. don’t tell IT this since I’m not supposed to be using my own gear.

Last night for some reason I decided to give it another go. A lot of people who’ve had this problem (and it seems limited to version one of the x360 which was replaced by a newer version a while back) keep talking about the touchscreen being the problem, so once again I decided to try disabling the touchscreen. When I was in device manager my eyes landed on the Pen Driver. When IT bought me this laptop she also bought me an over-priced pen/stylus for it. I think I used it once and honestly don’t even know where it is. Out of sight, out of mind.

On a whim I disabled the pen driver and… poof, the problem vanished. Then I re-enabled the touchscreen driver and still, the problem was gone. Now I’m kind of kicking myself that it took me this long to consider doing this but I guess all’s well that ends well. At least I’m hoping it has ended well…it’s only been 12 hours or so.

So if you have an early edition Spectre x360 with a wonky cursor try this:

Windows+X, pick Device Manager from the list that pops up, then look under Human Interface Devices. If you have an “HD-Compliant Pen” driver, right click on it and choose Disable and see if that helps.

Of course if your cursor is as wonky as mine was, just doing this can be a challenge. It was like playing a video game, trying to fight the glitch long enough to get the pointer over to the item I wanted to disable.

Update, September 24th 2018: OK it’s been 6 months or so since I wrote this post and the problem is still solved. So it was definitely an issue with the pen driver. Now my only issue is that when Windows updates (and I put this machine in the Windows Insider program so it updates pretty often) the driver gets re-enabled and I need to go in and disable it again. Not a huge deal, though, and I’m back to loving this laptop.

Cleaning up a WordPress wp_options table bloated with transients

I’ve got a pair of wordpress sites under my care that have an issue with the wp_options table getting bloated. I mean, really big. Millions of rows for a modest sized site. I suspect it has something to do with the Yoast plugin but I’m not 100% sure. At some point I need to figure out WHY this happens but in the meantime I’ve just been treating the symptoms, so to speak. Every month or so I go in and clean up the tables.

Basically what is happening is that a bunch of transient records aren’t expiring or getting deleted or whatever they are supposed to be doing.

Live example from one of these sites (which, I should add, is running on MySQL)

select COUNT(*) from wp_options
2284298

I’ve actually seen it bigger.

So how many of these records are transients?
select COUNT(*) FROM `wp_options` WHERE `option_name` LIKE ('%\_transient\_%')
2274527

So all but 9,771 rows are transients. Even 10K rows seems pretty big for wp_options but for now I’ll pick my battles.

So time to clean up. Step 1 is always to backup the database first, or AT LEAST backup the wp_options table. I prefer to wait to do this late at night or very early in the morning, just in case it impacts performance.

Once you have a backup, do the cleanup
DELETE FROM `wp_options` WHERE `option_name` LIKE ('%\_transient\_%')

Then recheck the count:
select COUNT(*) from wp_options
10054

Much better. And now we’re set for another month or two.

As I said, I’ve been having to do this for a couple sites for a while now, and so far haven’t seen any negative impact from nuking the transients, but as always, I don’t assume responsibility for any damage you cause to your site!

Working remotely

A few years ago we decided to escape the high cost of living in the Boston area and move south. Wound up in Raleigh, NC. Once we’d decided to move I asked my manager if I could work remotely. He said he was willing to give it a try. Honestly I was hoping I could get 6 months or so out of the deal: enough time to find local work.

Four or five years later, I’m still working remotely and couldn’t be more grateful for it. I haven’t set foot in the home office for years.

The up-sides of working remotely are obvious: no commute, you have complete control over your working environment and you, in theory, can work in your PJs. (I don’t, though. I find that ‘getting dressed for work’ is a ritual that helps flick my brain over into working mode.)

There are some downsides, of course. I miss out on a lot of back-channel communications; those “over the cubicle wall” meetings that happen spontaneously and which co-workers often overhear. Conference calls aren’t great when you’re the only one on the phone; it can be hard to hear what’s going on and I’m always aware that I’m inconveniencing my comrades when they need to set up a call just for me.

I actually spend more hours in front of my PC than I did when I worked in the office because I don’t have the distraction of chatting with co-workers or walking across the street with them to get a coffee.

Also while my manager is cool with me working remotely, the owner of the company and a couple of the highest-level execs don’t trust remote workers. They assume anyone who works remote is just goofing off. Because of this, I’m pretty anal about being available all day, every day. That means replying to emails or chat messages as soon as they come in, whenever possible. Still, it’s a small price to pay.

Since I’m the only member of my team that is remote, I don’t work unusual hours. I start at 8:30 or so and work until 5. I usually take a break at 5 to walk the dog then come back to finish up any lingering work that either won’t wait until tomorrow, or that is stuck in my craw and is going to distract me until I finish it.

I also have to be self-reliant. If I’m having a problem with hardware, I need to be able to fix it myself. In theory I guess I could ship my laptop back to the home office and ask IT to remove a virus (or whatever); I know there are less technical remote workers that do that. Our one IT person is over-worked enough without my doing that, so I go to great lengths not to ask for help. As a web developer I feel like you should be able to fix your own issues anyway (short of actual hardware failure, of course).

I use a couple of different systems so that if a problem crops up on one and I can’t fix it quickly, I can switch to working on another system until I have time to address the problem.

I do have a proper desk set up and I spend 95% of my time sitting there working. Every once in a great while I’ll feel the need for a change and I’ll take the laptop and sit on the couch in the living room, or even out on the balcony (though in NC there’re about 10 days out of the entire year that are warm enough to sit outside without being uncomfortably hot).

My Internet connection is 200 MBPS down/ 20 MBPS up. It’s effectively faster than the Internet I had at the office (theirs is faster but of course a lot more people are using it). Fortunately it’s pretty reliable. For a few years I paid for a back-up cellular system but I very rarely had to use it and when I did it was horribly slow. So now on the rare times our Internet goes out, if I don’t have any strictly local work I can focus on, I just take PTO.

Of course the very best part of working from home is my morale officer. She spends most of the day laying on the floor in the office, but if I’m getting stressed or frustrated I can always count on her to make me laugh or smile. Every office should have a dog.

ConEmu – a better Windows terminal experience

I’ve been working in terminals a lot lately. I decided a while back to learn Docker because I needed a better way to host all my dev sites than trying to cram them all onto one old PC running Linux, and Docker seems to be the ‘hot’ container system these days. Or one of them anyway.

I’ve got both Bash for Windows (the Ubuntu subsystem introduced a while back) and Git Bash on my machines and kind of bounce back and forth between them.

When I’m on a laptop I really prefer keyboard shortcuts to using the trackpad for stuff, but I couldn’t get simple stuff like Paste to work in Bash for Windows. I tried everything, CTRL-V, CTRL-SHIFT-V, SHIFT-INSERT, right-clicking.

Eventually I stumbled on the suggestion to just use ConEmu and it has solved all my problems and added some nice features like multiple tabs. It’s kind of a wrapper for other terminal programs so when you open a tab you can have it run Bash for Windows, or Git Bash, or Powershell, or even a Chocolately tab.

It’s free, works great. Ideally I would have preferred not to have to install something new but in this case it seemed like a worthwhile compromise.

I installed it from the site, but you can get it through Chocolately too:

choco install conemu