Do you have incident response to software issues prepared? What if your software starts crashing websites? Well, first you need an incident response plan for your software, which we provide, which also is why we were able to fix an issue in 20 minutes that crashed a dozen websites. Our software is on over 3000 websites and we were alerted by zen moments that our software had crashed four sites. After we resolved the issue we discovered he wasn’t the first to speak about the issue, rather the only one to work to find someone to address the issue. The rest were complaining on forums that don’t alert the company of the issue or new complaint posted.
Our software incident response to this issue and what we learned is the most important takeaway for all software developers. Our first response was to check forums and see if anyone else was complaining, but not actually alerting us, which they were. About a dozen sites out of over 3000 sites were affected.
Our new feature was solid on the live site we tested and staging site, but that didn’t matter at this point in time, what mattered was making sure everyone using our software didn’t have it crash their site, as that is on the list of worst things that can happen. The next thing we did is unheard of in the software world. We released a stable fix 20 minutes after being alerted to the situation. We repeat it only took 20 minutes and one programmer, to fix the issue and that included new code to remove our directory traversal patch, not just deleting code. We then continued to analyze what went wrong and privately spoke to zen_moments.
After a day of apologizing for the accident to those affected and continuing our analysis, we found the root of the problem, since zen was kind enough to allow us access to their logs. What we can tell thus far is that PHP 5 PEAR had a problem processing what was and still is solid code for any supported version of PHP, unfortunately a dozen sites, had web hosts using out of date PHP. One of those web hosts is allegedly according to a forum user 1 and 1 web hosting.
After analysis we realized that PHP 5 PEAR for a reason we’ve yet to explain thought admin panel files should be part of our WordPress plugin, when all our plugin did was state if the plugin or themes folder was accessed to give a blank index.html. Why that crashed entire sites is still somewhat of a mystery, but we have another new version in the works that should allow us to restore our new feature.
What did we do to make it right with the community using our software? We thanked zen moments by sending him 20 dollars, a dollar for every minute it took us to write the patch. In no shape or form does this mean, we will do this again in the future nor does it mean we are responsible if something goes wrong, as the GPL license clearly states, it simply was our way in this incident to thank him for his gesture of goodwill and sharing logs with us so we could see what appears to have gone wrong with a dozen websites using our software.
Since the issue had to do with web hosts running end of life PHP that even the PHP devs no longer support, the issue isn’t nearly as severe as we had feared, however we are going to attempt to try and take those issues into account when we have the time and resources, however you should not be using PHP 5 in the first place.
How do I update from PHP 5?
You have to ask your webhost to do so, sadly that is your only choice.
Nonetheless, to make sure we make our software the best, we have a freelance programming spot open for a PHP programmer who understands WordPress and WordPress security, as we write defense code and features. Understanding how old end of life code will do is a major plus.
We will continue to update our software, however to provide high quality software that helps you . The fastest ways to reach us about an issue like this is currently twitter @planetzuda and email which is in the readme files of our software. We generally check the forums a few times a week or try to check it once a day after a new release.