[boomshadow.net~]# » Tech

Backing Up and Restoring MySQL Databases

Jacob "Boom Shadow" Tirey — Tue, 27 Nov 2012 13:02:30 +0000

Backing up / Exporting

The best advice I ever heard is: if your data is important to you at all, then you should back it up. What would happen if you suddenly lost it? How devastating would it be? Backing up is such a simple step, and can potentially save you many hours of work, stress, and possibly your job (if the data is that important).

To backup a database, we simply need to export the database. To restore it, we need to import the database. I’ll show you how to do both using a few different methods:

Backing up a Single Database from within cPanel

In cPanel, you can download a copy of your database by simply logging into cPanel and go to the ‘Backup’ section.

Then click on the link for your database name. It will prompt your to download your database in a compressed archive (tar.gz) to your local computer.

Backing up a single database with command line

If you prefer the command line, you can backup your database using a single command:

mysqldump -Q --add-drop-table your_database_name > file.sql

Backing up all databases to separate files

If you would like to play it safe or know that you have many different databases that are all important, then you can run a ‘for’ loop to individually dump all databases to their own file:

Fist create a directory for all the dumps to go and create an empty list of all the database names:

mkdir /root/database_backups ; touch /root/database_backups/list-of-databases

Then, run the for loop to dump the databases and generate the list of database names:

for db in `mysql -e 'show databases' | grep -v Database` ; do mysqldump -Q --add-drop-table $db > /root/database_backups/$db.sql && echo $db >> /root/database_backups/list-of-databases ; done

Restoring / Importing

In the event that you’ve got missing data, corrupt tables, or generally something went wrong, don’t worry! You have backups that you can restore from. Here’s how you restore that databases back from the exports you made above.

Restoring from phpMyAdmin

First off, you’ll need to log into phpMyAdmin. From cPanel, it is found in ‘Databases’ section.

Once inside, you’ll select your database on the left-hand side. This is the database that you want to import that data into.

*** Warning: This will overwrite your existing tables.

Once you have the database selected, click on the ‘import’ tab at the top.

Click ‘Browse’ and navigate to the file on your local computer.

Look over the various options. Most times, you can just leave them as is. If you don’t know what the options are, I’d advise just to leave them set to default.

Hit ‘Go’.

Restore a single database from command line

If you are similar to me, you might prefer a command line. I think its much easier to import it this way as the command is so short & simple:

If you are logged in as root, simply run:

mysql db_name < file.sql

Make note of the direction that the arrow (greater-than sign) is pointing. This indicates the direction where the data is going. In this case, the arrow points to the left towards the MySQL database name. This indicates that the data is being imported FROM the .sql file in-TO the server's MySQL database, which is exactly what we want.

If you don't have root access, you can still import but the command requires a couple more flags and that you know the database credentials:

mysql -p -u db_username db_name < file.sql

It'll prompt you for that user's password. Enter it in and the import will begin.

Restore all databases from command line

Remember that command we did to export all databases to their own separate dump files? Well, lets say you wanted to easily import all of them back in for whatever reason. We just need to run another simple 'FOR' loop in order to get them back in. Basically, we are doing an individual database import, but for each database that is in that 'list-of-databases' that we generated earlier:

for db in `cat /root/database_backups/list-of-databases` ; do mysql $db < /root/database_backups/$db.sql ; done

Once that finishes, you'll have all your exported databases fully restored back as they once were: Fast, safe, and easy!

Final Thoughts

If you have a huge database, running a mysqldump can cause your entire website to go down. This is because you are locking the tables during the dump (cPanel backups can do the same thing). If you are worried about this, be sure to perform your backup export during a time of low site traffic. With that said, SQL backups are a great idea. An equally good idea is to occasionally try to restore your backups (to make sure they are good). Simply restore them to a different database name if you don’t want to overwrite an existing database.

Note about this article

This article is one I had written for the ServInt blog as part of the 'Tech bench' series. You can view it on the ServInt blog here. They are using my article with my permission.

cPanel’s Webinar on WHM 11.34 features

Jacob "Boom Shadow" Tirey — Wed, 07 Nov 2012 19:11:09 +0000

If you didn’t have a chance to catch today’s cPanel webinar on the new WHM 11.34 features you missed out. Surprisingly, the entire webinar lasted a mere 30 minutes and the features portion wasn’t, in my mind, the most interesting section. However, it had some good information on some of the more sought after features. I took note of the three that caught my eye the most:

Email auto-config scripts

One of the biggest changes I’ve always felt was needed was updating the email auto-configs scripts. cPanel has had this feature for quite some time, so its nothing new. The problem was that for the longest time, neither was what it supported. Up until just a few months ago, cPanel supported up to Outlook 2000. These auto-configuration scripts made life easier, when they would actually work. Now, that will change. Although the interface may not specifically say it, the latest cPanel Mail Client Auto Configuration scripts are tested to work with modern OS and email clients such as: Mac Mountain Lion and Windows Mail clients for 2010. Oh happy day!

Webdisk

The Webdisk got an update as well. Webdisk is another feature thats been around for some time, but had grown outdated to the point, again in my opinion, useless. Not anymore! Now, it has auto-config scripts for modern OS’s and even for mobile devices: iPad, iPhone, & Android. Mobile support is accomplished by referring you to a free app: WebDAV Navigator. I tested it out, and it works really well. Essentially, it has the power to turn your server into a private drop disk, complete with different user permission levels. You can setup as many webdisk user accounts as you like, give some of them read-only permission, even set them all to have the same directory, and bam! You have a community dropbox area on your server. I found it easy to setup, easy to use, fast, intuitive, and highly accessible for my phone. Directly from the App, it prompted me to upload photos from my photo library or take a new photo. I may start using it over FTP in many situations, such as quick blogging. This new version really brings Webdisk back into my everyday usage.

Email Archiving

Email archiving is a recurring feature that I’ve seen over the years. I even wrote up an article for a custom way to set this up with Exim. My article is happily now outdated and unnecessary. cPanel has done a great job of doing what my method did and better! It has custom levels of saving email for your personal preference or for compliance with your countries laws. You can tell it how log to save what types of email. For example, you can have it save all outgoing messages for 30 days while all incoming gets saved for 3 months. The archive is IMAP and Webmail accessible, or you can download a backup from within cPanel.

To enable email archiving, go to ‘Tweak Settings’ inside of WHM.

Note: Email archiving DOES count towards individual user’s disk quotas.
Note 2: Email archiving settings can be transferred with an account when doing WHM to WHM transfers, but both servers must have WHM 11.34.

Q&A

As much as I love the above features, I actually think the Q&A section yielded far more interesting information, but I suppose more in the terms of answering cPanel gossip:

1) 11.34 going to ‘Stable’ release

The new 11.34 is going to be moved to the next release Tier (Stable) in as soon as the next week or the week after. This means ever more clients will be affected. If you work in or around the hosting side of things, I’m sure you’ve had quite the influx of customers contacting you saying that their server is telling them that their MySQL or Operating System is out of date. Well, with 11.34 going to ‘Stable’ release, prepare to be surged with even more contacts. Thankfully, where I work, we’ve got things down to a science and upgrading either the OS or MySQL is easy and second nature.

2) New Interface moving to cPanel

cPanel confirmed that they are pushing this new feature rich visual WHM interface out to the individual customer cPanel interface in the next version (11.36). You will want to be sure you are well versed in it’s navigation and new features. You’ll probably have a bunch of your customers asking you some questions about the ‘new look’ when that release goes out.

3) Mobile support

The next version will see WHM have iPhone and Android support. This will be accomplished NOT through an App, but rather through HTML5 and CSS. cPanel’s head developer put explained that with a fabulous reason: ….if we create an iPhone/iPad app, that will leave Android behind. If we make an Android app, it will leave iOS behind. If the development is done through hooks and device detection in HTML5, they can have a mobile compatible solution for ALL devices, even Blackberry.

4) Regular Updates

cPanel has stated that they are moving to a “three versions-per-year” release system. The next one (11.36) will be coming out at the very beginning of next year. Thats a very quick turn-around, but I’m confident they can do it while continuing to release exciting and well developed feature like they have in 11.34. Their newly organized animal named developer teams and new cPanel Feature request/discussion system (http://features.cpanel.net/) will go a long way to seeing that accomplished.

Conclusion

I hope you find this information helpful. If you missed out on the Webinar and want to see it, no worries. cPanel will have it posted to their Video page later end of this week: http://videos.cpanel.net/category/webinars/

How to still get a free DynDNS account

Jacob "Boom Shadow" Tirey — Thu, 18 Oct 2012 14:01:59 +0000

What Happened to DynDNS

Sometime around November of last year (2011), the popular Dynamic DNS provider, DynDNS severely limited their free account offerings (See reference link #1 at the bottom of the page). They eliminated new free accounts from being created, citing that users could still get the 14 day pro trial. Pre-existing free accounts were grandfathered in, but with 2 very important restrictions placed on them:

Limited to 2 hostnames
They will expire if not updated every 30 days

Many existing free users found that they were dropped down to two while some had their accounts expired because they didn’t do the update within a month. Needless to say, this upset quite a few people and sparked much debate (See reference links #2,3,& 4 at the bottom of the page). I understand why they did it; they wanted to increase their profits. However, the problem was that there were many how-to guides out there that referenced DynDNS’s free service; there were many hardware devices such as routers and IP cameras that were hard-wired to use DynDNS.

Today, I’m going to show you a loophole that still allows for you to get a free DynDNS account.

The Discovery

While exploring the options of my D-Link router, I came across Dynamic DNS page. Normally, I skip straight to the credentials portion at the bottom where I’ve already got my pre-existing DynDNS free account setup. However, I see that D-Link offers ‘their own’ Dynamic DNS. See image:

Click image to see larger view

I was curious to see if their service was any good and what limitations hey placed. Their Dynamic DNS is offered at: http://www.dlinkddns.com/

I noticed something very interesting when poking around the FAQ and How-to. They say your credentials are usable at www.dyndns.com. See attached screen shot:

Click image to see larger view

To me, this says that D-Link’s service is merely a reselling of DynDNS’s service.

Confirmation

I created my account and then immediately popped over to DynDNS’s site to see if my credentials actually worked. They did! See image:

Click image to see larger view

In fact, not only did I have a free DynDNS account, I actually have access to ANOTHER free hostname. As you see in the above picture, it says I’m using 0 of 1 hostname, even though I’m already using one. D-Link’s FAQ only has one question/answer and it states that you can only have 1 single hostname. https://www.dlinkddns.com/faq. Well, I’m here to tell you apparently that’s not true. You can use DynDNS’s site interface to create another hostname; simply click on “Add New Hostname” and you’ll be taken to the setup screen. See screenshot:

Click image to see larger view

I can tell you that my legitimate, free, grandfathered-in DynDNS account doesn’t have that option. Also, once I add another hostname, the link for creating more hostnames goes away.

I have a few notes of interest: Using D-Link’s DynamicDNS is probably only for people who purchase and use their D-Link brand routers, though it doesn’t actually state that anywhere. I don’t know if these types of accounts are applicable to the 30 days update rule. I would assume so, as their assumption when they created this deal with D-Link was probably that the accounts would be used for that Dynamic DNS section in the router. This means that all active accounts will be on auto-update from the router.

Conclusion

In conclusion, you can use D-Link as a proxy for creating not just one but TWO free dynamic hostnames from DynDNS. So long as you keep them updated every 30 days, they should be good. I’ve been using mine to keep a home server public facing using ddclient as my update tool. I know this loophole is one that may close to new users as well, and then at that time I’ll have to branch out to other Dynamic DNS providers; but for now, I can keep creating new hostnames if need be, just use a different email account to sign up at D-Link’s site.

Personal note:

I don’t abuse this system. I only have need of 2 Dynamic hostnames, and that’s all I have made: my grandfathered-in account and the D-Link account. However, I like to know that I have means to create more if need be. Options are nice.

References

SSH Language Localization

Jacob "Boom Shadow" Tirey — Fri, 08 Jun 2012 00:09:28 +0000

# Updates:
# 06/20/12 | Revised for more details

When you get a new VPS, you’ve got a powerful new system that is a server all unto yours. There is quite a bit of customization you can do to make it suite your needs. Today, I am going to show you how to change the language of your SSH terminal.

Changing your VPS language setting allows you to type and sort by special characters not available in English. But more than that, it is language localization. You pick not only your language, but your country as well. This allows the server to reset date formats, use of commas and points in numbering systems, display options, etc. Bear in mind that changing VPS language settings won’t translate things such as file names or contents of files.

In addition, most GNU programs on your server use localization, wherein they customize their interface based on the VPS language localization settings. Intelligent applications using internationalization can greatly reduce the difficulty of interfacing with a server for non-English speakers.

While logged into your server via SSH, you can see what the current language setting is by running the following command:

locale

The output of the command will be something like this:

LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

en_US is the language code (English) followed by the country code (United States).

To change the language of your system–as well as all the other parameters above–you will modify the LANG. To see list of all the supported locales, run:

locale -a

You’ll notice that names on the list are named with the format: Language_Country. Simply looks for the two letter code for language and country that you want. For example, if you want to set it to Spanish/Puerto Rico, you would select: es_PR. If you want to set it to Farsi, you would use: fa_IR

Check out this handy guide to look up the right code for your language and location: http://lh.2xlibre.net/locales/

Once you have picked out the language and location you want to use, you will need to edit the following file:

/etc/sysconfig/i18n

And then simply modify the line the reads something like this:

LANG="en_US.UTF-8"

You will need to close your SSH session and restart a new session for the changes to take effect.

*Note: If you are using Terminal on a Mac, you may need to also type the following for your session to load the updated configuration:

su -

If you are having trouble getting these changes to take effect and show up when you initiate another locale command, then check that the default language is not set. In Nano, look at the entire file. Which lines are preceded by hash marks (#)? Those lines are designated as “comments” and not read by programs accessing the file. For example:

#LANG="en_US.utf8"
#SUPPORTED="en_US.UTF-8:en_US:en"
#SYSFONT="latarcyrheb-sun16"
LANG="C"

In this example, the LANG will always be the default (C). To change this, switch the line that is read by erasing and creating hash marks:

#LANG="en_US.utf8"
#SUPPORTED="en_US.UTF-8:en_US:en"
#SYSFONT="latarcyrheb-sun16"
LANG="C"

Now the session configuration will be based on the first line–not the fourth–and you can replace the language/location information as desired.

Example:

Here is a ‘Before’ language change:

Here is an ‘After’ language change (to Farsi):

Note about this article

This article is one I had written for the ServInt blog as part of the ‘Tech bench’ series. You can view it on the ServInt blog here. They are using my article with my permission.

How to forward – archive all email for a user

Jacob "Boom Shadow" Tirey — Thu, 03 May 2012 16:18:57 +0000

# Updates:
# 11/07/12 | As of the release of the new WHM 11.34, some of the archive tasks of this article can be done directly with cPanel. cPanel now has it’s own built-in support for email archiving. See my WHM 11.34 features article here.

Today I am going to show you something pretty nifty: how to forward a copy of all incoming and outgoing mail for a particular email user. If you are the cautious type, or simply the email hoarding type like myself , this can be really handy for easy archival of all your mail. Exim can be configured so that your server will automatically (and quietly) send a copy of every message you send and receive over to another email address. This is all done without you noticing anything extra in the ‘To’ or ‘CC’ fields. Silent and effective.

This could also be used to secretly keep track of your users’ emails and monitoring what they do. I had a friend of mine recently ask me to help him set this up for him on a new Sales employee he had. He wanted to make sure they were taking care of the clients properly. While I may not personally agree with the privacy concerns of spying on another person’s email, I acknowledge that it can be used in this purpose and there are those out there who have no problem implementing it for this reason.

However, that is an argument of principles best saved for another time. I personally believe that all knowledge is power and people will do with their lives as they feel is best. I will show you the tech side of how to set this up; you can do with it as you please.

*Note: You will need SSH root access to make the following changes

Find the Exim System Filter file

The changes we are making will be to the ‘System Filter File’. Obviously, we’re gonna need to know where it is located. It could be different on your cPanel box, so be sure to look it up.

Log into WHM and navigate to the section: Main >> Service Configuration >> Exim Configuration Manager. Alternatively, you can simply type ‘Exim’ into the search bar and it will be the first result.

Next, scroll down and locate the “System Filter File” section. It should list the default location. On my server, it is:

/etc/cpanel_exim_system_filter

Leave WHM open for now because we will be coming back to it shortly.

Edit the Exim System Filter file

One thing I must note is that you will not actually be directly editing the system filter file. cPanel staff has stated that if you do, the changes will get overwritten during cPanel updates. Instead, we are going to make a copy of it and make our changes to the new custom file. cPanel updates will not change your custom file.

*Note: Exim version changes, however, WILL overwrite custom files. When cPanel 11.34 comes out, Exim is likely to get a version change. Make sure you back up this file ahead of time.

Let’s get started. Log into SSH and copy the system filter file:

cp /etc/cpanel_exim_system_filter /etc/cpanel_exim_system_filter_custom

Next, edit the newly made file:

nano /etc/cpanel_exim_system_filter_custom

And place the following template at the very bottom:

### FORWARD ALL INCOMING AND OUTGOING MAIL FOR A USER ###

if ("$h_to:, $h_cc:, $h_bcc" contains "user@domain.com")
   or ("$h_from:" contains "user@domain.com")
then
   unseen deliver "archive-address@domain.com"
endif

The code above will send a copy of any email that is sent “To”, “CC”, and even “BCC” to your address. It will also send a copy for all outgoing mail. Be sure to replace the example addresses above with your actual email addresses. Save and exit the file.

Reconfigure & Restart Exim

Now, you need to go back to WHM and change the system filter file location. Basically, we’re going to reconfigured Exim to use that new file we made instead of the default one. Copy and paste the full path to your new file into the 3rd field:

Scroll down and hit ‘Save’. Finally, all you need to do now is restart Exim:

service exim restart

And that is that! Your server will now start archiving all your email to the address you specified. You’re going to start getting a lot of extra mail at that address so be sure you have some automatic message organization setup such as User level filtering in cPanel or Message filters in your email client to sort them all the messages into folders. If not, you may have a little bit of a messy inbox next time you log in

References

Special thanks goes out to cPanel Tristan. You always know your stuff and are great at helping! I really enjoy getting you in the ticket system and then also finding fixes in the forums that happen to be posted by you as well. It makes my day every time. Research for this article found on the cPanel forums here

Also, thanks goes to Vinayak on the cPanel forums here

Change a document root on cPanel

Jacob "Boom Shadow" Tirey — Fri, 30 Mar 2012 12:36:57 +0000

What is the document root?

When you visit a web site, you are accessing a particular folder on the web server. The server knows to only serve up those files located at that folder to incoming visitors. The location of that web folder is called the “Document Root”.

It is similar to using a coat check. You present your ticket to the attendant and they fetch it from the back room.
In this metaphor, the coat is the site you want to visit and the attendant is the web server. The visitor doesn’t know the exact location where the coat is stored, but the coat does reside at a specific location.

For example, when you visit johns-carpentry.com, the server is pulling up the files at: /home/johnc/public_html. The document root is set by the Apache configuration.

What we are going to discuss today is what if you wanted to change that location? You would need to change the ‘Document root’ for the domain. cPanel’s default location may not serve your needs or you simply want to reorganize. In any case, I’ll show you how to make that change on cPanel.

Changing Addon Domains

There are two types of domains on a cPanel box that can have document roots: Main (primary) domains and Addon domains. Addon domains are easy to change the document root. Simply log into your cPanel and navigate to: Domains >> Addon domains

Next, edit the Addon domain path. To do so, simply click the edit icon next to the path, and type in your new path.

It’s that simple!

Changing Primary domains

For changing the main/primary domain, you will need to have root SSH access. Edit the following (replacing your user & domain info):

/var/cpanel/userdata/USERNAME/DOMAIN.COM

Look for the following line:

documentroot: /home/USERNAME/public_html

Modify it according to your needs. Save it and exit.

If you are using an SSL on the domain, be sure to update the SSL’s template as well:

/var/cpanel/userdata/USERNAME/DOMAIN.COM_SSL
documentroot: /home/USERNAME/public_html

Then, rebuild the Apache conf and restart Apache:

/scripts/rebuildhttpdconf
service httpd restart

The change will be immediate. Simply clear your browser cache and force refresh the page!

Note about this article

This article is one I had written for the ServInt blog as part of the ‘Tech bench’ series. You can view it on the ServInt blog here. They are using my article with my permission.

How to install PHP memcache

Jacob "Boom Shadow" Tirey — Tue, 28 Feb 2012 13:32:25 +0000

This guide will help you install PHP Memcache on a CentOS server.

What is PHP memcache?

In my previous article, I showed you how to install Memcached, the service daemon. Now, if you would like your PHP software to interface with that daemon, you will want to install a PHP extension for it.

There are actually two separate implementations of a PHP Client that wraps the memcached (daemon); both are provided via the PECL library. One is called ‘memcache’ and the other is called ‘memcached’. I know that it is a little confusing that ‘memcached’ shares the same name as the daemon itself, but it IS a separate PHP wrapper.

This article will focus on installing ‘memcache’ as it is the most commonly used. You can use the information to install ‘memcached’ as well; the installation steps are the same. The main difference is that ‘memcached’ requires the libMemcached library. You can see a comparison of the two different PHP clients here: http://code.google.com/p/memcached/wiki/PHPClientComparison

Installation, the quick method

The easiest method is to simply use PECL’s ‘install’ command. This will grab the latest stable release, configure it with the default options, and add it to the server’s php.ini:

pecl install memcache

*Note: Un-installation is just as easy:

pecl uninstall memcache

Installation, the manual method

The quick ‘install’ method uses the default configuration options and should serve most peoples’ purposes. However, if you need more control over the installation options, you can manually install from source and add it flags as you need.

Download the source package via PECL:

cd /usr/local/src/
pecl download memcache

Unpack the tar and enter into the newly extracted directory (be sure to replace the X’s with your downloaded version):

tar -xvzf memcache-X.X.X.tgz
cd memcache-X.X.X

Configure and install:

phpize
./configure && make && make install

Verify the install

Verify the installation by displaying all the installed PHP modules. See if ‘memcache’ is listed:

php -m
memcache

Problems with manual installation

The ‘make install’ should load the module into your server’s extension directory automatically. However, if you do not see memcache listed in a ‘php -m’, you will need to add the module manually

First, you’ll want to very the location of your server’s extensions directory:

grep extension_dir /usr/local/lib/php.ini

It should return something similar to the following:

extension_dir = "/usr/local/lib/php/extensions/no-debug-non-zts-20060613"

Next, copy the memcache module into that directory (be sure to replace the path with the one found on your server):

cp modules/memcache.so /usr/local/lib/php/extensions/no-debug-non-zts-20060613

Finally, add the module to your php.ini:

echo 'extension=memcache.so' >> /usr/local/lib/php.ini

Thoughts

It used to be a common issue that the quick ‘install’ method could not be used for servers where the /tmp partition was mounted ‘noexec’. The problem was that the ‘configure’ process could not execute because the PECL client downloaded the module into in /tmp. When /tmp is mounted ‘noexec’, servers cannot execute scripts from /tmp.

However, PECL now downloads and executes the script from /root/tmp. No need to worry about compiler errors with /tmp. You can use the ‘install’ method even with a secured /tmp.

Note about this article

This article is one I had written for the ServInt blog as part of the ‘Tech bench’ series. You can view it on the ServInt blog here. They are using my article with my permission.

How to install Memcached

Jacob "Boom Shadow" Tirey — Wed, 08 Feb 2012 19:42:38 +0000

# Updates
# 01/02/13 – Updated RPM locations

This guide will help you setup Memcached on a CentOS server.

What is caching

Before diving into Memcache specifically, let’s take a step back. What is caching? Why should you care and why should you use it? Caching is used for two very important reasons: to speed up the delivery of the pages and to alleviate system resources. Caching is used to speed up dynamic sites; database driven sites will benefit most form caching. Think about a WordPress site. Each page you visit is not an actually file, but rather an amalgamation of the theme, widgets, posts, footers, headers, etc… Each time a page is accessed, PHP will generate the page requested on the fly from the database. It takes time to query the database to create the page. These database queries put a strain on the resources of your server.

However, what if instead of continually generating a new page, the same page, for every visitor, you were to turn those pages into static HTML files? No database querying is needed for the new visitors. A static file can be served up much faster and with significantly less resource consumption. Your visitors see their requested page sooner, and you save on CPU cycles. Everyone is happy. This is what caching does.

What is Memcached

Memcached is a general-purpose distributed memory caching system. It is one of the most popular caching tools and is used in such popular sites as: YouTube, Reddit, Zynga, Facebook, and Twitter.

According to Memcached’s official site, Memcached is defined as: Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.

In simpler terms, it decreases database load by storing objects in memory.

Installing Memcached (daemon)

The quickest and easiest method would be to install via Yum. First, you must grab the RPM that matches your OS:

CentOS 6 (64 bit):

su -c 'rpm -Uvh http://mirrors.kernel.org/fedora-epel/6/x86_64/epel-release-6-8.noarch.rpm'

CentOS 6 (32 bit):

su -c 'rpm -Uvh http://mirrors.kernel.org/fedora-epel/6/i386/epel-release-6-8.noarch.rpm'

CentOS 5 (64 bit):

su -c 'rpm -Uvh http://mirrors.kernel.org/fedora-epel/5/x86_64/epel-release-5-4.noarch.rpm'

CentOS 5 (32 bit):

su -c 'rpm -Uvh http://mirrors.kernel.org/fedora-epel/5/i386/epel-release-5-4.noarch.rpm'

CentOS 4 (32 bit):

su -c 'rpm -Uvh http://mirrors.kernel.org/fedora-epel/4/i386/epel-release-4-10.noarch.rpm'

Now, to install it with Yum:

yum install memcached

Start the memcached service:

/etc/init.d/memcached start

Configure the memcached service to start when the server boots:

chkconfig memcached on

Finally, disable the RPM so that it is not used for future Yum functions:

perl -pi -e "s/enabled=1/enabled=0/g;" /etc/yum.repos.d/epel.repo

Testing

It’s always a good idea to check your work. Make sure that memcached is running:

ps auwx | grep memcache

PHP memcache

For information on how to install the PHP extension so that your PHP software can interface with memcached (daemon), see my article here: How to install PHP memcache

Note about this article

This article is one I had written for the ServInt blog as part of the ‘Tech bench’ series. You can view it on the ServInt blog here. They are using my article with my permission.

Spoofed Bingbot attacks against WP

Jacob "Boom Shadow" Tirey — Thu, 02 Feb 2012 17:19:04 +0000

# Updates
# 06/18/12 – Bingbot attack returns, but attacks the wp-admin folder instead of just the login page. Updated to patch that. Also added 403 Error handling
# 02/04/12 – Revised wording to reinforce Bing’s perspective
# 02/02/12 – Modified block code to identify bingbot as ‘badbingbot’ in case ‘bingbot’ is already being used somewhere

In the last 24 hours, I have seen a large scale attack on the admin logins for thousands of WordPress sites. The attackers are using the User Agent ‘Bingbot’. Bingbot certainly is NOT the culprit in these attacks, but is, unfortunately, being spoofed. These spoofing bots are attacking the wp-admin page. So far, I have only seen this affect WP Mage and Mage Monster users.

A brute force attack on any login is certainly undesirable; you obviously don’t want any unauthorized person accessing your admin area. However, the real problem comes from the CPU strain this is putting on the hosting server. These constant requests to hundreds of websites on the server will drive up the CPU load, effectively tying up the resources. The server will not be able to serve up pages to legitimate traffic or legitimate search engine bots that are trying to crawl the sites.

There is a solution you can use to combat this problem. The solution is to block the Bingbot user agent from accessing your these admin pages. An IP based firewall block would do no good as the source IP’s have no common denominator; the IP’s are too spread out and come from many different geographical locations.

By implementing the following fix, you can reduce the loads back to normal operating levels allowing your site’s pages to load once again and you will not affect your rank with Bing. After a couple of days, the attack should have subsided and you should be able to safely reverse the changes. However, per Duane Forrester of Bing’s webmaster tools, you do not have to undo the block. There is no harm in permanently preventing your admin pages from being indexed (for reference, see his comment here).

Are you affected?

You can run a quick check to see if your sites are being hit with this spoofed Bingbot attack. Check your sites’ domlogs and look for any Bingbot User Agent requests that are hitting WordPress admin pages:

grep bingbot /usr/local/apache/domlogs/* | grep wp-login

If you receive numerous results from running that, especially from various IP addresses, it means that you are being affected. If you see many ‘POST’ commands in there, it means they are also trying to log into WordPress.

Example:

/usr/local/apache/domlogs/domain.com:24.153.219.159 - - [01/Feb/2012:15:27:18 -0500] "GET /wp-login.php?redirect_to=http%3A%2F%2FDOMAIN.COM%2Fwp-admin%2F&reauth=1 HTTP/1.1" 200 2206 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
/usr/local/apache/domlogs/domain.com:24.153.219.159 - - [01/Feb/2012:15:27:18 -0500] "POST /wp-login.php HTTP/1.1" 200 3084 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"

How to block them

Now that you’ve verified that this directly affects you, time to block them and restore your server’s vitality.

Edit the pre-virtual host conf

We need to tell the server what is a “bad bot”. In this case, it is Bingbot. Then, we tell it to block it:

nano /usr/local/apache/conf/includes/pre_virtualhost_global.conf

At the bottom of the file add:

SetEnvIfNoCase User-Agent ".*bingbot" badbingbot


	order allow,deny
	allow from all
	Deny from env=badbingbot


        order allow,deny
        allow from all
        Deny from env=badbingbot

If you do not want to block Bingbot on every domain and would rather only block it on a few particular affected domains, you can, instead, make the change to each domains’ virtual host via an include file. For more information, see cPanel’s documentation here: http://docs.cpanel.net/twiki/bin/view/EasyApache3/InsideVHost

Restart Apache

Now it is time to restart Apache so that our changes can take effect:

service httpd restart

Test it

Which Apache restarted, you should see your CPU load start dropping immediately. You will want to test it to make sure that bingbot is successfully blocked. Do what the attackers are doing, access your page while spoofing your User agent as Bingbot:

wget -U "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" http://domain.com/wp-admin/

Be sure to change ‘domain.com’ with your actual domain name. You should receive a 403 Forbidden error. If not, the file that is downloaded should be completely empty, meaning no content was served up.

403 Error Handling

When successfully blocked, the bot will be sent to the 403 Forbidden page (as you saw in the test above). However, depending on your theme, the 403 page may be a custom error page that still loads part of the site and many of the plugins. This will still cause a heavy strain on the server because it has to process this 403 error page for every request. You have two options: Option #1. Set the 403 page for every affected site to be a lightweight page with little text. You will want a minimalistic page that will not cause large CPU load if it is accessed dozens or even hundreds of times. Option #2. Disable the 403 page temporarily.

If you have many WordPress sites that are all being affected (this typically would fit Mage), then it would just be quicker to opt for Option #2.

To do that, edit the following file:

/usr/local/apache/conf/includes/errordocument.conf

Comment out the line about 403 Forbidden. Once done, the top of the file will look like the following:

# 400 - Bad Request
ErrorDocument 400 /400.shtml

# 401 - Unauthorized
ErrorDocument 401 /401.shtml

# 402 - Payment Required
ErrorDocument 402 /402.shtml

# 403 - Forbidden
#ErrorDocument 403 /403.shtml

# 404 - Not Found
ErrorDocument 404 /404.shtml

Then you must restart Apache and you’re done! The load issues should clear up.

Notes:

The attack should subside within a few days. You do not have to undo your changes you made. You can leave the admin page blocks in there. No search engine should ever need to index or even access your admin pages. This is true for any CMS, not just WordPress.

If you have implemented the disabling of the 403 Page, be sure to undo that one. You may actually want your 404 pages back when the attack has passed

Auto fix for file permissions and ownership

Jacob "Boom Shadow" Tirey — Sat, 22 Oct 2011 08:03:16 +0000

Updated: 03/30/13

Click here to see the Change Log

suPHP and FastCGI require files and folders to have a specific set of permissions/ownership from other handlers. Without these permissions set you will see a lot of errors such as: “403 Forbidden”, “500 Internal Server Error”, or simply generic errors that commonly have the word ‘permission’ in them.

It can be very time consuming to track down and check file permissions across a whole server. Luckily, fixing this on a cPanel box can be scripted. This gives us a quick and very easy script you can wget to any cPanel server. Simply run the ‘fixperms’ script, specifying the user (or all users), sit back and watch the errors just disappear. I use this script daily in my administrative work and it never fails! It is simply a good generic fix if you cannot find your permission problem, or if you have just switched your handler and need a quick way to change every user account on the server.

Credit does not go to me though. A good buddy of mine, Colin R., wrote this for ServInt. Thanks Colin for making lives easier!

***WARNING!!! The following scripts are intended for suPHP or FastCGI. If you are not running either of these two handlers, be aware of how the script works and the changes it makes. The code is posted at the end of this article; please take a moment to review it. For example, when running DSO, some files/folders may need to be owned by ‘nobody’ in order to function properly (such as in certain WordPress functions or PHP based file uploads). Running this fixperms will set everything to USER:USER. Under DSO, this is potentially not a problem for most sites, except a few core functions may not work. You can always change specific files later if any errors pop up.

Furthermore, it is highly recommended that you run a full backup of your server before running fixperms or any other script that makes changes to multiple files.

This ‘fixperms’ script is intended for cPanel servers only. It is dependent on cPanel’s internal scripts and file structure. If you’re on anything else (such as Plesk), it will simply fail to run. It won’t be able to do anything.

I know that criteria sounds very specific, but those two conditions cover a large number of the reseller/multi-user hosting servers out there. And that’s really the crowd that would benefit most from an automated script such as this.

That all being said, if you are running suPHP or FastCGI, press on; for this script will work flawlessly for you and potentially save you a TON of time & hassle.

Fixperms – for one single user

To use the fixperms script, simply log into your server as root, wget the file from our server, then run it. Type in the cPanel username and it will run only for that particular account.

It does not matter which directory you are in when you run fixperms. You can be in the user’s home directory, the server root, etc. The script will not affect anything outside of the particular user’s folder.

wget boomshadow.net/tools-utils/fixperms.sh
sh ./fixperms.sh -a USER-NAME

Fixperms – for all of the users

If you would like fix the permissions for every user on your cPanel server, simply use the ‘-all’ option:

wget boomshadow.net/tools-utils/fixperms.sh
sh ./fixperms.sh -all

Verbosity of Fixperms

By default, the script runs in a ‘quiet’ mode with minimal display. However, if you’re like me, you may want to see everything that is happening. You can turn on verbosity and have the script print to the screen everything that is being changed. I find this extremely useful when fixing large accounts that have many files. You can watch the changes as a sort of ‘progress bar’ of completion. The ‘-v’ option can be used per account or with all accounts.

For one single account:

sh ./fixperms.sh -v -a USER-NAME

For all accounts:

sh ./fixperms.sh -v -all

The code itself, what’s in it?

I understand that it can be a big security concern to simply ‘wget’ a file from a website you found, and then blindly run it on a production server. I understand your fear; I’m right there with you and would likewise be leery and very hesitant. However, I promise you that there is no malicious intent in this or anything you will ever get from my site. I have pasted the content of the file below for your examination.

Click here to see the code:

#! /bin/bash
#
# Date: Jan 26th 2012
# Author: Colin R.
# Revisions: Jacob "Boom Shadow" Tirey (boomshadow.net)
# Fixperms script for ServInt
#
#   Fixperms script for cPanel servers running suPHP or FastCGI.
#   Written for ServInt.net
#   Copyright (C) 2012 Colin R.
#
#   This program is free software: you can redistribute it and/or modify
#   it under the terms of the GNU General Public License as published by
#   the Free Software Foundation, either version 3 of the License, or
#   (at your option) any later version.
#
#   This program is distributed in the hope that it will be useful,
#   but WITHOUT ANY WARRANTY; without even the implied warranty of
#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#   GNU General Public License for more details. http://www.gnu.org/licenses/


# Set verbose to null
verbose=""


#Print the help text
helptext () {
    tput bold
    tput setaf 2
    echo "Fix perms script help:"
    echo "Sets file/directory permissions to match suPHP and FastCGI schemes"
    echo "USAGE: fixperms [options] -a account_name"
    echo "-------"
    echo "Options:"
    echo "-h or --help: print this screen and exit"
    echo "-v: verbose output"
    echo "-all: run on all cPanel accounts"
    echo "--account or -a: specify a cPanel account"
    tput sgr0
    exit 0
}

# Main workhorse, fix perms per account passed to it
fixperms () {

  #Get account from what is passed to the function
  account=$1

  #Check account against cPanel users file
  if ! grep $account /var/cpanel/users/*
  then
    tput bold
    tput setaf 1
    echo "Invalid cPanel account"
    tput sgr0
    exit 0
  fi

  #Make sure account isn't blank
  if [ -z $account ]
  then
    tput bold
    tput setaf 1
    echo "Need an account name!"
    tput sgr0
    helptext
  #Else, start doing work
  else
    tput bold
    tput setaf 4
    echo "Fixing perms for $account:"
    tput setaf 3
    echo "------------------------"
    tput setaf 4
    echo "Fixing website files...."
    tput sgr0
    #Fix individual files in public_html
    find /home/$account/public_html -type d -exec chmod $verbose 755 {} \;
    find /home/$account/public_html -type f | xargs -d$'\n' -r chmod $verbose 644
    find /home/$account/public_html -name '*.cgi' -o -name '*.pl' | xargs -r chmod $verbose 755
    chown $verbose -R $account:$account /home/$account/public_html/*
    find /home/$account/* -name .htaccess -exec chown $verbose $account.$account {} \;

    tput bold
    tput setaf 4
    echo "Fixing public_html...."
    tput sgr0
    #Fix perms of public_html itself
    chown $verbose $account:nobody /home/$account/public_html
    chmod $verbose 750 /home/$account/public_html

    #Fix subdomains that lie outside of public_html
    tput setaf 3
    tput bold
    echo "------------------------"
    tput setaf 4
    echo "Fixing any domains with a document root outside of public_html...."
    for SUBDOMAIN in $(grep -i document /var/cpanel/userdata/$account/* | awk '{print $2}' | grep home | grep -v public_html)
    do
	tput bold
	tput setaf 4
	echo "Fixing sub/addon domain document root $SUBDOMAIN...."
	tput sgr0
	find $SUBDOMAIN -type d -exec chmod $verbose 755 {} \;
	find $SUBDOMAIN -type f | xargs -d$'\n' -r chmod $verbose 644
  	find $SUBDOMAIN -name '*.cgi' -o -name '*.pl' | xargs -r chmod $verbose 755
    chown $verbose -R $account:$account $SUBDOMAIN
    find $SUBDOMAIN -name .htaccess -exec chown $verbose $account.$account {} \;
    done

	#Finished
    tput bold
    tput setaf 3
    echo "Finished!"
	echo "------------------------"
	printf "\n\n"
    tput sgr0
  fi

  return 0
}

#Parses all users through cPanel's users file
all () {
    cd /var/cpanel/users
    for user in *
    do
	fixperms $user
    done
}

#Main function, switches options passed to it
case "$1" in

    -h) helptext
	;;
    --help) helptext
	    ;;
    -v) verbose="-v"

	case "$2" in

		-all) all
		       ;;
		--account) fixperms "$3"
			   ;;
		-a) fixperms "$3"
		    ;;
		*) tput bold
     		   tput setaf 1
		   echo "Invalid Option!"
		   helptext
		   ;;
	esac
	;;

    -all) all
	  ;;
    --account) fixperms "$2"
      	 	;;
    -a) fixperms "$2"
	;;
    *)
       tput bold
       tput setaf 1
       echo "Invalid Option!"
       helptext
       ;;
esac

So there you have it. An effective permissions fix for your cPanel account. When you run this, people will think you’re a hero! So, go forth and save your users from the evils of site errors!

Note about this article

This article is one I had written for the ServInt blog as part of the ‘Tech bench’ series. You can view it on the ServInt blog here. They are using my article with my permission.

Change Log:

03/30/13 | Added support for Subdomains and Addon domains that have their document root outside the public_html folder, Cleaned up the output display, Removed the cPanel mail perms script because it essentially does nothing, moved fixperms.sh location to a central ‘utilities’ folder (left a 301 redirect for the old location)
01/03/13 | Corrected the inconsistency between using -all and –all (when employing verbosity)
06/03/12 | Updated warning verbage for DSO
05/10/12 | Bug fix for file names that contain special characters
01/26/12 | New feature rich script written by Colin R.