Since it has been a somewhat rocky month for Wuxiaworld and there’s always quite a few posts here as soon as anything goes wrong, I felt I really owed it to you guys to get everything on the record! This will be a long post…but you guys are LN novels, you are okay with reading, right? xD. This post will specifically be on ‘caches’ and our current issues with them.
Okay, so here goes, in a simplified manner, and bear in mind, I’m not a programmer, so I might be wrong on some details. First, the nature of self-hosted WordPress. WordPress websites are dynamic websites. What does that mean? Simply put, in the ‘old days’, websites were static; you created a ‘.html’ page, modified each page, then ‘uploaded’ them. Tadaa! Nice and simple. Buuut, that also meant that every time you wanted to edit a page or to the style, you had to do it all manually (or with the help of certain software). Booohoo!
With WordPress and other dynamic CMS came the rise of ‘dynamic’ websites. With WordPress, there is actually no such thing as a persistent ‘page’; rather, all the information for each ‘link address’ is stored inside a database, which then automatically generates the page on the fly when people come visit it. This is awesome in a number of ways, because then, with the proper software, you can just make modifications to the database, which will then automatically change the pages. It makes being a website manager infinitely easier.
For ‘static’ pages, the only thing which really matters is bandwidth; because all the pages are pre-created, there is very little load on the CPU. For ‘dynamic’ pages, things are different; out of the box, each time a user visits a WordPress website, the CPU of the server has to pull the necessary data out of the database and ‘build’ it into a page for you. Thus, if you have a lot of viewers, that puts a huge strain on the CPU.
Now, let’s talk numbers. Wuxiaworld has 5537 registered users, and around chapter release, we have up to 2200 viewers on simultaneously. Wow! Awesome right? Yes, it absolutely is! Buuuut, here’s the trick; if the pages were all ‘static’ html, the only thing which matters is bandwidth, which is no problem. But since the pages are being generated on the fly, that means that my CPU is being hit with hundreds or thousands of requests (just earlier, we had 9000+ simultaneous connections open to my server!). Eeek! 8-core CPU choking! 8-core CPU choking!
So the solution? Caching. Caching, simply put, is a process by which some software plugins will create a ‘fake’ static page and ‘save’ it; when a request comes in, instead of having the CPU ‘build’ the page, the CPU instead just loads up the prebuilt ‘fake’ static page and sends it to the user. Yaaay!
But caching isn’t a perfect solution either. First of all, you can’t cache people who are logged in, because then everyone would start to see everyone else’s saved information. This, btw, is why logging in fixes the ‘caching’ issues. The second, and more important issue is ‘purging’; what happens when the information on the page changes? If the cache isn’t ‘purged’, then for example, even though I have updated the ‘preview’ chapter in the database to a ‘full’ chapter, the CPU is still serving the ‘preview’ cache for that link, instead of destroying it and rebuilding a new, updated ‘cache’.
So, a critical part of caching plugins is the ability to ‘purge’ and get rid of those ‘fake static pages’ that are now expired. My caching plugin (Supercache) normally does that. The problem is, for some reason, some of the backend changes we made aren’t playing nicely with this ‘caching’ software, causing the cache to either not purge correctly (which happened today), or purge after 15-30 minutes of a new post instead of right away (like for mobile).
So what we are trying to do, going forward, is find another caching plugin that will play nicely with our backend and purge correctly, instead of either delaying 15-30 minutes, or necessitating me manually going in and purging the files by hand like what happened today. Once that happens, then we will be 100% set for good.
There’s a lot of other issues, like the DDOS, CDN implementation, Linux server tuning, etc., but that’s for another day. I just thought I would share with you guys a small part of running a self-hosted WordPress-powered website, some of the challenges, and some of the solutions 🙂