Funny timing for this to happen As I was ‘bragging’ just a couple of days ago, how my projects and servers are supposedly super secure in an article around bitcoin and cryptography, that I will link from ASAP (still has to be published).
Well, turns out I make mistakes too. Upz.
On the upside, I can fix them at lightspeed when it arises, cure and put additional security measures in place from it happening ever again! This article below comes straight from my post @ forum.vestacp.com ..
The reason I’m putting it up here on my blog too, I basically explained here already (just in case the post or forum gets removed). You never know what the future holds, right?
“Hacked by megla akash from Team_CC”
I currently have two servers with VestaCP. My dev server (18+ months up & running) and my newest production server (2+ months old). Both running Ubuntu 16.04 LTS with NGINX as a webserver but with different VPS providers. And the 2nd / my latest server just got hit by a hacker, and it got hit bad?! Not sure yet.
Literally found out about this 10minutes ago.
As that megla.txt with file contents:
hacked by megla akash from Team_CC
showed up in every public_html on my VestaCP powered server of default VestaCP user “admin”. Sadly, have 11 HUGE WP projects for that user with 100+ WP plugins per project. A second VestaCP user on my server with 3 other WP sites, didn’t seem to get affected. But that’s just based on the premise it didn’t contain megla.txt files (as I ran a `find / -name “megla.txt” > results.txt` on the server).
How the Hack …
I’m not only puzzled how it could have happened but also a bit scared what else got uploaded or has been modified on the server files and/or DB wise. Even more so, as I use strong & unique passwords, everything running at custom ports (both SSH as well as VestaCP admin), all sites have LetsEncrypt SSL certificates + CloudFlare, and on a WP levels got advanced & hardened iThemes security running (REST/XML-RPC disabled, no execution of PHP scripts in themes / uploads, long string filtering, illegal character filtering in parameters) with either Wordfence Security or JetPack as a second line of defense, moreover have centralized management (all sites are always up-to-date with MainWP over SSL). It can’t be they got access to my MainWP dashboard, as other things on different servers should have been affected then. And running end-point enterprise internet security on my workstations + a diversity of firewall solutions + pi-hole. Haven’t used public wifi or anything of sorts either. What else? No crazy chmods, chowns, and `su` has to be used to gain root access.
In other words: some help suggestions / insights would be appreciated, how to troubleshoot + fix this properly + prevent it from happening next time. While I’m going to … not really sure where to start. Hence using the F-word in the title. I’m not a newbie, so erhhhh .. its either something really silly that I overlooked or those hackers are truly skilled; WTF?!
(keeping this up-to-date by the hour .. with or without replies):
Progress report / things I’ve done
- removed all megla.txt files. So I don’t get indexed / traced as hacked
- changed password of VestaCP admin user (although it was setup in VestaCP firewall restricted to 1 single ip)
- ran `clamscan -r -i /home` from the KVM. Result: no infected files
- chkrootkit found nothing out of the ordinary either
- MainWP’s suruci sweep on all sites found a few things, but nothing major
I think I figured it out what happened (‘only’ took 7+ hours to figure out! LOL). I also had two WP Multi-User staging projects running on VestaCP admin account. Imported from a prior shared hosting account, and not hooked into MainWP (as that feature doesn’t exist). Aka not up-to-date, neither having a lot of WP hardening in it, as that’s tough to do on Multi-User environment without WPMUdev subscription (something I should have had; in hindsight). Thus I probably got sql hijacked based on old plugins, either 4.7x WP version on the WPMU projects, and from there on the entire VestaCP account got infected? That’s at least my best guess thus far. Should have kept track of the timestamps in the order those megla.txt files were created. Hmz. *update* found a bug in VestaCP (again) combined with NGINX. Wordfence uses .user.ini to create the WP WAF. That — supposedly hidden file — is downloadable on a NGINX server. Makes me wonder what other typical LAMP stack files are publicly accessible on complex WP environments. Scary!
How to fix
Well .. can’t spend too much time contemplating on the cause this, except how I will go over VestaCP’s LEMP webserver templates (again). Thereafter I’m just going to export all the pages, posts, etc. into XML files. Then delete the “admin” VestaCP account, as there seems to be no need to reinstall the server, create a new ‘admin’ account and build everything up from a WP point a view. Thats going to be an intensive weekend. But seems to be the safest option, although the VestaCP backups from yesterday seem unaffected.
How to prevent it
Divided projects over more VestaCP users, and definitely put WPMU projects apart from single installs. And next to all the other security measures I had in play, also have regular audits with that tool shared in the next post. No pain; no gain.
Troubleshoot Addendum: how to tighten security (in a MAGICAL way) !!!
Install a webserver audit tool called “Lynis”. See: https://cisofy.com/lynis/
Do not ‘apt-get’ it though, as the ubuntu repo’s have an older release of it. Instead follow this guide: https://www.digitalocean.com/community/tutorials/how-to-perform-security-audits-with-lynis-on-ubuntu-16-04
Then it’s a mere matter of executing:
lynis audit system
from a remote SSH / KVM terminal, and go through all the tips and suggestions from the terminal output. Thats ‘IT’; another free Information Technology article / guide / howto at its finest. Back to work! And releasing other great freemium and premium content! Including all those wordpress powered teambuilding sites (as the blog you’re on now runs on ‘something’ else).
- My Dutch/NL WordPress Plugins Translation contributions: August 2021 - Sunday August 1st, 2021
- Top 5 Patchmanagement Cloud Software solutions for IT system managers to Patch and Deploy Apps - Monday July 26th, 2021
- My Dutch/NL WordPress Plugins Translation contributions: July 2021 - Thursday July 1st, 2021
- Adobe After Effects (AE) / Photoshop FREE Download sites - Wednesday June 30th, 2021
- Endpoint Protection User Review: Avast Business CloudClare vs TeamViewer - Tuesday June 29th, 2021