Funny timing for this to happen As I was ‘bragging’ just a couple of days ago, how my projects and servers are supposedly super secure in an article around bitcoin and cryptography, that I will link from ASAP (still has to be published).
Well, turns out I make mistakes too. Upz.
On the upside, I can fix them at lightspeed when it arises, cure and put additional security measures in place from it happening ever again! This article below comes straight from my post @ forum.vestacp.com ..
The reason I’m putting it up here on my blog too, I basically explained here already (just in case the post or forum gets removed). You never know what the future holds, right?
“Hacked by megla akash from Team_CC”
I currently have two servers with VestaCP. My dev server (18+ months up & running) and my newest production server (2+ months old). Both running Ubuntu 16.04 LTS with NGINX as a webserver but with different VPS providers. And the 2nd / my latest server just got hit by a hacker, and it got hit bad?! Not sure yet.
Literally found out about this 10minutes ago.
As that megla.txt with file contents:
hacked by megla akash from Team_CC
showed up in every public_html on my VestaCP powered server of default VestaCP user “admin”. Sadly, have 11 HUGE WP projects for that user with 100+ WP plugins per project. A second VestaCP user on my server with 3 other WP sites, didn’t seem to get affected. But that’s just based on the premise it didn’t contain megla.txt files (as I ran a `find / -name “megla.txt” > results.txt` on the server).
How the Hack …
I’m not only puzzled how it could have happened but also a bit scared what else got uploaded or has been modified on the server files and/or DB wise. Even more so, as I use strong & unique passwords, everything running at custom ports (both SSH as well as VestaCP admin), all sites have LetsEncrypt SSL certificates + CloudFlare, and on a WP levels got advanced & hardened iThemes security running (REST/XML-RPC disabled, no execution of PHP scripts in themes / uploads, long string filtering, illegal character filtering in parameters) with either Wordfence Security or JetPack as a second line of defense, moreover have centralized management (all sites are always up-to-date with MainWP over SSL). It can’t be they got access to my MainWP dashboard, as other things on different servers should have been affected then. And running end-point enterprise internet security on my workstations + a diversity of firewall solutions + pi-hole. Haven’t used public wifi or anything of sorts either. What else? No crazy chmods, chowns, and `su` has to be used to gain root access.
In other words: some help suggestions / insights would be appreciated, how to troubleshoot + fix this properly + prevent it from happening next time. While I’m going to … not really sure where to start. Hence using the F-word in the title. I’m not a newbie, so erhhhh .. its either something really silly that I overlooked or those hackers are truly skilled; WTF?!
(keeping this up-to-date by the hour .. with or without replies):
Progress report / things I’ve done
- removed all megla.txt files. So I don’t get indexed / traced as hacked
- changed password of VestaCP admin user (although it was setup in VestaCP firewall restricted to 1 single ip)
- ran `clamscan -r -i /home` from the KVM. Result: no infected files
- chkrootkit found nothing out of the ordinary either
- MainWP’s suruci sweep on all sites found a few things, but nothing major
I think I figured it out what happened (‘only’ took 7+ hours to figure out! LOL). I also had two WP Multi-User staging projects running on VestaCP admin account. Imported from a prior shared hosting account, and not hooked into MainWP (as that feature doesn’t exist). Aka not up-to-date, neither having a lot of WP hardening in it, as that’s tough to do on Multi-User environment without WPMUdev subscription (something I should have had; in hindsight). Thus I probably got sql hijacked based on old plugins, either 4.7x WP version on the WPMU projects, and from there on the entire VestaCP account got infected? That’s at least my best guess thus far. Should have kept track of the timestamps in the order those megla.txt files were created. [b]Hmz. *update* found a bug in VestaCP (again)[/b] combined with NGINX. Wordfence uses .user.ini to create the WP WAF. That — supposedly hidden file — is downloadable on a NGINX server. Makes me wonder what other typical LAMP stack files are publicly accessible on complex WP environments. Scary!
How to fix
Well .. can’t spend too much time contemplating on the cause this, except how I will go over VestaCP’s LEMP webserver templates (again). Thereafter I’m just going to export all the pages, posts, etc. into XML files. Then delete the “admin” VestaCP account, as there seems to be no need to reinstall the server, create a new ‘admin’ account and build everything up from a WP point a view. Thats going to be an intensive weekend. But seems to be the safest option, although the VestaCP backups from yesterday seem unaffected.
How to prevent it
Divided projects over more VestaCP users, and definitely put WPMU projects apart from single installs. And next to all the other security measures I had in play, also have regular audits with that tool shared in the next post. No pain; no gain.
Troubleshoot Addendum: how to tighten security (in a MAGICAL way) !!!
Install a webserver audit tool called “Lynis”. See: https://cisofy.com/lynis/
Do not ‘apt-get’ it though, as the ubuntu repo’s have an older release of it. Instead follow this guide: https://www.digitalocean.com/community/tutorials/how-to-perform-security-audits-with-lynis-on-ubuntu-16-04
Then it’s a mere matter of executing:
lynis audit system
from a remote SSH / KVM terminal, and go through all the tips and suggestions from the terminal output. Thats ‘IT’; another free Information Technology article / guide / howto at its finest. Back to work! And releasing other great freemium and premium content! Including all those wordpress powered teambuilding sites (as the blog you’re on now runs on ‘something’ else).
- My Dutch / NL WordPress Translation contributions: October 2022 - October 9, 2022
- My Dutch / NL WordPress Translation contributions: August 2022 - August 23, 2022
- [NL] Ultra Light digital nomad travel in Europe with Rynair, Wizz Air, etc. - August 20, 2022
- My Dutch / NL WordPress Translation contributions: July 2022 - July 27, 2022
- My Dutch / NL WordPress Translation contributions: April 2022 - April 2, 2022
Communities I have been 'hanging out' regularly over the past two decades.
- Nomadlist.com since 2022-04
- Empire.Kred since 2021-07
- Karma-Lab.com since 2019-01
- Invision Community since 2018-05
- Insanely Mac since 2018-02
- eGPU.io since 2017-04
- Tech Inferno since 2016-08
- forum.VestaCP.com since 2016
- WordPress.org since 2015-12
- Couch Surfing since 2014
- BitcoinTalk.org since 2014-03
- Warrior Forum since 2013-07
- Empire Avenue since 2013
- Envato (i.e. CodeCanyon) since 2011-07
- TripAdvisor since 2010-09
- HardWare.info since 2003-08
- SynthForum.NL since 2001-11
- gathering.tweakers.net since 2000-11
Note: links go to my profile on those respective sites. Connect with me if you like.
- EarnWithEarnie.io since 2022
- EarnWithEarnie.net since 2018
- BestProfitsTeam.com since 2017
- CycleClubMembers.com since 2015
- BestSTIFORPteam.com 2013-2021