So we got hacked!!!
and sadly enough hackers deleted our database with 1500 km of data in it.
One fine afternoon I was trying to login to my system but wasn’t able to. I went and checked my Node program and it was saying no user found. I started shivering and my feet went cold, I opened my mongodb and did show tables on my database and my worst fears came true. My database was no longer existing. I had faced similar issue in the past as well, I even asked a question about it on stack overflow, but as a junior/naive and the single dev at my company, I blamed it on extra large size of my mongodb log files which might have tripped it off my RAM and deleted database in the process(I just wanted to give database deletion a reason).But post that one good thing I did was I set up proper logs in MongoDB which would record all my interaction with the database. In case, if db.dropdatabase is executed. I would know exactly from where it was executed. In all my naiveness(I know that’s not a word, but you get it, Right?) I only took backup of my database when I wanted to take the backup and not regularly. Also I was running my Mongodb on port 27017 which is the standard port, with absolutely no authentication and no firewall. Basically, I was inciting the hackers to come and hack me!!
So I checked my logs and I found that somebody from Netherlands(Ip address resolution to a location, can be masked as well) has hacked into our database and executed this command.
I was literally stunned!!!. I never thought that even I could be a victim of hacking… All those posts which talked about security suddenly seem meaningful and the notion that “who would hack us anyways”, went away like a fart in thin air.
That evening was the craziest evening I had ever spent. With absolutely no idea on network security. I set out on a mission to learn everything about security.
I am a backend developer by profession but at Metaiot Technologies (I am one of the founding members of the company and the Software Lead), I handle everything, from frontend, to devops, to PoC’s in different languages and frameworks, to data and latest addition being security and firmware.
The rest of the blog will be about how can you secure your EC2 instance while deploying a MEAN Stack app and avoid problems like me in the future. If you are like me, who used to give 2 cents about security in an App, this posts targets you and wants to sensitize you towards the need of security in your next app.
List of things I did.
- Put on automatic backup in Mongodb via crontab, to keep storing my data safely every one hour. How is here.
- Never run your mongodb on port 27017. It’s the standard configuration.
- Run it using X509 certificates. I am not running a sharded cluster but a simple Replica Set in production, how to securely connect to a replica set is given here. It consists of replica set servers talking securely(internal auth) and also external auth(Nodejs and apps talking to Mongo replicaset). How to make Node talk securely to mongo via mongoose is given here. If you using Native mongodb driver, you are probably in luck as the driver documentation explains it with an example here.
- Seal your servers using Firewall and if you are using Ubuntu 16.04 I recommend using ufw(Uncomplicated FireWalls). How to set it up can be read here. I am only now allowing connections from my trusted servers and ssh.
This is my primary security setup and hackers can probably break it very fast, but I am still in the process of learning. What I would really like is security experts here on Medium and else where who have deployed MongoDB in production to shed more light on this topic for noobs like us and help us build a better, secured web for the world. This is my first post in a series of post in which I will highlight my journey in learning about aspects of Web security and how we handle it.
P.S. The database was actually the test database and we were really close to production, if it would have happened even a week after, we would have been in a big big soup. So a big shout out to the Hackers!!! Thank you for opening my eyes towards security and helping me become a better developer in the process.
