I was reading a book by Ashley Davis on bootstrapping microservices*, and at the beginning of the book, the author sums up his philosophy of development as:
Iterate
Keep it working
Move from simple to complex
Fair enough! That sounds like a reasonable dev philosophy. Even if I found something in there I dislike, I can’t argue with the man. It’s his philosophy, and it’s his right for it to be whatever he likes it to be!
Reading those three bullet points made me realize that I don’t have a development philosophy, and I’ve been developing for over a decade! It’s time for me to sum up my dev philosophy!
My development philosophy summed up, v0.0.1:
Explore and prepare
Break down the complexity to micro implementable chunks
Develop with end-users in mind
Optimize and iterate
Keep shipping
1) Explore and prepare
I’m not the one to jump into the codebase without reading the manual first. Based on my anecdotes, that makes me a minority. People usually jump right into the code and figure out the bigger picture afterward. I prefer to have the big picture (if possible) before I start changing things.
2) Break down the complexity to micro implementable chunks
Decomposition is my key to handling complex systems. It’s a no-brainer, but it’s fascinating how often people forget about it when they take on a problem. I always try to break things down to the smallest possible unit of work when I’m working on a project.
3) Develop with end-users in mind
It’s easy to forget when writing software that actual people will use it. Often, you want to finish the damn thing as quickly as you can and continue with your life. In the end, it’s all about your fellow human beings. That mantra makes me a better developer, or I’d like to think so.
4) Optimize and iterate
I nearly didn’t add this one because it’s so obvious and implied for most developers. I did choose to add it because it’s one of the biggest and arguably the most important parts of the development process.
5) Keep shipping
Unless it’s something done for the pure enjoyment of doing it, software should be shipped and used. Perfect is the enemy of good. Waiting for all the stars to align to publish your code is rarely a good thing to do. Ship early, ship often.
For some time now, I have wanted to set up a blog using a static site generator. There was no real reason behind that wanting. It was just the plain old curiosity combined with the drive to learn new things.
I’m not sure when exactly the trend began, but I remember articles named “I switched my blog to [Hugo/Jekyll] ” started popping up across the interweb frequently some two years ago.
It wasn’t until I had to write a documentation page for one of my side projects that I started to think seriously about SSGs.
I ended up writing documentation manually in HTML at that time. I just wanted to push it out ASAP, and I rejected the thought of reading the documentation of any SSG, even for measly 10 minutes.
Even though I didn’t use SSG for that task, it was at that moment that I decided I will most definitely use a static site generator for my web properties that are existing purely for displaying static content. Hell, I’m a developer, no excuses not to.
Starting with WordPress
When I decided to write my first blog post last year in April, I wrote it in vim. It was sitting on my disk for two days because I had no idea what to use to publish it online.
SSGs were popular, but the decision fell on WordPress because I have installed it many, many times before. It was familiar, and I felt comfortable with it. I knew the plugin ecosystem, and I knew I could connect to the WP dashboard from anywhere and type out a post inside its WYSIWYG editor and publish it – without having to think about anything else.
I connected to the VPS I rent, installed PHP, MySQL, and Nginx. I extracted the WP zip and went through few screens of the installation procedure. After that, I took the freely available theme and pasted my blog post into the WP editor. That was all I had to do – 10 minutes from start to finish. Easy!
If I wasn’t obsessed with optimizing things that don’t matter much, and I didn’t consider it as a fun way to spend Saturday evenings, I would leave it at that. I didn’t because I am obsessed with things some would consider stupid – like optimizing a blog that holds a dozen blog posts.
Things that started to frustrate me
When I wrote the first blog post that got some attention, hundreds of people started commenting on it. There were some positive comments and some comments that were not so nice. I didn’t care about any of them but one. It was the comment that said my blog was slow.
I was thinking – here I am, a guy that talks about optimizing stuff all the time, and my website consisting only of text is slow AF. And it was. There was no excuse. I was making people download a megabyte of data only to display a few lousy paragraphs of text.
Just for the front page where the last ten blog posts were listed, your browser had to:
download 1.04 MB of data
make 38 requests
wait for 1.56 seconds
And that was after an optimization plugin for WordPress combined and minimized the assets and cached what it could. Awful! The only thing I could do that was worse would be not displaying the text to people that have Javascript disabled. I made sure that wasn’t the case. I don’t want to end up in hell.
Now, to be clear, this is not a WordPress problem. It was my problem. If I had built the theme from scratch, I would have more control. In that case, I could’ve been more careful about the assets I was linking.
I could’ve had a better caching mechanism and maybe a CDN in front. That would make the experience better for the reader. It would only mask the underlying problem, though. I’m serving only text – why the fuck do I need so many resources?
Hello Hugo
It boiled down to the decision between Hugo and Jekyll in the end. I forgot why I chose the former. Maybe it was someone’s tweet or a blog post I read.
Initially, I thought I’ll need to spend at least a day going through the documentation. I was mistaken. The documentation is well written, and almost everything has an example.
In the end, I didn’t even use any of the WordPress migration tools. I don’t have many posts (not yet!), so I decided to convert them manually to markdown files. Also, it gave me the chance to re-read the content I wrote and check if I can find any typos to fix.
If documentation is easy to follow, you may ask why the hell it took me over a month to finish the migration, because I’m posting this on Jun 27th and I started migrating the blog on May 17th.
It was mostly being occupied with other things. I planned to finish this in two months, putting in an hour or two per week into it. I’ve been investing more of my off time into writing – something I will do more of this year (and now I have a faster blog!).
In total, I spent between ten and twelve hours on migration. It was mostly me changing some CSS settings and reversing them. I don’t have an eye for design, and it’s always been hard for me.
I started with a theme I thought looks cool – Anatole, and then I added my custom styling on top of it. In the process, I played with the layouts and learned most of the insides of Hugo and its theming. It will come in handy in the future, I have no doubt about that.
Regarding the process of writing a post and publishing it on the blog, it’s as simple as it can be
Write a blog post in markdown
Run hugo to build it
SCP to the server
When I say simple, I’m saying it’s simple and easy for me – a person with a programming background that feels comfortable with markdown and command line.
I’m not trying to say that this is something that everyone should do. I believe that WordPress provides a great platform for aspiring bloggers that don’t want to or don’t know how to go through this process.
Just give me the stats
Here are the stats for the website after the migration. I can say it’s much better now. There is still room for improvement, of course, but I’ll refrain from going fully mad and trying to cut down more stuff. I’m happy with it for now.
To show you the front page of my blog, your browser now needs to:
It’s all fun and games until someone loses an eye. Likewise, it’s all fun and games until someone loses access to their private and/or business data because they trusted it to someone else.
You don’t have to be an expert seeker to be able to quickly duck out (it’s like the verb ‘googling’, but used to describe searching the interwebs through a decent search engine, like DuckDuckGo) all the stories about little guys being fucked over by “don’t be evil” type of corporate behemoths.
A drinking game recommendation (careful, it may and probably will lead to alcoholism): take a shot every time you find out how someone’s data has been locked and their business was jeopardized because they didn’t own, or at least back up their data.
Owning your data and your tools
Owning your data is more than just having backup copies of your digital information. It’s also about control and privacy. It’s about trust. I don’t know about you, but I don’t trust a lot of services with my data (the ones I do are few and far between).
As this is a post about self-hosting, I won’t start preaching (trust me, it’s hard for me not to) how you should consider switching from WhatsApp to Signal, Google Maps to OpenStreetMap, or how you should quit Instagram and Facebook. You’re creating a lot of data there, and they don’t do pretty things with it. Fuck, I’m already preaching. Sorry about that.
Note: I’m not fully off social media. I’m using Twitter and LinkedIn. Everything on Twitter is public/disposable and I don’t use their private messaging feature. LinkedIn is there for professional correspondence and I will start to taper it off slowly, but that one is tough to quit.
Disclaimer: I’m aware most of the people are not power users, and not everyone will want to spend time learning to set up their own alternatives to the services mentioned and create the backup strategies as I’ve done. It does take some time (mostly to set everything up) and some money. If you’ll take anything from this post, it should be to always back up your data (yes, even though it’s replicated across 5 Google’s datacenters). If shit hits the fan, it may take you a while to adopt new tools or flows, but you will still have your backup. Do your backups early and often.
What’s my setup?
I created a simple diagram to roughly show how my personal setup works. Before you say anything – I’m aware that there’s a group of people that wouldn’t consider my self-hosting as pure self-hosting. I’m using Vultr* to host my web-facing applications and not a server in my house. Unfortunately, the current situation doesn’t allow me to do that (yet).
So, here’s the diagram. The detailed explanation continues below.
I’ve separated the diagram into 4 parts – each part represents a different physical location of the data.
The part that gets the most action is the yellow part, living in the cloud.
Vultr
I’m living in Germany, so the obvious choice was to spin up my instances in Vultr‘s* data center in Frankfurt, as ping is the lowest to that center for me.
Right now, I have six compute instances running there. You can see types of cloud compute instances in the picture below. It’s pretty similar to what you would get from DigitalOcean or AWS EC2.
Why did I choose Vultr*? They have pretty good customer service there, and I just happened to stumble on them before DigitalOcean got big and popular and AWS became the leader in the cloud computing game. Having said that, for purely private use, I wouldn’t opt for AWS even if I had to choose now. I’ll leave it at that.
The breakdown looks like this:
1 x $10/mo VPS + 1 x $5/mo object storage – Nextcloud
1 x $5/mo VPS – various dev tools + analytics (Plausible)
2 x $10/mo VPS – several web projects that I run for myself and friends
Everything combined costs me $55 per month.
Nextcloud
Nextcloud is the powerhouse of my everyday data flow and manipulation. With the addition of apps, it’s a really powerful one in all solution to serve as an alternative to widely popular offerings of the FAANG crowd. Once properly set up, not much maintenance is needed.
Tasks are the alternative to Todoist or Any.do which I used previously.
Notes are the alternative to Google Keep. Not as fully featured as Evernote or OneNote I have also tried out at one point, but it’s good enough for me.
Calendar is the alternative to Google Calendar I used previously.
Contacts are the alternative to Google/Samsung contacts I used previously.
I’m also able to stream music from Nextcloud to my phone, using Nextcloud music. For the client, you can use anything compatible with Ampache or Subsonic. My choice is Power Ampache. I’m not streaming a lot of music though. I always have 30-40 GB of MP3s on my phone that I rotate every now and then.
All the data from Nextcloud is in sync with Synology at my home via CloudSync. A big plus is a nice dark theme for the web UI:
Gitea
I’m a developer and more than I need the air to breathe and coffee to drink, I need version control. My weapon of choice is git, which is lucky because there are a lot of hosting solutions for it out there. I was thinking about this one for a while and it boiled down to GitLab vs Gitea.
For my needs, GitLab was overkill, so I went with Gitea. It’s lightweight, easy to update, and just works. Its interface is clean and easy to understand, and because UI is similar to that one of GitHub, people that I collab with find it as a seamless switch. On the negative side, if you want to customize it, it can be a pain in the butt.
Monica CRM / Kanboard
Monica is a personal CRM. Some people think I’m weird because I’m using a personal CRM. I find it awesome. After I meet people, I often write down some information about them that I would otherwise forget. I sometimes make notes about longer phone calls if I know the information from the call will come in handy later on. Colleagues’ and friends’ birthdays, ideas for their presents, things like that – they go into the CRM.
Kanboard is a free and open-source Kanban project management software. I use it to manage my side projects, but I also use it for keeping track of books I read, some financial planning, study progress tracking, etc. It’s written in PHP, it’s easily customizable and it supports multiple users. Usually, when I do some collaborations, I will immediately create an account for that person on both Gitea and Kanboard.
Development tools & Analytics
Plausible is my choice for analytics that I use on several websites that I own. It’s lightweight, it’s open-source, and most importantly – it respects your privacy. There’s a how-to that I wrote on how you can install it on an Ubuntu machine yourself. The bonus thing is that I really like developers’ approach to running a business. They have a cool blog where you can read about it.
Development tools that I’m mentioning are basically a bunch of scripts I have developed and gathered over time. Text encoders/decoders, color pickers, WYSIWYG layout builders, Swagger editor, etc. If I use something often and it’s trivial to implement on my own, I’ll do it.
Home
Desktop PC and NAS are part of my ‘home’ region.
Desktop is nothing special. I don’t play video games, and I don’t do any work that needs a lot of computing performance. It’s the 8th generation i5 with integrated graphics, 1TB SSD, and 16 gigs of RAM. OS I’m using is Ubuntu – the latest LTS version. It’s installed on both the desktop and laptop.
Everything except the OS and the apps is synced in real-time to Synology by using the Synology Drive Client.
Synology NAS I’m using is the DS220j. It’s not the fastest thing, but again, it works for me. I have two Western Digital Red drives (shocking, huh?), 2TB each.
Every last weekend of the month, I will manually backup all the data to Blu-ray discs. Not once, but twice. One copy goes to a safe storage space at home and the other one ends up at a completely different location.
Offsite backup
This is my ‘everything is fucked, burnt or stolen’ situation remedy. I’m not particularly happy with the physical security I’ve set up at home, so one of the concerns is the theft of disks and backups. Other than moving to a different location where it would be easier to work on upgrading the physical security, my hands are tied regarding this subject (not for long hopefully).
Other things could happen, like fire, flood, etc. Of course, it’s a bit of a hassle, but I believe in being prepared for any type of situation, no matter how improbable it may seem.
Laptop & Smartphone
When you’re self-hosting, it will, naturally, also reflect on the apps that you use on your portable devices. Previously, my phone’s home screen was filled with mostly Google Apps – calendar, keep, maps, drive. There were also Dropbox, Spotify/Deezer. It’s different now.
I have De-Googled my phone, using /e/ and F-Droid. There are compromises you’ll have to make if you choose to go down this path. Sometimes it will go smoothly, but sometimes it will frustrate the hell out of you. It was worth it for me. I value my freedom and privacy so much more than an occasional headache caused by buggy software.
This is the list of the apps I use frequently that are related to self-hosting:
OsmAnd~ – global mobile map viewing & navigation for offline and online OSM maps
PulseMusic – my main music app that I use to listen to the music collection stored directly on my phone (30-40GB at any time that I rotate from time to time)
Nextcloud – this is the sync client for the phone and a file browser
K-9 Mail – really, really ugly looking email client that is also the best email client for Android I have ever used
As I mentioned previously, my Laptop is running the latest Ubuntu LTS, just like the desktop PC. To have things partially synced to the NextCloud, I’m using the official desktop client. Listing other tools that I use as a developer that may be tangibly related to self-hosting would be another two thousand words, so I won’t go into that right now.
Conclusion
Is it worth the time and hassle? Only you can answer that for yourself.
Researching alternatives to commercial cloud offerings, and setting everything up surely took some time. I didn’t track it, so I can’t say precisely how much time, but it was definitely in the double digits. If I had to guess, I would say ~40 hours.
Luckily, after that phase, things run (mostly) without any interruption. I have a monthly reminder to check for the updates and apply them to the software I’m running. I don’t bother with minor updates, so if it’s not broken, I’m not fixing it.
If I motivate even one person to at least consider the option of self-hosting, I will be happy. Feel free to drop me a message if you decide to do that!
* Links to [Vultr](https://www.vultr.com/?ref=8305838) contain my referral code, which means that if you choose to subscribe to Vultr after you clicked on that link, it will earn me a small commission.
