Skip to main content

Website backend and Homelab

·

So for anyone curious on How i host my website, this time in full detail, you are in the right place! Here i wil list all software i have deployed and that run the website. For more information you can ask/contact me.

What is a Homelab?
#

So i run whats called a Homelab at home. Now not everyone might be familiar with the term, but Homelabbing is basically when you host stuff yourself at home for experimental or privacy reasons. You wont be relying on a major cloud provider or bigtech company and you yourself have full control over it. A homelab consists of hardware and software. The hardware can be anything from a dedicated server to your grandpa’s old computer to a mobile phone, to a raspberry pi or arduino, the choices are limitless, as long as your hardware can support and run what you want. And the software will most of the time be Free and Opensource, but it can also be proprietary software. Just make sure its downloadable by the consumer without having to sign any contract. And finally you need some sort of host Operating System which can run that software. Most of the time a Linux distro like Debian, Ubuntu, Fedora, Arch, etc etc. This Operating System will be inbetween the software/services you will be running and the physicall hardware of the host system, but you can use pretty much any OS.

My Homelab
#

My homelab consists of a few old computers that are scattered around my bedroom and which i either built/bought myself or some i got donated to me by family and friends. I am an huge FOSS (Free Opensource Software) and selfhosting fan and want to free myself as much from bigtech as possible as bigtech cant be trusted and they will always find ways to make more money by squeezing out the juice of their consumers (and not in a good way). I dont have a server-rack which some homelabbers do have, but for now I am okay with that and its also called a HomeLAB and not a HomeDATACENTER. But i might possibly move to a dedicated serve rack with some nice gear and 10-20Gig fiber internet if i later have my own house and stuff. But for now it does the job! I am still in the process of moving away from bigtech and i use some bigtech stuff myself still to this day, but i try to find a balance between comfort/possibility and privacy and try to step away from bigtech wherever possible/i am okay with going through some challenges/hassle. In the end going completely bigtech-free is near impossible as a lot of technologies that are specifically made by bigtech, run the world and economy. But then i will say one of my favorite sentences: “Close the doors you can close”, litteraly meaning try your best to get as much stuff converted as possible and close the (back)doors you can close and thus increasing your privacy and independence as much as you can.

My servers/physicall hardware
#

Internet/network
#

So because i live at my parents house, i am limited in networking. Our ISP is vodafone/ziggo and they have whats called a HFC network. This network consist as the name says off a hybrid between fiber and coaxial. Fiber is laid ffrom the datacenters to the street’s distribution box and from that distribution box to the houses runs regular old style coaxial cables. This works fine but this is not full FTTH (Fiber to the home). Recently we had a 750Mbps download and 70 Mbps upload plan and had one of those compal modems with the problematic Intel puma chipset. Our ISP ended up giving us a free 1Gbps download, 100Mbps upload plan, which also came with some newer gear 9sagemcom mode and 2 Internet extenders/Aps. This works fine and this is what is the basis of my homelab. Because my family is not that tech savvy, i am also the one who installed and is in charge of that network so thats amazing! I am kind of like the IT tech support guy when it comes to stuff like printers, wifi, windows issues and more. This is also the network i make use off myself for personal stuff like entertainment and relaxation.

Because i could not run an ethernet cable to upstairs where my bedroom is which is where all my homelab equipment is, means i am using the wifi extenders that came with our modem. They wirelessly connect to the main modem+router in our fuse cabinet downstairs and make it so i get pretty good wifi and signal upstairs and i can connect and ethernet cable from those Aps into my homelab gear. So to seperate my homelab from our family’s network, the first thing the ethernet cable from the AP connects to, is an old HP 280g2 computer that runs opensense and that isolates my network as its basically a firewall+router solution. And this makes it so i can use a different subnet for my servers. I wrote an Article about implemnenting opnsense. After the opnsense router i connect a cat6 cable to my network switch, an Zyxel gs1900-8 port switch which i installed OpenWRT on. Sadly i accidentally overwrote both partitions on the flashchip so i am stuck with OpenWRT, but it has not let me down yet so i dont mind using it. To that switch connect all computers i have running as servers and my main workstation laptop conects to it via an USB c dock.

My main computer/laptop.
#

SO i use a laptop as my main daily driver. Its not as refined as a good ol desktop but it does the job, although it can be loud. But i needed a laptop for engineering class as we rely on stupid windows-only softwares which is kinda sad. But the laptop is better than the pc i built as it has a laptop 5070 and has an i7 and 24gb ram, which my custom built desktop did not have (it did not have a discrete gpu). It was a gift by my parents because i passed highschool in 1 go and also because i needed a powerfull laptop for my study, so i am gratefull for that! If my study did not rely on windows specific software or needed a beast of a laptop, i would have gotten a cheaper Linux laptop or even a macbook for portability. But this laptop is also my daily driver and i am, already used to carrying it around and its my daily driver for linux, windows and games and i really like it and i am used to it by now! It connects to a USB c dock via a thunderbolt cable so i can easily unplug or plug in my setup so i can take the laptop to school and when i am home, connect it up again via just 1 cable (and a charger).

Main Server
#

The main server is my old gaming desktop that i built a few years ago. Because for my study i needed a windows gaming laptop, this pc was standing around the place doing pretty much nothing. And it had quite beefy specs, an I5 12400 6 core 12 thread CPU and 16 Gigabytes of ddr5 ram and had a 1TB hdd and a full ATX motherboard with m.2 slots and a gigabit ethernet port. THis is perfect as its the most powerfull pc i have currently and 12th gen is awesome! This runs Proxmox, a debian-Linux based operating system that acts like a hypervisor. Basically an environment thta allows running multiple containers/virtualmachines hsoting your services on just 1 machine. This makes it so you dont need a bazillion servers to host your stuff. Also its often used in enterprise setting and it allows making backups of those VM/containers so you can restore a backup when something goes wrong. And the 12th gen cpu, i can always upgrade to a I7 12th or 13th gen or any CPU with more cores.

As i said it has quite a few m.2 slots and has a 1TB hdd. I use the 1TB hdd for bulk VM/container storage and a 500GB sata ssd for storing VMs/containers that need fast disk speeds. I have a 256Gb nvme ssd that stores the OS. I plan on adding a GPU later for some local AI llm (large language models). As i want to try hosting my own AI thats free of bigtech and wont sell your data. Because i got the laptop for my study and its way more powerfull, this is the best task my old trusty self-built desktop can run.

NAS-Storage
#

So i also have a pc which is my NAS. Whats a NAS? Well, NAS stands for “Network-Attached-Storage” and its basically a computer that acts like a storage server. It holds some hdds and runs its own Operating system and then apears like a network drive. Its similar to google drive or onedrive but independent of bigtech and any cloud. My nas is a PC i got from Marktplaats (a Dutch marketplace, kind of like EBAY) for 100 euro (first ever recent deal i gotten in my life basically). It also has reasonable specs. An I5 10400, 8 Gigabytes of ddr4 ram and a micro ATX board. I put it in a new case which has more space for hdds. The case itself is gigantic, but ti does provide a lot of space inside and has big fans that are kinda loud but i am used to it. This pc runs truenas Scale which is a debian based NAS operating system thats very popular and is often used in other datacenter settings too.

I bought 4 2TB hdds from ebay that where refurbished and from a reputable seller and where in perfect health. The drives are not your standard SATA dekstop/laptop drives but rather use whats called SAS (Serial Attached SCSI). SAS is often used in enterprise/datacenter settings. Because the drives where SAS, i need whats called a HBA (Host Bus Adapter). Basically i need an interface device so i can talk and interface to/with the drives. Thats an LSI 9207 which supports up to 8 drives. Funfact: Sata drives can work on a SAS system/HBA card but reverse does not work. Its a budget card as i am not that rich but it works fine. It does need to be in IT mode so it can work with softwares/operating systems like Truenas. This nas is where i store my backups, personal files, media and photos on as i want to try and abandon some subscriptions and cloud services as “trust-nobody” is what i think of companies these days. Also it stores the backups of all the VMS i have. I do have it offload to a cloud service as you should always have a backup for your backup and you will regret to not have a backup when your NAS fails and it being the only thing that had your data on it that just died.

Homeassistant-smarthome
#

So i also deployed Homeassistant. Homeassistant or HA for short is a opensource smarthome platform thats powered by the community and can be customised as much as you like and supports multiple devices and mostly can be run offline and selfhosted. This is where my smarthome automations and stuff live on and where all my IOT devices connect to. This makes it so i can turn off the lights in my bedroom upstairs from my phone in any place of the house. And because its 100% Local and runs on its own hardware, it means no bigtech or China/America/Russia/Israel can snoop what you are doing.

It was originally created by a Dutch developer called Paulus Schoutsen, who initially wanted to make a few python scripts to control his smart lights but it quickly grew out to a whole community ran smarthome system that is backed and owned by a independent company called Nabu casa. The lead developer/programmer: Franck Nijhof is also from the Netherlands. Its not as intuitive or “plug and play” as traditional commercial available smarthome platforms like Apple’s Homekit, Google’s google-home and Amazon’s Alexa, but it is a lot more private and you 100% know that no data is shared with the just mentioned companies unless you choose to do so. Its based on python but also integrates some javascript/css and also some Yaml for configuring stuff. More info about Homeassistant is on Wikipedia and on the Homeassistant website.

Homeassistant supports 2 different deployment methods. You can deploy it in a container or you can run the Operating System distribution of it on a mini pc. I chose the last option. The Operating Sytem is called HaOS (Homeassistant OS) and its a custom Linux distribution built using Buildroot, which is a build environment and toolset that allows building purpose made Operating systems for emebdded or Application specific tasks. The Mini pc i run it on is a dell optiplex 3040 mini and it has an i3 6100, 8GB ddr3 ram and has a 500GB laptop drive. The HaOS deployment does support all features staright out of the box. Ha is great due to the endless integrations and addons that the community made to support pretty much every device in existence. I also can put my mind at peace, knowing that any internet or tech-company outage wont cause my smarthome platform to break and that i can always turn on or off my lights and that no data leaves my network without my consent.

Containers and VM’s
#

So i mostly host containers. Containers are a sort of environment that you can run any service and its dependencies in. It uses the kernel of the Host system and its just an environment and not a new system. VM’s or Virutal Machines on the other hand do actually allow you to run a full OS and it basically “Emulates” a computer. In this you can run any OS like windows/linux/BSD or whatever you might like. Its basically a computer in a computer. I wont go over every service or container/vm i have as i have many already and not all are important/related to this website for the moment. If you are interested in that, you can ask me on social media or mail me or look on my tiktok or other socials cause sometimes i post stuff there from behind the scenes.

Blog
#

So the blog uses a container as i dont need a full kernel/os running and just need minimal dependencie and overhead.

Hugo & Theme
#

So to generate the website, i chose a solution called Hugo. It allows generating a full website template with ease and its Opensource too! Hugo also has support for themes and the community has already released a whole bunch off them.These themes make it easier as you dont have to manually write every line of css or fill in the bare website and shaping it into whatever you want. You can just choose a theme, configure the parameters the theme expects/exposes and you can have a nice looking website in a little bit off time. Those themes also provide partials where you can define custom footers or can add comments.

Hugo itself can be customised a lot and you can even write/import some off your own code by either including it as partials or putting it in the /layouts/ folder. Thats how i managed to import my own 404 page and added the comments and likes sytem and customised the footer. Also i managed to change the default assets for the theme or hugo by putting my own android-chrome-{dimension}.png and apple-touch-icon.png and favicons in the /static/ folder. You can add

Posts
#

You can put posts on your website by putting them in the /content/posts/ folder. Alternatively you can add content in the /content/ folder. You can generate a new post using the below command using the hugo framework by just pasting them in terminal.

  • Add new post:
hugo new content/posts/{my-awesome-post}.md
  • Add new content
hugo new content/{my-content}.md

This generates the folder and or markdown file. The posts and content are written in markdown. Its a bit more fancier than regular text as it allows custom formatting and such. This makes it useful to write in diferent styles/sizes and embed code or commands in a neat way. Github README also is written in markdown and some of you might already be familiar with it. Its also commonly used on blogs, hence I wanted to use it and Hugo supports markdown really well.

nginx
#

As the main webserver solution serving the static HTML and CSS of the page(s) and all the content, i deployed Nginx. Nginx is a very popular solution and can function as quite some things. It can function as a webserver, proxy, mail server, load balancer, http cacher. I use it’s webserver function for the blog and other services and use the reverse-proxy bit in my nginx-proxy-manager instance running locally on the network. It simply serves the static Html and javascript and css template that hugo generated. I created the hugo website on my local computer and then used rsync to transfer it to the container of the webserver and into its /var/www/blog.309electronics.net directory. Then when the webrequest hits Nginx, it serves the website.

Anubis
#

This website makes use of Anubis to protect the blog and all my hard work and hand-typed work from Ai scraper bots. Ai scraper companies have changed the social contract around scraping, so a simple Robots.txt file will not really work anymore. Ai scraper companies will simply ignore this file as its only passively telling them to not do something. Its like telling a bully he should stop bullying. In many cases you need to stop them in other ways.Thats why i chose a captcha.

Sadly most captchas are ran by bigtech like cloudflare or amazon or whatever and i want to stay away from them. So i started searching for an Opensource solution. Then i came across some when I noticed that the OpenWRT, Wine’s bugzilla, rpmfusion, steam developer, techinfodepot websites all ran on a thing called Anubis. This is an automated captcha that provides any user that visits, a small (Proof-of-work) challenge that proves that the user can run javascript in the browser and that the user is likely a human trying to visit and is not some automated bot/system. Its not the best implementation, but it works for many quite well/good enough. So i decided to also adopt Anubis onto my website and deploy it. And while i am not a huge anime fan at all, i do like seeing the cute anime character pop up while it runs the challenge instead off a boring white corporate-blank page.

I also love that its an in-development project by someone who made it out of frustration/dissatisfaction. Ai bots where scraping and taking down the server off the developer “Xe Iaso” who made Anubis, so they came up with Anubis to ward off bots. Sometimes the best projects come out of dissatisfaction/frustrations with existing stuff. Linus Torvalds also made the Linux kernel out of dissatisfaction with the current UNIX situation and Richard Stallman also started GNU out of dissatisfaction with current copyright rules related to code. So if you are curiuous about all the Anubis mentions scattered around the blog or are curious about why you get to see the page below, this hopefully answers it. Its just to defend against the bad side of Ai that we unfortunately have to deal with these days, as those scrapers show no mercy and the AI hype has only just begun and AI simply needs tons of data to be trained to be actually usefull.

Anubis can also be configured a lot and you can fine tune it to your likings. It allows setting policies for specific useragent strings or identifiers (basically the make and model of a car, but then for web browsers, so you cna identify what browser or service accesses your website) and even allows changing its behaviour based on server load. I will always be monitoring the server and maybe change some Anubis policies or optimize them.

Anubis makes use of cookies to store a session token. This token is valid for 6 hours and it makes it so Anubis can detect if it has already verified you or not. Otherwise everytime you would visit the website, it would have to check you again and again and thats just incovenient. SO it does use cookies but purely functional ones that wont sell any data and are just like your ’ticket’ for visiting the website so security (anubis) knows if you have access.

Anubis automatic captcha page
#
Anubis

The Likes-API
#

So i also wanted to add a Likes system to my blog, so people could like the post and interact. Just without google’s firebase API ofcourse. So i started thinking about solutions. Some googeling, doom scrolling and Claude Ai use later, i came up with a solution! Creating a Likes system is not the hardest task. All you need are a database to store the likes amount in, some service that exposes that database and allows interacting and you are done.

SQLite
#

For the database i will use SQLite, which is a pretty simple database solution which often just consists off a single file with “.db” extension. This will store all the likes and have an entry per post.

Python FastAPI
#

So for the Interactivity bit and the bit that will make it an API, i will use Python. Yes it might not be the fastest or prettiest language, but a Likes-Api can be very simple and also Python is the language i first coded in as a beginner so i am a bit more used to coding in python. And its good enough for the task. Python provides a FastAPI Module/library that allows users to easily create an API for a website. This will be the skeleton off the API. Below is the frotnend code:

<script>
  const API = "https://api.{yourwebsite}.com";
  const POST = window.location.pathname;
  const STORAGE_KEY = "liked:" + POST;

  async function fetchLikes() {
    try {
      const res = await fetch(`${API}/likes?post=${encodeURIComponent(POST)}`);
      const data = await res.json();
      document.getElementById("likes-count").textContent = data.likes;
    } catch {
      document.getElementById("likes-count").textContent = "n/a";
    }
  }

  async function addLike() {
    if (localStorage.getItem(STORAGE_KEY)) {
      document.getElementById("likes-msg").textContent = "You already liked this post.";
      return;
    }
    try {
      const res = await fetch(`${API}/likes?post=${encodeURIComponent(POST)}`, {
        method: "POST"
      });
      const data = await res.json();
      document.getElementById("likes-count").textContent = data.likes;
      localStorage.setItem(STORAGE_KEY, "1");
      document.getElementById("likes-btn").style.opacity = "0.5";
      document.getElementById("likes-msg").textContent = "Thank you :) !";
    } catch {
      document.getElementById("likes-msg").textContent = "ERROR: Cant connect to the Likes-API. Please try again later.";
    }
  }


  if (localStorage.getItem(STORAGE_KEY)) {
    document.addEventListener("DOMContentLoaded", () => {
      document.getElementById("likes-btn").style.opacity = "0.5";
      document.getElementById("likes-msg").textContent = "You already liked this post.";
    });
  }

  fetchLikes();
</script>

As you can see, its not really difficult to implement (Okay, maybe for me). Below is the backend code that runs in the container:

from fastapi import FastAPI, HTTPException, Request
from fastapi.middleware.cors import CORSMiddleware
import sqlite3
import os

app = FastAPI()

# CORS so your blog frontend can talk to the API
app.add_middleware(
    CORSMiddleware,
    allow_origins=["https://{yourwebsite}.com"],
    allow_methods=["GET", "POST"],
    allow_headers=["*"],
)

DB_PATH = "likes.db"

def get_db():
    conn = sqlite3.connect(DB_PATH)
    conn.row_factory = sqlite3.Row
    return conn

def init_db():
    conn = get_db()
    conn.execute("""
        CREATE TABLE IF NOT EXISTS likes (
            post_path TEXT PRIMARY KEY,
            count INTEGER DEFAULT 0
        )
    """)
    conn.commit()
    conn.close()

init_db()

@app.get("/likes")
def get_likes(post: str):
    conn = get_db()
    row = conn.execute(
        "SELECT count FROM likes WHERE post_path = ?", (post,)
    ).fetchone()
    conn.close()
    return {"post": post, "likes": row["count"] if row else 0}

@app.post("/likes")
def add_like(post: str, request: Request):
    conn = get_db()
    conn.execute("""
        INSERT INTO likes (post_path, count) VALUES (?, 1)
        ON CONFLICT(post_path) DO UPDATE SET count = count + 1
    """, (post,))
    conn.commit()
    row = conn.execute(
        "SELECT count FROM likes WHERE post_path = ?", (post,)
    ).fetchone()
    conn.close()
    return {"post": post, "likes": row["count"]}

And then i only had to add a partial to layouts/partials/comments.html and thats it.

{{ partial "likes.html" . }}

I Also made a Post If anyone is curious, although I kind of did copy paste some things back and fort from that post to this page and vice versa. Its still mostly a WIP and i dont know if i will keep the system as its now kind of a taped together solution that is not the best but does its job. Also there are still some bugs. I dont know if this system will be implemented forever or if i will pull the plug and find another solution but i will let that know.

The Comments-API
#

So i managed to deploy Remark42 as the comments system. It requires users to sign in and then verify their email. This makes it so i can better identify people and or respond to them and can prevent random bots from using the unauthorised comment function as thats turned off at the moment.

Newsletter
#

So i also plan on deploying a newsletter system. It will be fully opt-in and its optional, but i will add it so people who are interested can sign up to get notified when i post a new article on the blog or any (major) update. It will not be used to access paywalled content and all content wil stay and be free forever. Also i wont shove advertisements down your face or sell your email to a databroker or advertiser. It makes use of the privacy focussed selfhostable listmonk but its still in development and is not yet ready for the public. This post will be updated when it actually is available and working and fully tested.

CDN (Content Delivery Network)
#

So i also deployed a CDN which will store all the digital assets like videos, audio and pictures and maybe some code and or css/js if i plan to go that way. The CDN as the name implies delivers this content via the network (Okay kinda bad explanation but its clear i hope). Other companies and blogs also use a CDN. I do this also because then i can keep the webserver for the blog only for serving the static HTML templates and import any external assets or css/js via the CDN. So purely for size optimisation and i can manage my media assets more easily. I am in the works of transfering over eveyrthing from the local storage (thus directly uploaded to the blog’s webserver container) to the CDN and edit the markup posts to link to the CDN instead of the local path.

Its still a bit slow and i have not optimised it fully yet, but i will definitely keep the CDN and load all assets from there in future posts as its not only convenient but also saves space on the physical webserver container.

End
#

Thats it for now. Possible future updates might be made to this or any other page, but then for that you can see the edited/modified on date.

Author
309Electronics