Self-hosted

CyberChef

CyberChef

CyberChef is a web application with an abundance of tools, including decoding and encoding to and from various formats such as Base64, ROT13, XOR, and Hex. It is often referred to as the Cyber Swiss Army Knife, and I can see why. There are over 400 tools.

I’ve personally used CyberChef on many occasions, most commonly to decode Base64 or PEM certificates. To this day, I am still discovering new tools within CyberChef. Most recently, I’ve found a tool called Magic that helps you decode items with multiple layers of encoding.

A really cool thing about CyberChef is that it is developed and maintained by the GCHQ (Government Communications Headquarters) in the UK, which is the UK’s cyber, security, and intelligence agency.

In this post, I will show you how to use CyberChef and how to host it yourself with Docker and a Cloudflare tunnel.

Using CyberChef

Accessing CyberChef

If you search for CyberChef, there are a few versions online, but the only official version is hosted on the GCHQ GitHub, which can be accessed at https://gchq.github.io/CyberChef/

CyberChef Layout

Once you have accessed CyberChef, you will see four main panels.

The first panel is the tools, also known as operations. The next panel is the recipe. The next two panels are the input and the output.

Baking

To use CyberChef, you need to bake it. Baking consists of using operations to make a recipe.

To make a recipe, you need to drag an operation onto the recipe panel.

For example, if I wanted to decode the following Base64 string VGhpcyBpcyBCYXNlNjQgaW4gQ3liZXJDaGVm.

I would drag the From Base64 operation over to the recipe panel.

…

Read More Read More

Nginx Proxy Manager Setup

Nginx Proxy Manager Setup

Nginx Proxy Manager (NPM) is an open-source Docker image that lets you run a reverse proxy and can even handle SSL certificates for you using Let’s Encrypt. It’s great when you can’t use a Cloudflare Tunnel or an Entra App Proxy. I’ve been using Nginx Proxy Manager for a while now, and it’s been perfect for what I need.

In this post, I will show you step-by-step how to setup Nginx Proxy Manager with Docker and configure a Proxy Host, Redirection Host, 404 Host, add a custom SSL certificate, and enable SSL.

Prerequisites

  • Access to the DNS for the domains you want to use.
  • Access to create port forwards for port 80 and 443.
  • Docker host.

Initial Setup

  • Make a folder to store your configurations for Nginx Proxy Manager.
  • Make a new docker-compose.yml file.

Your docker-compose.yml file should look something like this.

services:
  app:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: unless-stopped
    ports:
      # These ports are in format <host-port>:<container-port>
      - '80:80' # Public HTTP Port
      - '443:443' # Public HTTPS Port
      - '81:81' # Admin Web Port
    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencryptCode language: YAML (yaml)

The docker compose file will create a folder named data in the location where the docker-compose.yml file is. The data folder holds the SQLite database for NPM, log files and other configuration files. A folder named letsencrypt will also be created, where your Let’s Encrypt data is stored.

  • Run the following command to start the Nginx Proxy Manager docker container docker compose up -d
  • Go to the address of your Docker host on port 81 to access the NPM admin interface.
  • Login with the default admin user.
Email: [email protected]
Password: changemeCode language: plaintext (plaintext)

You will be forced to change it on the first login.

  • Change the user details as needed.
  • Change the password.
…

Read More Read More

Entra Application Proxy

Entra Application Proxy

There are a few ways to grant external access to an internal application without doing any port forwarding. The way to do this in Microsoft’s world is through an Entra Application Proxy.

The name is a bit of a mess, as Microsoft renamed the Microsoft Entra application proxy program to Microsoft Entra private network connector. The Microsoft Entra private network connector is part of Microsoft Entra Private Access, which is part of Microsoft Global Secure Access.

Basically, Microsoft Entra Enterprise Applications can be configured with an Application Proxy, which will use the Microsoft Entra private network connector to proxy the connection.

In this post, I will show you step-by-step how to set up a Microsoft Entra private network connector, configure an internal web application to use an Entra Enterprise application proxy, and add authentication before access is granted to the web application.

Prerequisites

  • Microsoft Entra ID P1 or higher license.
  • External domain added to Microsoft 365.
  • Windows Server for the Private Network Connector.
  • Internal DNS name for the application.
  • Access to the external DNS records.
  • SSL certificate in PFX format with a password.

The Process

The process will be broken up into the following sections.

Private Network Connector

We need to set up the Entra private network connector as the Entra application proxy will proxy its connections via the private network connector.

  • Log in to the Microsoft Entra Admin Center
  • Click on Global Secure Access

If needed, click on Activate to Activate Global Secure Access for your tenant.

  • Under Global Secure Access, click on Connect > Connectors.
…

Read More Read More

Shlink with Docker and Cloudflare Tunnel

Shlink with Docker and Cloudflare Tunnel

I write a lot of PowerShell scripts for application installations, and many download the install files from a URL. Because the URL is hardcoded in the scripts, if the URL changes, I need to update all the scripts. This exact issue happened to me a few years ago, and my solution at the time was to write a Python script to update the PowerShell scripts. You can read about that solution in my blog post, Script to make Scripts.

However, if my scripts had used a URL shortener instead, I could’ve edited the short URL to resolve the issue. The problem with most URL shorteners is that you can make short URLs all day long, but you can’t edit them, and if they let you edit them, it’s a paid feature, and they also start limiting how many hits your short URL can get. I am not a fan of any of that.

Fortunately, a URL shortener named Shlink exists. Shlink is a self-hosted open-source URL shortener. With Shlink, you can do whatever you want with your short URL, including editing them.

Once I got Shlink set up, I re-wrote my PowerShell scripts to use my Shlink short URLs. Now, if an install URL changes, I can edit the Shlink short URL, and I don’t need to update my PowerShell scripts.

In this post, I will show you step-by-step how to set up Shlink with Docker and Cloudflare Tunnels.

The Shlink setup will use a Cloudflare tunnel for external access and a MariaDB database for the Shlink database. I selected MariaDB as I am more familiar with it. However, Shlink does support other databases. We will also use the Shlink web client as a UI to easily manage the short links.

Prerequisites

  • Domain for the short links.
  • DNS for the domain hosted in Cloudflare.
  • Docker host.

The Process

  • Make a folder to store your docker configurations for Shlink.

I’ll call mine shlink.

.env Setup

  • Copy the sample.env file as a new .env file.
  • You will need to define the variables in the .env file.
  • The variables are:
    • CONTAINER_NAME is the name of your Shlink stack. There will be 4 containers spawned.
      • The one with _app appended to it is the Shlink application image that runs all of Shlink.
      • The one with _db appended to it is the MariaDB image for Shlink to store all the configurations.
      • The one with _web_client append to it is the Shlink web client image that gives you a nice UI to manage your short URLs.
      • The one with _cf appended to it is the Cloudflare tunnel image for the Shlink application.
    • DB_NAME is the name of the MariaDB database that Shlink will use.
    • DB_ROOT_PWD is the MariaDB root password.
    • DB_USER is the database user that Shlink will use.
    • DB_USER_PWD is the password for the database user that Shlink will use.
    • SHORT_DOMAIN is the domain you will be using for your short URLs. (only include the FQDN (Fully Qualified Domain Name))
    • REDIRECT_DOMAIN is where invalid short links, direct visits to the short domain, or 404 pages will be sent. (this needs to start with https://)
    • APP_PORT is the port that the Shlink application will run on and the port you will use with the Cloudflare tunnel.
    • ADMIN_API is the initial API key that will be used to configure your Shlink and will be preloaded to the Shlink Web Client.
    • WEB_CLIENT_PORT is the port the Shlink web client will use.
    • CF_TUNNEL is the Base64 of your Cloudflare tunnel for Shlink.

For example, I will use the domain dxt.zip as the primary domain for Shlink. I will be redirecting all non-short URL traffic to my blog, thedxt.ca. I will use port 8788 for the Shlink application and port 8787 for the Shlink web client to manage Shlink.

…

Read More Read More

UniFi Network Application MongoDB Upgrade

UniFi Network Application MongoDB Upgrade

If you have been using the UniFi controller for a very long time, there’s a chance you are running an older version of MongoDB. When Ubiquiti released version 8.1 of the UniFi network application server, they finally bumped up the supported MongoDB version from 4.4 to 7.0.

The MongoDB upgrade path only supports jumping one major version at a time, and some manual steps are needed. Yes, you could take a settings-only backup and reroll the whole setup, but where’s the fun in that?

In this post, I will show you step-by-step how to upgrade the Mongo database version from 4.4 to 7.0 for the Unifi network application server running on docker.

Prerequisites

  • Backup your UniFi config. My blog post, UniFi Network Server Settings Backup and Export, covers how to do this.
  • Running UniFi network application version 8.1 or newer.
  • The service name of the MongoDB for UniFi from your docker compose file.
  • The container name of your MongoDB for UniFi.

The Process

The entire process revolves around docker exec as we will execute commands directly into the Mongo database container.

You can follow this process on other UniFi controller setups that aren’t done with docker. However, I’ve only validated the process with on a docker setup, but the command structure would be similar.

To be extra safe during this process, I will bring down my UniFi docker stack and only bring up my UniFi database container.

I will run the command docker compose down to bring the UniFi controller offline. This helps ensure the UniFi network application doesn’t try to write to the database while the database is upgrading.

  • Bring up the MonogoDB container for UniFi with the following command. Replace Mongo_Service_Name with the service name for MongoDB from your docker compose file. sudo docker compose up Mongo_Service_Name -d

If you are using my docker compose setup from my blog post, UniFi Network Server with Docker, the MongoDB service name will be unifi-net-app-db

For me, the command will be sudo docker compose up unifi-net-app-db -d

Before starting any upgrades, let’s double-check which version of MongoDB is installed and what the compatibility level is currently set to. We will do that by using the MongoDB shell with the command mongo. We will tell Mongo that we will provide some commands from which we want the output by using the argument eval followed by our commands.

The command to check which version of MongoDB we are running and what our compatibility level is set to is the following command db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )

To run this against our Mongo docker container, we will use docker exec with a shell command followed by the mongo shell commands we want to run.

  • Run the following command to check what MongoDB version is running. Replace Mongo_Container_Name with the name of the MongoDB container that you are using with UniFi. sudo docker exec Mongo_Container_Name sh -c 'mongo --eval "db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )"'

For me, the command looks like

sudo docker exec unifi-net_DB sh -c 'mongo --eval "db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )"'

Your output should be something like this

MongoDB server version: 4.4.29
{ "featureCompatibilityVersion" : { "version" : "4.4" }, "ok" : 1 }

The output above confirms that we are running MongoDB 4.4, and our feature compatibility is set to the same version.

If you get an error that mongo is not found, this is due to running a newer version of MongoDB that no longer supports the mongo command as that command has been deprecated in newer versions.

The replacement command for mongo is mongosh. However, the output differs from the old mongo command when gathering the compatibility levels.

…

Read More Read More

UniFi Network Server Settings Backup and Export

UniFi Network Server Settings Backup and Export

When you self-host the UniFi Controller or the UniFi Network Application server, it’s a good idea to take a backup before making major changes. Backups are always a good thing to have.

In this post, I will show you step-by-step how to take a settings only backup of the UniFi Network Application server.

The Process

  • Login to the UniFi Network Server.
  • Click on Settings.
  • Click on System.
…

Read More Read More

UniFi Network Server with Docker

UniFi Network Server with Docker

LinuxServer.io makes some great Docker images. I’ve been using their UniFi Controller image for a very long time. Sadly it is being deprecated in January 2024. Fortunately, they have a new replacement image named UniFi Network Application. There’s a bit more to the setup as the database portion has been decoupled from the image.

In this post, I’ll show you step-by-step how to set up the UniFi Network Application with Docker.

The UniFi Network Application uses MongoDB for its database we will need to set that up. MongoDB is composed of two elements, the main database and a stats database.

When you first make a MongoDB container it needs to be initialized there are a few methods to do this. I’ll be using the shell script method as I was able to make that word with all the environment variables I wanted to use.

Setup

  • Make a folder to store everything in. I’ll call mine unifi-net-app.
  • Download the docker-compose.yml, the init-mongo.sh, and the sample.env files from my GitHub https://github.com/thedxt/UniFi-Docker and place them in the folder.
  • The sample.env file shows you the variables you will need to define.
  • The variables are:
    • CONTAINER_NAME is the name of your UniFi stack. There will be two containers spawned.
      • The one with _CORE appended to it is the LinuxServer.io UniFi Network Application image.
      • The one with _DB appended to it is the official MongoDB image. It is currently pinned to version 4.4 as that’s the highest version UniFi supports.
    • MONGO_DBNAME is the name of the MongoDB database that stores all the data. The second database for stats will have _stat appended to it.
    • MONGO_USER is the database user for the UniFi database and the stats database.
    • MONGO_PASS is the password for the database user for the UniFi database and the stats database.
    • TIME_ZONE this sets the time zone for the UniFi Network Application. Here’s a list of the valid TZ identifier entries.
…

Read More Read More

Cloudflare Tunnel with Docker

Cloudflare Tunnel with Docker

There’s been a few times where I needed to setup access to an internal web application but I couldn’t put it on 443 or 80 because something else was using those ports and a reverse proxy would break one of the applications. A solution to this is Cloudflare Tunnel.

Cloudflare Tunnel used to be called Warp when it was in beta and was eventually renamed to Argo Tunnel. When Cloudflare made Argo Tunnel free they renamed it to Cloudflare Tunnel. The magic of Cloudflare Tunnel is handled by a small but powerful client that is known as cloudflared.

What makes a Cloudflare Tunnel awesome is the fact that you can use it to host an application externally without opening any ports on your firewall. It does this by creating an outbound only tunnel directly to Cloudflare.

For my situation I needed to setup access to an internal web application but I didn’t want to do another port forward to make it work. My solution was Cloudflare Tunnel with Docker.

The way I set it up is slight different than what Cloudflare’s documentation says as I wanted to use the Zero Trust dashboard and Docker but also have it in a Docker Compose file, as cloudflared seems to get updated at least once a month and I wanted it to be easy enough to recreate. Here’s how I did it and how everything works.

…

Read More Read More

Setting Up Cloudflare Access

Setting Up Cloudflare Access

I’ve been a fan of Cloudflare for a while now. I love how fast they can propagate DNS changes and I typically like to use them as a DNS resolver. An issue that I’ve ran into many times, is how to protect something with MFA (Multi-Factor Authentication) that doesn’t have any support for MFA.

This problem is common with legacy web applications and very common with SCADA (Supervisory Control And Data Acquisition) web applications. The issue I was trying to solve was how to put MFA in front of a SCADA web application.

I decided to use Cloudflare Access. Cloudflare Access goes by a few names some of them are Cloudflare Zero Trust Network Access (ZTNA), Cloudflare Access, and Cloudflare Zero Trust Access. For simplicity I’m going to refer to it as Cloudflare Access.

Cloudflare Access is really nice because you can put it in front of any web application and it will require the user to authenticate before they can even reach the website.

If you have a lot of applications like this you can even set it up so the users can login to a portal to see all of the applications that are available to them. You can do all of this without the user having to install anything. What’s even better is that it’s free for 50 users.

Here’s how I setup a SCADA web app with Cloudflare Access.

…

Read More Read More