All projects
SkySim logo
Production

SkySim 2022

A from-scratch Minecraft recreation of Hypixel Skyblock, run as a production game since 2022. Grew to 80,000+ unique players with 300 peak concurrent, held at 99.95% uptime. Powered by many custom systems for its diverse gamemode, with load-based server deployments and a monetized rank system.

Role
Head Administrator & System AdministratorAll production & testing environments
Timeline
2022 — 2025
Scale
80k unique players @ 300 concurrent
Team
6 current · 5 previous
Visit site
01 · Context

Context & problem

The idea behind the server was worth taking seriously, but it did not have hosting that could carry it. I saw the potential, offered to provide hosting free of charge, and began helping to run the server.

The audience was straightforward: MMORPG players who enjoyed the server and wanted a stable place to play. Growth was gradual rather than engineered. Over time, the community channel expanded to more than 5,000 members.

Money was the main constraint. High-concurrency Minecraft is hardware-hungry, so instead of over-provisioning, I bought dedicated instances and kept deployments efficient and right-sized for the load. The hosting model had to be cheap enough to sustain and strong enough to keep the server online.

02 · Role

Role & scope

As Head Administrator and System Administrator, I owned the infrastructure that kept the server online: the hardware stack, proxy and filtering layers, deployments, storage, and monitoring across both production and testing environments. I built and operated that stack directly rather than handing it to a hosting provider.

Attack mitigation was a joint effort. Working with our upstream network provider, I deployed several filtering systems to stop volumetric and bot attacks before they could disconnect players. On the game side, my role was narrower: I helped with the design and ideation of custom systems but was not solely responsible for them. Payments were largely handled by Tebex, which covered the storefront, payment processing, and the plugin that connected purchases to the network.

03 · Architecture

How the system fits together

The network was split between a small number of large game instances and a fleet of short-lived ones. The public world — the hub and the main islands that carried most player traffic — ran on multiple mega instances. Instanced game modes such as dungeons ran on mini servers that were created and torn down on demand by custom orchestration software.

Proxy layer. Velocity sat in front of the network. A custom Velocity plugin watched for bot behaviour and fed blacklist data straight to a custom XDP filter layer. That layer ran on one of up to four proxy servers and dropped matching connections at the kernel level.

Traffic filtering. XDP handled high-volume packet decisions close to the network interface, while volumetric filtering was deployed with our upstream provider. The two layers were complementary: one dealt with packets, the other with player-level bot patterns.

Load-based orchestration. Custom software decided when new lobbies were needed using two signals: the player count in the current lobbies and TPS. That kept the fleet sized to real demand instead of a fixed guess.

Data layer. A custom storage engine kept cross-server player data in blob files, with versioned records so older data could be migrated automatically.

Observability. Grafana tracked connection-level signals such as drop rate and connections per second. Netdata covered the game servers themselves.

04 · Deep dives

The parts that took real expertise

Game systems & custom content

Custom content was a shared effort. I contributed to the design and ideation of new systems rather than owning their implementation. My responsibility was making sure the infrastructure underneath them could carry the load and the data they produced.

Network filtering & attack mitigation

The filtering path was the part I owned most directly. A custom Velocity plugin inspected player behaviour at the proxy and produced blacklist entries. Those entries were consumed by a custom XDP layer on the proxy host, which dropped matching connections in the kernel before they reached the game logic. The same setup worked alongside volumetric filtering deployed with our upstream provider, so attacks were absorbed at two levels instead of one. The goal was simple: an attack should cost us packets, not player connections.

Graph of packets dropped during an attack, peaking near 600 million packets
Packets dropped during a volumetric attack, peaking near 600 million.
XDP filter console log showing a bot attack being detected, IPs being blacklisted, and suspicious IP counts rising into the thousands
XDP filtering during a bot attack on 29 July 2025 — suspicious IPs climb into the thousands while the filter blacklists them in batches. Open full size

Load-based server deployments

Orchestration was driven by two signals: player count in the current lobbies and TPS. When those signals showed that the existing fleet was under pressure, the custom software brought up additional lobbies. Using TPS alongside population mattered because a server can be full of players and still be healthy — or lightly populated and already struggling. The result was a fleet that followed actual demand rather than a fixed allocation.

Payments & paid ranks

Payments were handled end to end by Tebex: storefront, payment processing, and the plugin that delivered purchases into the network. I did not have to build or operate much of it. My role was to make sure the integration fit the network and that purchases arrived correctly.

Reliability & uptime

Availability was monitored continuously: dedicated checks constantly pinged the proxy servers for availability. Updates were rolled out gradually. Lobby servers stayed online while changes were pushed across the rest of the network, which kept a working entry point for players and limited the impact of a bad deploy.

Player data & persistence

Player data lived in a custom storage engine built around blob files and optimised for cross-server access. Each record carried its own data version. Older records were migrated automatically, while records already on the current version were used as-is, which let the storage format evolve without breaking existing player data. A backup script ran continuously, keeping the last week of data by writing new backups and deleting expired ones. One week was enough to cover all issues we encountered.

05 · Problems

Hard problems & trade-offs

The hardest class of problem was exploitation. Players found duplication bugs in newly shipped systems and edge cases, while crash exploits and memory leaks were the other recurring class of failure. When a duplication exploit was reported, we put the network into maintenance mode, reproduced the issue from the information provided, and deployed a fix — in one case within an hour. Keeping the server offline during that window was the right trade-off: a short outage was better than letting the economy be duplicated further.

Crash exploits and memory leaks demanded the same discipline: reproduce, isolate, patch, and keep the network in maintenance mode until the fix was verified.

06 · Results

Results & impact

The growth was gradual: the server started without proper hosting and grew to carry 80,000 unique players, with 300 online at peak and 99.95% uptime. The community channel passed 5,000 members. For a high-concurrency Minecraft network run on dedicated hardware and a tight budget, those numbers reflect both the infrastructure choices and the operational discipline around them.

07 · Reflection

What I would do differently

The project started from an old core, and that constrained almost everything that followed. Expanding it was difficult: the codebase was laggy and monolithic, while the live server kept demanding new updates. The choice was between rewriting the core and continuing to build on top of it. Most of the time we took the middle path — we did not rewrite the core, but we re-did the important parts where the old implementation was holding the product back.

That middle path is not free. It keeps the product alive, but it leaves the old system in place longer and makes every future change carry more context. The lesson I took from SkySim is that the boundary between keeping and replacing has to be chosen deliberately.

What I would do differently is make that boundary explicit earlier: identify the load-bearing parts, replace those, and stop treating a core rewrite as a single all-or-nothing decision. The incremental path worked, but it was chosen under pressure rather than designed up front.

08 · Expertise

Expertise demonstrated