For some time I was thinking about a new server setup. The current setup was updated about four years ago. It still runs stable but the disks are aging. I had to replace one or two so far. Also the power consumption of the setup os pretty high. Driving 24x 6TB SAS disks, a Cisco SG-550 10GB switch, and a full PC board with dual-powersupplies is probably not the most efficient setup in 2023.
Requirements
What should the new setup look like, and be capable of
- 10GBit/s networking
- 40+TB live storage
- Delivering 10GBit/S data rate for video editing from the server/NAS
- Hosting 20+ containers
- Network firewalling and routing
- Backup solution to a remote site
- No Cloud service
- low noise
- low power usage
Solution
After posponing the project for more than a year, as it was working just too well right now.. I finally sat together with Andres and we where looking around for some hardware which would suite the bill.
Compute
We split up the compure and the storage, as these can be separated quite well. For the compute we have gone for a standart Intel NUC PC. It’s small, quiet and efficient. The only limmitation is the networking on the NUC. Since the NUC would do the firewalling and routing, it needed at least two hardware interfaces. The solution for this is a USB-C Gigabit Ethernet dongle.
Storage
Since we now have a dedicated hardware for the storage, we opted for a well known and designed hardware.
Synology 1832xs+ has been the favorite device.
- built-in 10Gbit/s netowrkport
- 8x 3.5″ disk bays
- 2x NVme SSD’s for caching
- Linux based
Today Synology is really picky on validated disks. Never the less we still opted for some “cheap” 20TByte drives, the Seagate Exos X20 drives.
RAID / Drive-setup
We installed 8 drives into the NAS, but now the biggest question was on how to setup the RAID volume. We wanted the “best” performance and at least a n+2 reduancency. The obvious choise would have been a RAID-6 configuration, which would have given us 6x18TB= 108TByte of usable storage. But to create such a huge raidset, it would have taken a few days to sync. Also when a failure would occure, the re-sync would have taken ages! Further the I/O load and perfomance of a RAID-6 is not superb.
Instead we opted for a RAID-10 configuration with the two SSD’s as cache disks. Yes we lost 50% of the storage space, but we gained a 4x disks speed, as we have a stripe of 4 disks. With each disk delivering about 250MByts/s we can fill the 10GBit/s netowrk port.
Results tested from my desktop PC, connected over a 10GBit/s network.
Networking
Right now I have a Cisco SG550XG-8F8T, which is working fine, but needs a lot of power and makes a lot of noise too. So I opted for a Zyxel XGS1250-12 which has 3 10GBit/s ports and a empty SFP+ slot for future use and 8 Gigabit ports.
Services
Which services do we need to migreate over from the current server? I tried to make a list. I wanted to separate the differnt domains and services as much as possible by using docker-containers.
All containers will run on the NUC and will use either local storage for things like the DB-files or the NAS for large storage items like images and videos.