Use One Big Server - Speculative Branches
Microsoft Azure with AMD CPUs.
- 64 cores
- 128 threads
- ~2-2.5 GHz clock
- Cores capable of 4-6 instructions per clock cycle
- 256 MB of L3 cache
16 slots of DDR4-3200 RAM per socket. The largest capacity “cost effective” DIMMs today are 64 GB. Populated cost-efficiently, this server can hold 1 TB of memory. Populated with specialized high-capacity DIMMs (which are generally slower than the smaller DIMMs), this server supports up to 8 TB of memory total.
One server today is capable of:
- Serving video files at 400 Gbps
- 1 million IOPS on a NoSQL database
- 70k IOPS in PostgreSQL
- 500k requests per second to nginx
- Compiling the linux kernel in 20 seconds
- Rendering 4k video with x264 at 75 FPS
$1,318/month
In AWS, one of the largest servers you can rent is the m6a.metal server. It offers 50 Gbps of network bandwidth, 192 vCPUs (96 physical cores), and 768 GB of memory, and costs $8.2944/hour in the US East region.
A similar server, with 128 physical cores and 512 GB of memory (as well as appropriate NICs, SSDs, and support contracts), can be purchased from the Dell website for about $40,000.
