Distributed MCServer

[xoft]

I've had another of those "lightbulb" moments, so I'm gonna offload it here for later inspection and refinement.

People keep asking us whether MCServer can run on multiple machines in parallel. Theoretically it is not possible due to technical reasons. *BUT*! Let's ask, what's the most visible most expensive server task? I doubt anyone would say anything else than "world chunk generation". The good news is, our architecture makes it not only possible, but even practical, to run chunk generator on multiple machines, with one MCServer using the chunks, then.

(The mischievous news is that neither Vanilla nor Bukkit could do that easily, because of their chunk generation architecture. We'd be the first!)

The idea is, we could make the ChunkGenerator code relatively independent of the other MCServer code, then split it to another project, the ChunkServer. That ChunkServer would receive requests for chunks from MCServer, generate those chunks and send them over to MCServer. There could be several such ChunkServers serving one single MCServer instance. Heck, we could even make it run on the same machine, thus gaining multi-threaded world generation.

[NiLSPACE]

It would be pretty interesting to have this. You could let a powerfull PC generate a world and in a manner of speaking use an Android smartphone to handle the players.

[keyboard]

oh, that sounds a great doable idea!

[tonibm19]

Good idea!
Is there anyway to do this on bukkit/spigot? It would be very usefull for me

[xoft]

I don't think it's possible at all with those, at least if they're based even a little bit on the vanilla generator.

The vanilla generator generates chunks incomplete, and then finishes them when their neighbors are generated. This is especially visible with trees and abandoned mineshafts, when you look at the edge of a world with Minutor or any other mapping software, you can see that there are no trees at the edge and no mineshafts. Therefore, in vanilla there's no easy way to request "one complete chunk". It could be hacked but then the performance would suffer (you'd need to generate 3x3 neighbors for the middle chunk to be complete -> 1/9 performance.

On the other hand, MCServer's generator prides itself on giving complete chunks, so it's rather easy to distribute generating and then send those chunks over the network. You will lose some performance, too, because a lot of neighboring data can be (and is) cached between generating neighboring chunks, but then, you get twice the cache size, too, so it may offset the penalty somewhat.

The question is, is this feature so much needed that someone can *make me* implement it?

[tonibm19]

The question is, is this feature so much needed that someone can *make me* implement it?

I think it would be a very good feature, but I think now you should add more important things

[NiLSPACE]

IF this would be implented, would that mean that you could join while the server is generating the word?

[xoft]

No, there is a special codepath preventing this - the world's initialization (which includes the spawn generation) occurs before the actual listening threads are allowed to run.

[xoft]

Pingback from a related topic: RasPi + CUDA
https://forum.cuberite.org/showthread.php?tid=1169