4mil Chunk generation slowdown.

[ThuGie]

So i just downloaded the latest build,
And set it up to generate 4mil chunks.
Some chunks where already generated as i tried it to high at first :p.
End result was 17,8gb world file.

What i did notice after almost 22 hours.
That it slowed down about 50% after a while.

Server specs:
Windows 7 Home Premium x64
16GB Ram
Quad core AMD Cpu
Only seemed to use about 33%

Server never used more then 1,6gb of ram btw.
I uploaded the log, so you can see.

Also if anybody wants the pre-generated world for what ever reason just ask and i will upload it :p.

[xoft]

I'd venture a guess that the slowdown at the end was (among others) the result of generator's caches becoming too short for the size of the world you were generating. That could be worked around if ChunkWorx was re-coded to use Hilbert ordering instead of even-odd linear passes. Up until now, no-one really had the need for that

[NiLSPACE]

He didn't use ChunkWorx. It was preparing the spawn area

[xoft]

The spawn area preparation uses the same even-odd linear passes, so the point stands, only the change would need to be done in a different place.

[xoft]

Could any good soul process the log file to give us a graph of the generator's chunk speed on the Y axis and time on the X axis?

[ThuGie]

Oh right, just in case somebody thinks its the hdd, running on a 64gb ssd.

Also why does it often show Chunk generator performance: but suddenly just stops doing that ?

[NiLSPACE]

Could any good soul process the log file to give us a graph of the generator's chunk speed on the Y axis and time on the X axis?

Could you give me a small example how I can use those graphs in your PerfPages or PluginMemory plugin? Is something like this possible:

-- All other html stuff.
Out:write([[
var Data = []] .. All values here .. [[];
]]

[xoft]

Just output it in a simple tab-crlf-delimited file (<xvalue><tab><yvalue><cr><lf>), give it a .xls extension and open it in Excel or OpenOffice. Much easier

[NiLSPACE]

Done:

local File = assert(io.open("LOG_1443990276.txt"))
local ExcelFile = assert(io.open("out.xls", "w"))
ExcelFile:write("TimeStamp\tSpeed\n")

for line in File:lines() do
	local Speed = line:match("Chunk generator performance: (%d+%.%d+)")
	if (Speed) then
		local Time = line:match("%s%s%[.-%]"):sub(4, -2)
		
		ExcelFile:write(Time, "\t", Speed,"\n")
	end
end

File:close()
ExcelFile:close()
print("done")

Though I don't think excel likes our timestamp format.

(Graph isn't included in the excel file)

[xoft]

And here's the graph: