4mil Chunk generation slowdown.
#1
So i just downloaded the latest build,
And set it up to generate 4mil chunks.
Some chunks where already generated as i tried it to high at first :p.
End result was 17,8gb world file.

What i did notice after almost 22 hours.
That it slowed down about 50% after a while.

Server specs:
Windows 7 Home Premium x64
16GB Ram
Quad core AMD Cpu
Only seemed to use about 33%

Server never used more then 1,6gb of ram btw.
I uploaded the log, so you can see.

Also if anybody wants the pre-generated world for what ever reason just ask and i will upload it :p.


Attached Files
.zip   LOG_1443990276.zip (Size: 945.17 KB / Downloads: 155)
Reply
Thanks given by:
#2
I'd venture a guess that the slowdown at the end was (among others) the result of generator's caches becoming too short for the size of the world you were generating. That could be worked around if ChunkWorx was re-coded to use Hilbert ordering instead of even-odd linear passes. Up until now, no-one really had the need for that Smile
Reply
Thanks given by:
#3
He didn't use ChunkWorx. It was preparing the spawn area Wink
Reply
Thanks given by:
#4
The spawn area preparation uses the same even-odd linear passes, so the point stands, only the change would need to be done in a different place.
Reply
Thanks given by:
#5
Could any good soul process the log file to give us a graph of the generator's chunk speed on the Y axis and time on the X axis?
Reply
Thanks given by:
#6
Oh right, just in case somebody thinks its the hdd, running on a 64gb ssd.

Also why does it often show Chunk generator performance: but suddenly just stops doing that ?
Reply
Thanks given by:
#7
(10-06-2015, 05:02 AM)xoft Wrote: Could any good soul process the log file to give us a graph of the generator's chunk speed on the Y axis and time on the X axis?

Could you give me a small example how I can use those graphs in your PerfPages or PluginMemory plugin? Is something like this possible:
-- All other html stuff.
Out:write([[
var Data = []] .. All values here .. [[];
]]
Reply
Thanks given by:
#8
Just output it in a simple tab-crlf-delimited file (<xvalue><tab><yvalue><cr><lf>), give it a .xls extension and open it in Excel or OpenOffice. Much easier Smile
Reply
Thanks given by:
#9
Done:
local File = assert(io.open("LOG_1443990276.txt"))
local ExcelFile = assert(io.open("out.xls", "w"))
ExcelFile:write("TimeStamp\tSpeed\n")

for line in File:lines() do
	local Speed = line:match("Chunk generator performance: (%d+%.%d+)")
	if (Speed) then
		local Time = line:match("%s%s%[.-%]"):sub(4, -2)
		
		ExcelFile:write(Time, "\t", Speed,"\n")
	end
end

File:close()
ExcelFile:close()
print("done")

Though I don't think excel likes our timestamp format.

(Graph isn't included in the excel file)


Attached Files
.xls   out.xls (Size: 491.07 KB / Downloads: 156)
Reply
Thanks given by:
#10
And here's the graph:
[Image: generating_4m_chunks.png]
Reply
Thanks given by:




Users browsing this thread: 2 Guest(s)