AnvilStats

[xoft]

AnvilStats has reached the first release-able milestone. It can calculate per-biome blocktype statistics for a level. Thus, it can be used to measure the amount of ores in the vanilla world and adjust the generator parameters.

Usage:
AnvilStats <method> [<path>]
<method> is a number, currently only 0 is supported, for blocktype statistics.
<path> is a path to MCA files, if not specified, current directory is used.

The process creates N + 1 threads (where N is the number of available cores) and processes all MCA files. In the end, three statistics files are written:
- Biomes.xls - amount of biomes in the level
- BlockTypes.xls - amount of blocktypes in the level
- BiomeBlockTypes - amount of blocktypes in each individual biome

I'm planning to add more statistics (mobs, spawnertypes, trees etc.) and even new methods (draw biome map, draw height map, ...)

Currently, you can only build the project from sources, there is no binary release. Also, the build has been tested on Windows only, it probably won't work on Linux (porting volunteers are welcome

EDIT: Added example output files as attachments to this post.
EDIT2: Updated Biomes.xls with fixed format string.

---

Now I just need a large-enough world to run tests on. Unfortunately MLG ( http://www.minecraftforum.net/topic/1877...generator/ ) runs only single-threaded, so the generation is painfully slow. I might write my own MLG knockoff faster than it generates a decent world

[xoft]

In rev 899 AnvilStats already calculates the statistics for mob spawners. I think I'm done with statistics now, I want to try to make it draw a biome map

[xoft]

AnvilStats in rev 908 can draw a nice map of the biomes. It's limited to making tiles, one region equals one map tile. It's not as good as Amidst, but it's good enough for us to be able to compare biomes in various Anvil worlds.

[xoft]

So I've had MLG running for three days on my computer, so far it has generated a world about 30.000 x 50.000 blocks in size and 30 GiB worth on the disk. I think that might be good enough a sample

[FakeTruth]

wow XD that's huge

[NiLSPACE]

so with this we can finaly see how manny ores there are etc?

[xoft]

Yes.

It's still generating, says "76 % done, 1 day 2 hours remaining". I'm gonna let it finish, then I'll run the stats. Anyone want a 50k x 50k world? Expected size ~53 GiB

I'll attach the stats here once I have them. Maybe even the biome map, if it's not too large.

[xoft]

The generator finished, finally, last night. Now I'm trying to run AnvilStats on the generated world (52 GiB), and it's taking forever. And worst thing is, the CPU is not doing much, there's another bottleneck - the cFastNBT is not fast enough. It's still allocating all its cNBTTag-s from the heap and that uses some locking, which means the CPU is switching between threads like crazy - I've got 20k threadswitches in each of the 5 threads. I guess I'll have to develop a cHyperFastNBT that will allocate enough memory for all the NBT Tags at its start (by a guess) and then use that memory piece by piece, rather than calling "new cFastNBTTag".

---

And now that I think of it, I think the statistics' counters will overflow with such a huge world. I'll need to make the counters 64-bit.

[xoft]

So it takes my computer 1.5 hr to process the 50 GiB into the few kilobytes of statistics. Right now I have the wrong statistics (overflowed int math), I'll run it again with fixed datatypes..

[xoft]

And here are the statistics for the huge level. This time the counters didn't overflow, so the block counts should be accurate. That means the statistics have processed 9.765.625 chunks, 639.590.465.536 blocks. That is almost 600 GiB of data just for the block types, if uncompressed. The statistics pass took 70 minutes; slightly better than before because I tweaked cFastNBT preallocation size.

What surprised me quite a lot is that the river biome is the second "most popular" biome, after ocean. Although the overall differences are minimal, it might have been just a coincidence. Still, I've always considered river biome to be of minimum area.