rev 1293

[keyboard]

The lava and water simulators use the very same engine (if they are set to the same value, Floody in this case). Disabling one may have alleviated the issue because there are no longer that many blocks to simulate.

I'll try your world in a moment, I've just found a bug in the block update code that I want to fix first (water didn't dry up properly because of it)

Oops, I also forgot to say, that if I just turn off the water simulator and leave only the lava simulator, I still get the error...

[xoft]

Unfortunately I don't see any such increase in memory usage. Did you try to access the m_TotalBlocks? I have no other idea on how to debug this issue

[keyboard]

Yeah, I tried the following scenario:

-Water simulator off
-Lava simulator on
-Switched m_LavaSimulator and the m_pDefaultWorld to Public membes
-Applied a try catch block on the new operator in the cChunkMap::cChunkLayer::GetChunk() function to prevent an unhandled exception, and see what happened.
-Put a breakpoint in the catch block to see what's going on at the time of the exception

Result:
-The bad alloc exception happened
- For Lava:
m_TickDelay: 30
m_AddSlotNum: 22
m_SimSlotNum: 23
m_TotalBlocks: 2473
-Created a crash dump that is 2gb in size (can't break that into 1mb slices and upload it to the forum hehe )


I can try to see where is the most memory used to see if it help us track down the issue.

[xoft]

2k5 blocks is definitely not enough to cause the issue. I'm regularly working with 200k blocks queued in the simulator.

The crash dump usually compresses really well, try 7zipping it up and perhaps upload it to mediafire or something similar, if it's not too large. Also pack your PDBs and EXE, so that I can have a look at the dump.

[keyboard]

Link to download crash dump, with exe and PDB. If it doesn't work, I can create another crash dump.
http://www.mediafire.com/download.php?c26373n9g1o75il

[xoft]

I have to say, I'm stumped. I have no idea where the memory goes; the crashdump doesn't say anything out of the ordinary.

The only thing that I can think of doing right now is overloading the memory allocation and when memory usage hits 1.8 GiB, dump all allocated memory together with the stack info (already possible). But such a dump would take ages - a regular dump of 150 MiB takes about 5 - 10 minutes.

[keyboard]

ohh, are you able to reproduce the error with my exe?

[xoft]

No, the exe cannot run, says "not a valid Win32 application". I'm running on WinXP, so if you compiled with VS2012, I'm not able to run such a program.

---

I'm desperate, so I'm writing a memdump.xml analysis tool to make some sort of a summary for the allocations; then we can try the allocation hooking and dumping.

Maybe you could write the allocation hook in the meantime? The LeakFinder.cpp source has all that you need in the MyAllocHook function (line ~759), just add a global variable counting the number of bytes allocated and when that counter reaches 1 GiB, do the same thing that cServer does when you give the command "dumpmem" in the console. That will produce a "memdump.xml" file in the exe folder listing all memory allocations with their stack traces (and takes ages to generate )

[xoft]

Well aren't I good! I think I've written a really useful tool for visualizing the memory usage. It reads a memdump.xml file produced by the "dumpmem" console command, and outputs a list of functions and total memory they and their children allocated, as well as a DOT file for drawing a graph (using GraphViz) representing the hierarchy of memory allocation.

Have a look here: http://mc-server.xoft.cz/img/memdump.svg (you need a decent browser, it's a huge picture in SVG format, 1 MiB in size; scroll to the center to find the actual image data)
And the list (example extract of a 54 KiB text file):

Code:

894936    tolua_AllToLua_open
1168855    cPluginManager::AddPlugin
1170151    cPluginManager::LoadPlugin
1170583    cPluginManager::ReloadPluginsNow
1175920    cPlugin_NewLua::Initialize
7846804    cRoot::LoadWorlds
9542448    cChunkMap::cChunkLayer::GetChunk
9558944    cWorld::InitializeSpawn
9558944    cChunkMap::TouchChunk
9558944    cRoot::StartWorlds
9558944    cChunkMap::GetChunk

Now I only need the memory allocation data. And I think I'll add some filtering, having CryptoPP and Lua in that graph is pretty useless.

---

I'll now write the code to dump memory when memory usage reaches 1 GiB.

[keyboard]

No, the exe cannot run, says "not a valid Win32 application". I'm running on WinXP, so if you compiled with VS2012, I'm not able to run such a program.

Yeah, VS2012 uses new API that was introduced in windows vista or later.

Well aren't I good! Smile I think I've written a really useful tool for visualizing the memory usage. It reads a memdump.xml file produced by the "dumpmem" console command, and outputs a list of functions and total memory they and their children allocated, as well as a DOT file for drawing a graph (using GraphViz) representing the hierarchy of memory allocation.

Have a look here: http://mc-server.xoft.cz/img/memdump.svg (you need a decent browser, it's a huge picture in SVG format, 1 MiB in size; scroll to the center to find the actual image data)
And the list (example extract of a 54 KiB text file):

Code:

894936    tolua_AllToLua_open
1168855    cPluginManager::AddPlugin
1170151    cPluginManager::LoadPlugin
1170583    cPluginManager::ReloadPluginsNow
1175920    cPlugin_NewLua::Initialize
7846804    cRoot::LoadWorlds
9542448    cChunkMap::cChunkLayer::GetChunk
9558944    cWorld::InitializeSpawn
9558944    cChunkMap::TouchChunk
9558944    cRoot::StartWorlds
9558944    cChunkMap::GetChunk

Now I only need the memory allocation data. And I think I'll add some filtering, having CryptoPP and Lua in that graph is pretty useless.

Oh, that's cool! it looks good to analyze memory usage.

I'll now write the code to dump memory when memory usage reaches 1 GiB.

oh ok, let me know when I can run some tests