rev 1293

[xoft]

Done, rev 1302, when it reaches 1 GiB memory usage, stops everything, dumps used memory and then aborts itself. Please be patient, the dump really takes ages; I added a small indefinite progressbar to show that the server process didn't freeze

So try that one out and if it really stops (fingers crossed), send me the memdump.xml file.

Off topic, I don't think it's necessary to quote previous posts, we're humans, we live in contexts It just makes the forum look huge and difficult-to-comprehend

[bearbin]

Isn't 1GiB mem limit bad for large servers? Today's bukkit servers eat up more than 4 gigs for a mid-to large one.

[xoft]

This limit is only in the Debug version, so it doesn't influence production servers at all, it's only for when developers are hunting down those memory leaks.
With MCServer currently being compiled as 32-bit (on Windows), 2 GiB is the max memory available anyway (well, 3 GiB in special cases, but noone cares to set that up)

[keyboard]

mmm it seems that I can't seem to trigger the code that detects when it goes higher than 1gb. I'm beginning to think that the issue is in the C++ libraries. I'll see if I can track those too.

EDIT:
mmm, i was able to capture a memdump by putting a try catch for bad_alloc and then dumping the memory. It made almost 1gb of xml data. Also, I made leakfinder to track the internal CRT operation to see if those are the culprit.

EDIT2:
grrr, there is no info of the function names, i suspect that it didn't write the function name because it ran out of memory

[xoft]

The problem probably is that for each allocated block another block is allocated for the stack trace, and the useful data / housekeeping data ratio is very low. So when the application allocates 300 MiB of useful data in small blocks, the housekeeping data already exhausts the available memory.

Put a breakpoint in the try catch block for bad_alloc, and when it gets hit, have a look at LeakFinder.cpp's global variable g_CurrentMemUsage. I expect it to be in the order of tens of megabytes to hundreds of megabytes. Put that number minus a few (tens of) megabytes to the condition on line 890. Feel free to commit that number into the repository, since it will make more sense than my artificial one.

[keyboard]

I had to change the value to 65970823 bytes, and I was able to trigger the code. Looking at the total, it appears to be RtlInitializeExceptionChain function.

Code:

129629676 RtlInitializeExceptionChain 64814838 BaseThreadInitThunk 56007424 cChunkMap::cChunkLayer::GetChunk 55661344 cChunkMap::GetChunk 41665595 cIsThread::thrExecute 41575622 cClientHandle::DataReceived 41575622 cProtocolRecognizer::DataReceived 41575622 cSocketThreads::cSocketThread::ReadFromSockets 41575426 cSocketThreads::cSocketThread::Execute 41541855 cProtocol125::DataReceived 41541855 cProtocol132::DataReceived 41541855 cProtocol132::ParsePacket 41541627 cProtocol132::ParseClientStatuses 41541627 cClientHandle::HandleLogin 41541482 cServer::AuthenticateUser 41541482 cClientHandle::Authenticate 41541482 cAuthenticator::Authenticate 41541482 cRoot::AuthenticateUser 41177548 cClientHandle::StreamChunks 41177520 cClientHandle::StreamChunk 41166824 cChunkMap::AddChunkClient 41166824 cWorld::AddChunkClient 23223515 cRoot::Start

Attached are the totals, and .dot file.

Here is the link for the full memory dump:
http://www.mediafire.com/download.php?3u6tcyfuvw9y362

[xoft]

That is weird, only ~64 MiB allocated and you're already running out of memory? That's some housekeeping overhead
I don't think it's any of the top functions in the totals, because those are just functions that are a part of each thread's start routine. So every thread contains these at the bottom of its callstack. I'll need to have a look at the graph to see what's going on. Hopefully we'll get some answers this time.

Just out of curiosity, how long did it take to produce the full memory dump?

[keyboard]

it took 1 hour to dump hehe. I took a quick nap while waiting lol...

Yeah I was surprised that it is too low. I was expecting something close 1 gb at least...

[xoft]

The graph has nothing out of the ordinary either. I'll have to shift my focus. Obviously there are not big blocks at play here, but rather lots and lots of small memory blocks. I'll adjust the MemDumpAnalysis tool to report the number of allocations, too.

I'll also try tweaking the LeakFinder again, I'll need you to retest something for me in, lets say, 10 minutes.

[keyboard]

Sure, let me know when I can start testing