rev 1293

[xoft]

Done, rev 1302, when it reaches 1 GiB memory usage, stops everything, dumps used memory and then aborts itself. Please be patient, the dump really takes ages; I added a small indefinite progressbar to show that the server process didn't freeze

So try that one out and if it really stops (fingers crossed), send me the memdump.xml file.

Off topic, I don't think it's necessary to quote previous posts, we're humans, we live in contexts It just makes the forum look huge and difficult-to-comprehend

[bearbin]

Isn't 1GiB mem limit bad for large servers? Today's bukkit servers eat up more than 4 gigs for a mid-to large one.

[xoft]

This limit is only in the Debug version, so it doesn't influence production servers at all, it's only for when developers are hunting down those memory leaks.
With MCServer currently being compiled as 32-bit (on Windows), 2 GiB is the max memory available anyway (well, 3 GiB in special cases, but noone cares to set that up)

[keyboard]

mmm it seems that I can't seem to trigger the code that detects when it goes higher than 1gb. I'm beginning to think that the issue is in the C++ libraries. I'll see if I can track those too.

EDIT:
mmm, i was able to capture a memdump by putting a try catch for bad_alloc and then dumping the memory. It made almost 1gb of xml data. Also, I made leakfinder to track the internal CRT operation to see if those are the culprit.

EDIT2:
grrr, there is no info of the function names, i suspect that it didn't write the function name because it ran out of memory

[xoft]

The problem probably is that for each allocated block another block is allocated for the stack trace, and the useful data / housekeeping data ratio is very low. So when the application allocates 300 MiB of useful data in small blocks, the housekeeping data already exhausts the available memory.

Put a breakpoint in the try catch block for bad_alloc, and when it gets hit, have a look at LeakFinder.cpp's global variable g_CurrentMemUsage. I expect it to be in the order of tens of megabytes to hundreds of megabytes. Put that number minus a few (tens of) megabytes to the condition on line 890. Feel free to commit that number into the repository, since it will make more sense than my artificial one.

[keyboard]

I had to change the value to 65970823 bytes, and I was able to trigger the code. Looking at the total, it appears to be RtlInitializeExceptionChain function.

Code:

129629676     RtlInitializeExceptionChain
64814838    BaseThreadInitThunk
56007424    cChunkMap::cChunkLayer::GetChunk
55661344    cChunkMap::GetChunk
41665595    cIsThread::thrExecute
41575622    cClientHandle::DataReceived
41575622    cProtocolRecognizer::DataReceived
41575622    cSocketThreads::cSocketThread::ReadFromSockets
41575426    cSocketThreads::cSocketThread::Execute
41541855    cProtocol125::DataReceived
41541855    cProtocol132::DataReceived
41541855    cProtocol132::ParsePacket
41541627    cProtocol132::ParseClientStatuses
41541627    cClientHandle::HandleLogin
41541482    cServer::AuthenticateUser
41541482    cClientHandle::Authenticate
41541482    cAuthenticator::Authenticate
41541482    cRoot::AuthenticateUser
41177548    cClientHandle::StreamChunks
41177520    cClientHandle::StreamChunk
41166824    cChunkMap::AddChunkClient
41166824    cWorld::AddChunkClient
23223515    cRoot::Start

Attached are the totals, and .dot file.

Here is the link for the full memory dump:
http://www.mediafire.com/download.php?3u6tcyfuvw9y362

[xoft]

That is weird, only ~64 MiB allocated and you're already running out of memory? That's some housekeeping overhead
I don't think it's any of the top functions in the totals, because those are just functions that are a part of each thread's start routine. So every thread contains these at the bottom of its callstack. I'll need to have a look at the graph to see what's going on. Hopefully we'll get some answers this time.

Just out of curiosity, how long did it take to produce the full memory dump?

[keyboard]

it took 1 hour to dump hehe. I took a quick nap while waiting lol...

Yeah I was surprised that it is too low. I was expecting something close 1 gb at least...

[xoft]

The graph has nothing out of the ordinary either. I'll have to shift my focus. Obviously there are not big blocks at play here, but rather lots and lots of small memory blocks. I'll adjust the MemDumpAnalysis tool to report the number of allocations, too.

I'll also try tweaking the LeakFinder again, I'll need you to retest something for me in, lets say, 10 minutes.

[keyboard]

Sure, let me know when I can start testing