+- Cuberite Forum (https://forum.cuberite.org)
+-- Forum: Cuberite (https://forum.cuberite.org/forum-4.html)
+--- Forum: Discussion (https://forum.cuberite.org/forum-5.html)
+--- Thread: rev 1293 (/thread-826.html)
RE: rev 1293 - xoft - 03-24-2013
Done, rev 1302, when it reaches 1 GiB memory usage, stops everything, dumps used memory and then aborts itself. Please be patient, the dump really takes ages; I added a small indefinite progressbar to show that the server process didn't freeze
So try that one out and if it really stops (fingers crossed), send me the memdump.xml file.
Off topic, I don't think it's necessary to quote previous posts, we're humans, we live in contexts
It just makes the forum look huge and difficult-to-comprehend
RE: rev 1293 - bearbin - 03-24-2013
Isn't 1GiB mem limit bad for large servers? Today's bukkit servers eat up more than 4 gigs for a mid-to large one.
RE: rev 1293 - xoft - 03-24-2013
This limit is only in the Debug version, so it doesn't influence production servers at all, it's only for when developers are hunting down those memory leaks.
With MCServer currently being compiled as 32-bit (on Windows), 2 GiB is the max memory available anyway (well, 3 GiB in special cases, but noone cares to set that up)
RE: rev 1293 - keyboard - 03-24-2013
mmm it seems that I can't seem to trigger the code that detects when it goes higher than 1gb. I'm beginning to think that the issue is in the C++ libraries. I'll see if I can track those too.
EDIT:
mmm, i was able to capture a memdump by putting a try catch for bad_alloc and then dumping the memory. It made almost 1gb of xml data. Also, I made leakfinder to track the internal CRT operation to see if those are the culprit.
EDIT2:
grrr, there is no info of the function names, i suspect that it didn't write the function name because it ran out of memory
RE: rev 1293 - xoft - 03-24-2013
The problem probably is that for each allocated block another block is allocated for the stack trace, and the useful data / housekeeping data ratio is very low. So when the application allocates 300 MiB of useful data in small blocks, the housekeeping data already exhausts the available memory.
Put a breakpoint in the try catch block for bad_alloc, and when it gets hit, have a look at LeakFinder.cpp's global variable g_CurrentMemUsage. I expect it to be in the order of tens of megabytes to hundreds of megabytes. Put that number minus a few (tens of) megabytes to the condition on line 890. Feel free to commit that number into the repository, since it will make more sense than my artificial one.
RE: rev 1293 - keyboard - 03-25-2013
I had to change the value to 65970823 bytes, and I was able to trigger the code. Looking at the total, it appears to be RtlInitializeExceptionChain function.
Code:
129629676 RtlInitializeExceptionChain
64814838 BaseThreadInitThunk
56007424 cChunkMap::cChunkLayer::GetChunk
55661344 cChunkMap::GetChunk
41665595 cIsThread::thrExecute
41575622 cClientHandle::DataReceived
41575622 cProtocolRecognizer::DataReceived
41575622 cSocketThreads::cSocketThread::ReadFromSockets
41575426 cSocketThreads::cSocketThread::Execute
41541855 cProtocol125::DataReceived
41541855 cProtocol132::DataReceived
41541855 cProtocol132::ParsePacket
41541627 cProtocol132::ParseClientStatuses
41541627 cClientHandle::HandleLogin
41541482 cServer::AuthenticateUser
41541482 cClientHandle::Authenticate
41541482 cAuthenticator::Authenticate
41541482 cRoot::AuthenticateUser
41177548 cClientHandle::StreamChunks
41177520 cClientHandle::StreamChunk
41166824 cChunkMap::AddChunkClient
41166824 cWorld::AddChunkClient
23223515 cRoot::StartAttached are the totals, and .dot file.
Here is the link for the full memory dump:
http://www.mediafire.com/download.php?3u6tcyfuvw9y362
RE: rev 1293 - xoft - 03-25-2013
That is weird, only ~64 MiB allocated and you're already running out of memory? That's some housekeeping overhead
I don't think it's any of the top functions in the totals, because those are just functions that are a part of each thread's start routine. So every thread contains these at the bottom of its callstack. I'll need to have a look at the graph to see what's going on. Hopefully we'll get some answers this time.
Just out of curiosity, how long did it take to produce the full memory dump?
RE: rev 1293 - keyboard - 03-25-2013
it took 1 hour to dump hehe. I took a quick nap while waiting lol...
Yeah I was surprised that it is too low. I was expecting something close 1 gb at least...
RE: rev 1293 - xoft - 03-25-2013
The graph has nothing out of the ordinary either. I'll have to shift my focus. Obviously there are not big blocks at play here, but rather lots and lots of small memory blocks. I'll adjust the MemDumpAnalysis tool to report the number of allocations, too.
I'll also try tweaking the LeakFinder again, I'll need you to retest something for me in, lets say, 10 minutes.
RE: rev 1293 - keyboard - 03-25-2013
Sure, let me know when I can start testing