APIFuzzing - a lua plugin for fuzzing and checking the API

[Seadragon91]

Created a branch hook_testing where I currently added fuzzing for the entity in HOOK_SPAWNED_ENTITY. I get a few crashes and a std::bad_alloc.
If someone wants to debug it. Add the plugin. enable it and start the server. It starts then directly fuzzing of the entities. A message will be printed for every entity that spawned.

I highly recommend to run the server in a vm or with an tool that limits the memory!

[Seadragon91]

I tested APIFuzzing with cuberite debug mode.
Got an assertion failure for ... cBoat:MaterialToString 

Passed 355 to cBoat:MaterialToString. The string value oak is the default, that will be returned for an unknown value:
Windows returns the expected: oak

Clang (3.9.1): returns oak and here it gets funny. Why?

This in the code:

Code:

#ifndef __clang__
    return "oak";
#endif

It looks like clang completely ignores the ifndef. Had a thought and added a else branch that returns the string magic and clang suddenly returns magic 
I replaced the code above with "return oak" and no warning appears in compiling. I guess if I could test a cuberite version compiled with clang 3.4 it will crash. Cannot install it, to old

Checked the linux version from the build server, it's compiled with clang 3.5. It also returns oak, no crash occurs. I think that clang 3.5 and above have a better detection for return values.
The best solution I think is to bump the minimum version of clang to 3.5 and change it in travis. Also then a pull request that removes all the #ifndef __clang__ in functions like cBoat:MaterialToString and the assertions

[xoft]

Clang is giving you a very mild case of "undefined behavior". You're passing a value that is not a valid eMaterial into the function, so the compiler is free to do anything, including launching a nuke towards your location

Weird thing is that this should have been caught in the API layer, I thought we had enum minmax protection in ToLua++.

[xoft]

Honestly, I don't understand how it works now. I can't find the code in ToLua++'s parser that emits the enum checks, but it is there somewhere - it works for the eDamageType enum (tolua_isEDamageType() is called from within cEntity:TakeDamage() ) but not for the cBoat::eMaterial type.

[Seadragon91]

One difference I see is that eDamageType is global, but cBoat::eMaterial is not.

[Seadragon91]

And cMonster::FamilyFromType is also not global and I get an assertion there, too

[xoft]

Okay, that's a lead.

[xoft]

Ah, I finally see it.

ToLua++ stores the enum types as fully qualified, so it knows "cBoat::eMaterial is an enum". But then it gets a function signature that has "eMaterial" param and it doesn't see that it is the "cBoat::eMaterial" enum.

[Seadragon91]

Tested something new and had an error there. This error caused strange problems: Out-of-memory and high write disk usage problems. I guess because of that incorrect fuzzing I had an memory corruption.

[Seadragon91]

Found the problem it's are the long long input numbers (100000000000). A chunk / block at this position will cause problems and on next start the server will load this chunk and out-of-memory will occur.