Bindings generation

[xoft]

I'm trying to improve LuaAPI documentation by using the information from ToLua++ when it generates the bindings. The parser knows more about each function than the runtime can query - the number and types of parameters and return values (including overloads) and whether it is static or not.

I believe I could even make it read the doxycomments and attach them to the functions, classes and enums. The problem is that that would require a change in the ToLua++ sources, which means more difficult maintenance, if a new ToLua++ version comes out. However, considering that it is working alright for us now and it hasn't been updated for years, I pretty much doubt that it would indeed need any updates at all, so this entire thing could be viable.

Another nice point that I managed to do already is that the bindings can be generated without the actual ToLua++ executable, the basic Lua interpreter is actually enough. This could simplify cross-compiling, because we could use system-provided Lua in such a case, and won't have to compile any native executable while cross-compiling. However, we shouldn't remove the ToLua++ executable completely, because it's much easier to use than "native" (read: missing) Lua on Windows.


Another thought that has crossed my mind was to actually get rid of the ToLua++ parser altogether, and instead define the API in the form of a description - something like the APIDesc plugin uses, extended with the parameter and return value information. Then a Lua script could be used to generate the bindings from that description.
On the positive side, we could make the script smarter than a C++ parser, so it could know for example about call chaining (cChatMessage:new():AddText():AddUrl():AddText()...), could implement static functions that can use either calling convention (cClass.StaticFn() or cClass:StaticFn()), it could use the cLuaState class, it could understand callbacks. At the very least, it wouldn't generate bogus return values, such as ToLua++ now does for all "const cClass &" parameters (int SomeFn(const AString & a_Text) currently generates a binding that has two return values, the int and the string).
On the negative side this would mean that each binding would need to be described in the description file, rather than adding a "// tolua_export" comment to a function.
Back to the positive side, this would mean that each new binding would be properly documented

Thoughts?

[NiLSPACE]

Getting rid of ToLua++ sounds pretty good. Too bad you have to edit a second file though. The tolua_comments were pretty easy to do.

[xoft]

Here's an example of the documentation I'm extracting so far:
https://gist.github.com/madmaxoft/a6a7d8...e983b4de9f
(This is without modifying ToLua++ at all, so there's no Desc - no doxy comments parsed)

[xoft]

I've had partial success extracting the DoxyComments, even without actually editing the ToLua++ parser at all. Thanks to Lua's being so flexible, and the parser having enough hooks, it's actually possible to inject the DoxyComments properly.

The only problem I'm facing right now is with the following C++ code format:

/** DoxyComment */
void Function(void); // tolua_export

The ToLua++ parser strips away the lines not marked for export before even starting the parser, so the DoxyComment line is lost before it could be processed.

Here's an updated example of the extracted cWorld documentation:
https://gist.github.com/madmaxoft/3e2cc1...060f54e6fa

[xoft]

Doxycomments completely solved.

I'd like to try reversing the direction now - what would it be like if we exported the API functions through their description and have a Lua script generate the bindings, rather than parse the C++ code.

Here's a PR for my progress so far: https://github.com/cuberite/cuberite/pull/3204

[xoft]

Reversing the direction is unlikely now, there's not that much benefit and it'd be a lot of work.

For now I've settled with manually updating the documentation based on DoxyComment diffing provided by the enhancements to Tolua++'s parser, and removing the need for compiled ToLua++'s parser when system Lua is available.