Developing an example Protection plugin

[xoft]

I think that in the current state of things, MCServer is lacking a serious tutorial on how to write plugins, from scratch. So I decided that I'll try to write one up. And just so that it isn't in vain, I'll make a real plugin and this tutorial will also serve as its documentation. Expect this to be a long-going effort, a series of posts related to the life of a plugin

Initial decision
So, I have decided to write a Protection plugin. I know some basic Lua and I have an extensive background in programming, so it shouldn't be too difficult. I know how MCServer works, and I can use google for when I need some details about Lua. Should the need arise, I can also make modifications to the Lua API exposed by MCServer, but that is not a precondition to writing plugins, it's enough to know that there is such a possibility, in the very last-resort scenario.

Workflow analysis
Now that I know that I want to do it, I need to really think about what it is that I want to do. What will my plugin exactly do, how will users interact with it, how will it work with other plugins, etc.
A Protection plugin should allow trusted users (let's call them VIPs) to define regions where only certain players may interact with the world. Interacting here means breaking blocks, placing blocks, opening chests and doors, and activating buttons. It should allow an indefinite number of such protected areas. For simplicity, the areas will be rectangular in shape (axis-aligned box). Areas may intersect each other, in such a case, users allowed in both areas will be able to interact with the intersection of the areas (think of "shared fences" here).
A feature for later inclusion will be the ability to allow interaction in an area based on user's group. For now, we will base our plugin solely on single users.
VIPs would use the plugin in the following way:
A1, Obtain a specified "wand" (configurable item, which will be used for marking the areas' corners)
A2, Click one corner of the wanted area, right-click the opposite corner of the area
A3, Use a command to make the area protected for a list of users
For administrative purposes, VIPs should be allowed to view a list of areas for a specified block, with the following workflow:
B1, Obtain a specified "wand"
B2, Click the block to be queried
B3, Use a command to list the protection areas to which the block belongs
To remove protection, VIPs will need a way to identify areas. A simple numerical ID will be used for that, this ID will be listed in the list (B3) and also will be shown when the area is created (A3). Removal workflow:
C1, Determine the area ID, using the Area list (B1-B3) or when the area is created (A3)
C2, Use a command to delete the area by ID
When multiple users are assigned to a single area, we need a way to add and remove users to the area's list:
D1, Determine the username
D2, Determine the area ID (B3 or A3)
D3, Use a command to add or remove the user to the area. If the area has no users, it will be deleted altogether (?).
It may be useful for when a user is deemed bad, to have a way of removing all protection areas for a specified user, with the following workflow:
E1, Determine the username
E2, Use a command to delete all areas with the specified username. If any area has no users, it will be deleted altogether (?).
For some special cases, it might be beneficial to allow specifying the area not by a wand, but by coords:
F1, Determine the coords for the area
F2, Use a command to make the area protected for a list of users

Now with this cleared up, we can finally come up with the commands needed for operating this plugin:
ProtWand - gives the VIP the "wand" item (A1)
ProtAdd - adds a new protected area, marked out previously by the wand. Parameters specify usernames to allow (A3)
ProtList - lists all protected areas for the previously marked block. (B3)
ProtDelID - deletes a protection area by its ID (C2)
ProtAddUser - adds a user to a protection area (D3), parameters are area ID and username
ProtRemUser - removes a user from the protected area (D3), parameters are area ID and username
ProtRemUserAll - removes a user from all protected areas (E2), parameter is the username
ProtAddCoords - adds a new protected area by coordinates. Parameters are 6 coords, then usernames to allow (F2)

(to be continued)

[xoft]

Existing work
Of course it's reasonable to check whether it has been done before, what kind of problems it has encountered, what results it has achieved and what improvements we're offering.

A quick search in the forum yields several results, most of them Spawn protect, which is a different thing. There's also a Protection plugin by Supertransformer:
https://forum.cuberite.org/showthread.php?tid=877
Upon quick inspection, that plugin does only simple protection, without any usernames, and has a hardcoded limit of 50 areas. It therefore makes sense to create the planned plugin, because the planned features represent a much higher value. It's also clear that the new plugin cannot reuse the code from the old plugin, because the feature set is quite different.

(tbc)

[tonibm19]

I need a protection plugin with the feature of adding owners to an area.
When it's made please send me the first working version.

[xoft]

Available technology
There are several ways to do what the plugin needs to do. We need to choose the proper way.
The plugin needs to store the protection areas somewhere. There are multiple available storage schemas:

• Binary flat file(s)
  
• Lua source file
  
• SQLite database
  
• XML file
  

The binary flat files may have the advantage of fast overall reads and writes, but they are quite prone to errors, one cannot change the format easily, once deployed, and most of the operations will need only partial data anyway. Also, the files are not human-readable nor is there a standardized editor that an admin could use in case of corruption / tweaking. The plugin would have to implement most of the low-level read / write logic.
Lua source files are easy to write and super-easy to read (just do "require filename.lua"). However, they do not support partial reading and updating or deleting items either will be difficult, or the plugin will need to hold all the data in the RAM. Neither is a good solution. The files are human readable. The plugin would have to implement a (simple) writer, the reader is built in to Lua.
SQLite database is fast for reading and writing, is easily updated for new format, does support partial reads and writes and handles updates and deletions very well. The files aren't human-readable, but there's plenty of 3rd party editors out there. The API provides everything that the plugin needs for reading and writing.
XML file is fast enough for reading and writing, but difficult to partial-read, partial-write, update or delete. They are human-readable and quite legible. The API supports easy reading, the plugin would need to implement writing.

From this overview, it's quite obvious that the best choice here is to use the SQLite database for storage.

As for the actual protection, the plugin could either use hooks to forbid the interaction, or it could react to the interactions and try to undo them. The hooks approach is definitely better here.

---

Performance considerations
It is wise to do a little calculation on the plugin performance beforehand and thus see the benefit / penalty ratio. Let's consider a middle-sized server, with 80 people on it. There's a build-event currently on the server, so everyone is busy building and breaking blocks. From personal experience, builders can be as active as 5 placed blocks per second. This means that the plugin will need to perform 80 * 5 = 400 "checks" per second.
It is concievable, that on such a server there would be some 400 regular visitors, with an average 6 protected areas. That would make an estimated total of 400 * 6 = 2.400 protected areas on that server. If the plugin was to check each area with each check, it would mean doing 2.400 * 400 = 960.000 "IsInArea" comparisons per second. That's already a huge number, so we need a better strategy.

Note that we don't need to check the protected areas for players that are not currently present in the server. So instead of 2.400 areas that would mean checking 80 * 6 = 480 areas, or 480 * 400 = 192.000 "IsInArea" comparisons per second. A bit better.

Another optimization comes from the fact that we already know what player we need to check, so if we had only the list of areas for that player, we'd only need to check 6 areas, or 6 * 5 = 30 "IsInArea" comparisons. Compare that to the stunning 960.000 comparisons at the beginning!

In order to be able to optimize like this, we need to make some design choices. We need to be able to quickly retrieve the list of protected areas for a specified player. The easiest way to do that is to load the player's areas when they login (or when there is a change to the areas by the VIPs) and forget them when they log out. The areas will be stored in a table, and the table will be stored in a player->areas map table:

-- globals:
PlayersToAreas = {}

-- in the hook:
local AreasForThisPlayer = PlayersToAreas[Player:GetName()];
for Area in pairs(AreasForThisPlayer) do
  -- check Area
end

[xoft]

Refactoring the thoughts
So far, we've worked under the general assumption that we want the whole world "locked" and the areas to define where players can build (define-allow). But there's another scenario, maybe even more common: Let the players interact with most of the world, except for some areas that are reserved for specific players (define-deny). The second approach means that we won't be able to use the above optimization, because we really need to check even the protection areas for not-connected players.

But we can still optimize even this second variant. It's very probable that the protection areas will be spread out throughout the world, rather than all intersecting one space. So instead of loading per-player areas, let's load per-space areas. Space, in this sense, will be a defined rectangle of blocks. So for example if a player is at XZ [160, 260], we'll load all the areas that affect the space of [100, 200] - [250, 350]. If the player moves to [140, 260], we'll load all the areas that affect the space of [50, 200] - [200, 350] - so we'll always have a space of 150x150 blocks and the player will be more than 50 blocks away from the edge of this space. The expectation is that there won't be too many protected areas in the space.
Still, we will keep a per-player map of protection areas that each connected player can "reach", as defined by the Space above. Only the map will have to change when the player moves.
Here's a diagram to illustrate:

The dashed rectangle represents the current space, 150 x 150 blocks around the player. Area1, Area2 and Area3 will be loaded, while Area4 won't.

---

Performance, revisited
With the new approach, what are the expected performance stats? Let's use the same scenario - a server with 80 connected players and 400 regulars, with average 6 areas each. Let's also suppose that the areas are at least 16x16 blocks in size and less than 50 % of them overlap.
With this data, we can calculate that there are at most 200 protected areas within each player's space (150 / 16 = 10 areas in each direction, 50 % overlap -> 2x more areas). So each interaction needs to be checked with 200 areas, i. e. 80 * 5 * 200 = 80.000 comparisons per second. For a worst-case scenario, this is a nice number. For a more real scenario, let's use a "town" with 20x20 building sites and 12 block wide streets; each building site being a single protection area. That means a maximum of 25 areas within each players space (150 / 32 = 5 areas in each direction), 80 * 5 * 25 = 10.000 comparisons per second. That is an acceptable number.
If we need to optimize even more, we can make the Space smaller, 150 blocks seems quite large anyway. Dividing the space dimensions in half means we'll be slicing the comparisons by a factor of 4. However, we must not shrink the Spaces too much, because we need to be able to account for player's reach (up to 5 blocks away from current position) and we don't want to be re-loading the spaces too often when the player moves.

[xoft]

High-level code structure
We're finally nearing the actual code-writing. Now it'd be nice to just sum up the thoughts and decide on what groups of functionality there is, split the project into modules:
- Obviously, we need hook handlers for the interface with MCServer and for doing the actual protection. We also need command handlers that would handle commands from players accordingly. A simple Handlers module will do.
- We need to remember a few things about the current "state of things" for each VIP - what commands they're running, what coords they selected etc. Let's make a module CommandState for remembering this. Also, this module will implement the most logic behind the commands
- For each player we need to keep track of the protected areas around them. This will be the PlayerAreas module.
- To isolate all the modules from the underlying storage technology, we'll have a Storage module that will provide high-level functions, such as AddArea, AddUserToArea etc. and it will do the actual DB access.

Database structure
We decided to use SQLite for the storage, which means we need to structure our data in a database-like manner. In order to do that, we need to identify the "objects" that we will be using.
Protection areas are one such object. Each area will have a list of users; that can be represented in several ways. We could store a concatenated list of users for each area, but that doesn't work well with finding and removing users. It'll be much better to have another DB table, Users, pairing usernames with area IDs. Also, for administrative review, it might be beneficial to know who created each area.
So let's recap the structure:
- ProtectionAreas: ID, Coords, CreatorUserName
- AllowedUsers: AreaID, Username

[xoft]

Base code skeleton
Finally, after almost a week of planning, I feel that have done my homework, did the research on what is needed, and prepared myself thoroughly for the actual coding. So let's start writing some code. I like to start my code as a skeleton - put together the main parts, make a high-level picture of what needs to be done. First, write the toplevel functions, they won't do anything yet, but make the code organized; also this allows for some last-minute thoughts about the architecture to surface. Then, go deeper, fill in the functions, make them call lower-level functions; for the time being, those will do nothing, return a dummy value etc.
For example, for the very start, we don't need the storage to actually do anything, we'll have the big access functions, such as AddArea() and GetAreasNearToCoords(), but they will not be interfacing with the DB, AddArea won't do anything except for maybe logging the request, and GetAreasNearToCoords() will return one dummy area. This is to simplify testing of the code that will make use of the storage. This way we'll have a working skeleton done quite fast and can delve into the details later on.

[xoft]

First code iteration
The first code has been written, it is the very skeleton, as mentioned before, plus a few things already implemented. I'm still feeling my way around Lua and MCS API, so it took lots of retries to get this actually right.
The code does nothing much - it keeps track of each user's two sets of wand coords (lclk and rclk); other than that, it is just a skeleton full of empty functions.
The important thing to notice here is the overall stucture.

The CommandHandlers.lua file will implement all the functions that are called by MCS as a response to commands. It also has the function InitializeCommandHandlers() that registers all the commands. The commands themselves are stored in a separate file, CurrentLng.lua, that is meant to be translateable in the future. THe only command currently implemented is the ProtWand command, which gives the predefined wand item. Note the use of a cConfig class for the actual wand item storage - this way we can centralise the wand item and make it configurable later on.

The CommandState.lua file implements a cCommandState Lua class. This class currently holds the two sets of coords, used for storing the rclk and lclk wand click coords. Each user is expected to have one object of this class, even if the user is not a VIP, because there are multiple permissions for the various commands, so rather than checking permissions, we'll process all users' rclk and lclk data, but only those with permissions can actually do something with the data stored here.
Of special note is the GetCommandStateForPlayer() function, it expects a cPlayer object and returns the cCommandState object for that player. If that player doesn't have a cCommandState object yet, the object is created and stored in the global map. Once created, this function will return the same object for that player.

Config.lua implements the cConfig class. Currently it only holds the specification of the Wand item, but it may hold other configuration in the future. It is expected to be extended in the future so that it loads the config from an INI file, rather than being hard-coded. The class also defines some useful access functions, notably IsWand, which checks if the given item is the correct Wand.

CurrentLng.lua, as already mentioned, stores all the strings that are to be translatable. For now, it holds the commands; it will be later extended for all the user-visible strings. Translators will make copies of this file and then translate the strings contained within; server admins will then overwrite this file with the language file of their choice. Note that the other language files must not have a ".lua" extension, otherwise MCServer will try to load them and they will clash together.

The HookHandlers file implements the various hooks needed to perform the plugin's function. Of most importance are the OnPlayerLeftClick() and OnPlayerRightClick() hooks, which will be used for blocking the unwanted users. Right now, only the wand interaction is implemented.
Also, when a player joins or leaves, their cPlayerAreas and cCommandState is created / deleted, as appropriate. Without this, the plugin will leak memory, especially with the cCommandState being allocated per each player logging in.

The PlayerAreas file implements the cPlayerAreas class, one instance of which will be assigned to each player. This class holds the list of protection areas near around the player, and whether the player is allowed interaction in them, or not. It already has a utility function CanInteract() that determines whether a player can interact with the specified block or not, based upon the stored areas. However, the implementation is still missing key components - modifying the list, creating and destroying.

ProtectionAreas.lua implements the Initialize() function called when the plugin starts. It doesn't do anything fancy, the work is offloaded to individual modules.

Storage.lua is currently empty, it will implement the cStorage class representing the abstraction for accessing the database.

The version attached to this post is also committed to the main MCS repository in rev 1518.

[xoft]

Second iteration
The second iteration finally brings some functional code. To test that the actual protecting will work, the cStorage class is prototyped, with the loading function always returning one hard-coded area, (10,10) - (20,20). This way, we can focus on the actual act of protecting - handling the hooks so that the area cannot be touched by a player.
Also, it became clear that the PlayerAreas map needs to map player IDs to areas, instead of player names, because if a player connects with two clients (perfectly legal in MCS), their areas would get mixed with the name approach. With the ID approach, everything should work fine, each player will get their own instance of the cPlayerAreas class.

This version of the code has been tested to actually do something - now no player can interact with the blocks in the hard-coded area. Both block placing and block breaking are disabled. However, player's tools are still damaged when breaking blocks. This might be a bug in MCServer, and needs further examination later on.

The version attached to this post is also committed to the main MCS repository in rev 1557.

[xoft]

Storage startup
The next step is somewhat out-of-order, it was not exactly the most logical step to do, but since it posed an interesting challenge, I decided to tackle it first.
The SQLite DB that we're planning for the data storage needs to have its tables set up properly. We could either distribute an empty DB file with the plugin, or we could set the tables up in code. Since the tables structure may change in the future, which would necessitate this upgrade code anyway, I decided for the second option. So upon plugin startup, the storage needs to open the DB file and then check the structure. We want to ensure that the DB has at least the tables and columns we'll be using later, so the code first defines those tables and columns in variables, and then calls a function for each table that "massages" the current table, if any, into the needed structure. This piece of code is rather interesting and I'm quite proud of coding it, so I'll be commenting on the actual code. It is the cStorage:CreateTable() function inside Storage.lua.

function cStorage:CreateTable(a_TableName, a_Columns)
	local sql = "CREATE TABLE IF NOT EXISTS '" .. a_TableName .. "' (";
	sql = sql .. table.concat(a_Columns, ", ");
	sql = sql .. ")";
	local ErrCode = self.DB:exec(sql);
	if (ErrCode ~= sqlite3.OK) then
		LOGWARNING(PluginPrefix .. "Cannot create DB Table, error " .. ErrCode .. " (" .. self.DB:errmsg() .. ")");
		return false;
	end

First, the table is created, if it doesn't already exist. SQLite has a simple command for that, but it's impossible to know whether it created the table or it did exist before, so we need to continue in either case.

	local RemoveExistingColumn = function(UserData, NumCols, Values, Names)
		-- Remove the received column from a_Columns. Search for column name in the Names[] / Values[] pairs
		for i = 1, NumCols do
			if (Names[i] == "name") then
				local ColumnName = Values[i]:lower();
				-- Search the a_Columns if they have that column:
				for j = 1, #a_Columns do
					-- Cut away all column specifiers (after the first space), if any:
					local SpaceIdx = string.find(a_Columns[j], " ");
					if (SpaceIdx ~= nil) then
						SpaceIdx = SpaceIdx - 1;
					end
					local ColumnTemplate = string.lower(string.sub(a_Columns[j], 1, SpaceIdx));
					-- If it is a match, remove from a_Columns:
					if (ColumnTemplate == ColumnName) then
						table.remove(a_Columns, j);
						break;  -- for j
					end
				end  -- for j - a_Columns[]
			end
		end  -- for i - Names[] / Values[]
		return 0;
	end
	local ErrCode = self.DB:exec("PRAGMA table_info(" .. a_TableName .. ")", RemoveExistingColumn);
	if (ErrCode ~= sqlite3.OK) then
		LOGWARNING(PluginPrefix .. "Cannot query DB table structure, error " .. ErrCode .. " (" .. self.DB:errmsg() ..")");
		return false;
	end

As a second step, the database is queried for columns in the table, using a special PRAGMA SQL command. That command returns all of the columns' IDs, names, types and whatnot, we simply extract the column name (i-loop) and remove it from the list of columns that the function received (j-loop). Since we can receive columns that don't specify only a name, but also some type specifiers, we need to strip those away before comparing the column names, that's what the string.find() and string.sub() inside the j-loop are for. The code is not exactly top-performance, but since it's executed only once on plugin initialization and only for 2 tables, I think it's better to have less performant code that is easy to understand, rather than a jumping-through-hoops code that no-one understands.

	if (#a_Columns > 0) then
		LOGINFO(PluginPrefix .. "Database table \"" .. a_TableName .. "\" is missing " .. #a_Columns .. " columns, fixing now.");
		for idx, ColumnName in ipairs(a_Columns) do
			local ErrCode = self.DB:exec("ALTER TABLE '" .. a_TableName .. "' ADD COLUMN " .. ColumnName);
			if (ErrCode ~= sqlite3.OK) then
				LOGWARNING(PluginPrefix .. "Cannot add DB table \"" .. a_TableName .. "\" column \"" .. ColumnName .. "\", error " .. ErrCode .. " (" .. self.DB:errmsg() ..")");
				return false;
			end
		end
		LOGINFO(PluginPrefix .. "Database table \"" .. a_TableName .. "\" columns fixed.");
	end
	
	return true;
end

At this point, the a_Columns variable holds all the columns that we'll be needing and that aren't in the DB. So the last step is to alter the DB table, adding each column.

This is from rev 1559 in the MCServer SVN.