Network API
#1
I'd like to discuss the API for networking that will be exported to Lua, and could also be used by the C++ server itself, internally.

There are essentially three objects: Client, Server and Link. The Client represents the active point of the connection - it actively connect to the other part. The server is the passive part, it listens for connections and accepts or drops them. Once these two connect to each other, a Link is formed, representing a single connection. The Link is the actual byte-sending pipe doing all the hard work, the Client and Server are just some abstractions helping to establish Links.

Since the Client and Server are used only as factory functions, we're better off wrapping those factory functions into a Network namespace instead, together with utility functions (hostname lookup etc.). However, we still need a way to terminate the server, so there's a special type, ServerHandle, just for that.

From this point of view, the following set of functions come to mind:
Network:
- Connect (Host, Port, OnSuccessFn, OnReceiveFn, OnErrorFn) -> bool
- Listen (Port, OnAcceptFn, OnReceiveFn) -> ServerHandle / nil + ErrorMsg
- HostnameToAddress(Hostname, OnResolvedFn, OnErrorFn) -> bool

Link:
- Send (string)
- (OnReceiveFn)
- GetLocalAddress()
- GetRemoteAddress()
- Close()
- Drop()

ServerHandle:
- Close()

The Link can be used to send data to the remote peer, or query information about both endpoints of the link. It also inherits the OnReceiveFn() callback and calls that callback whenever there's data incoming from the remote peer. Note that Send() is asynchronous, it will not wait for the data to be sent. The Close() method closes the link gracefully (waits for queued outgoing data to be sent), the Drop() method resets the connection.

The Network's Connect() method would receive four parameters, the address to connect to (IP + port, hostname + port), a callback to call on success, a callback to call when there's incoming data on the link (the newly created Link will inherit this callback) and a callback to call on error. The method would return immediately and the connection will be established asynchronously on the background. Once connected, the OnSuccessFn() will be called; it will receive the newly created Link instance and can immediately send data over the link.

The Listen method would receive three parameters, the port to listen on (both IPv4 and IPv6 ports will be used simultaneously(?)), and a callback to call on incoming connection. The OnAcceptFn() callback will be called for each new incoming connection, it will receive the newly created link and the identification of the remote peer as parameters. The OnReceivedFn() callback will be inherited by each newly created link

The ServerHandle's Close() method will simply close the server, and all the Links associated with that server.

Example plugin code:
-- Simple "echo" server with a welcome message:
function CreateEchoServer(a_Port)
	return Network:Listen(a_Port,
		function(a_Link, a_RemoteHost, a_RemotePort)  -- OnAcceptFn
			-- Example: filter out remote hosts beginning with "10.1.":
			if (a_RemoteHost:match("10\.1\..*")) then
				a_Link:Drop()
			end

			-- Send the welcome message:
			a_Link:Send("Welcome to the echo server\n")
		end,
		function(a_Link, a_IncomingData)  -- OnReceiveFn
			-- Repeat the incoming data into the output:
			a_Link:Send(IncomingData)
		end
	)
end


-- Simple webservice API client:
function CallWebHook(a_CommandToSend)
	Network:Connect("api.example.com", 80,
		function(a_Link)  -- OnSuccessFn
			a_Link:Send("POST /endpoint1 HTTP/1.1\r\n")
			a_Link:Send("Host: api.example.com\r\n")
			a_Link:Send("Content-Length: " .. string.len(a_CommandToSend) .. "\r\n\r\n")
			a_Link:Send(a_CommandToSend)

			-- We don't need to see the answer, so we just close the connection once all the data has been sent:
			a_Link:Close()
		end
		-- No OnReceive handler means that incoming data will be ignored
		-- No error handler means that errors are silently dropped
	)
end
Reply
Thanks given by:
#2
Does this implement both TCP and UDP?
Reply
Thanks given by:
#3
Only TCP. UDP is a completely different beast.
Reply
Thanks given by:
#4
Why not just use LuaSocket for this?
Reply
Thanks given by:
#5
Because LuaSocket blocks the whole server.
Reply
Thanks given by:
#6
It is possible to use LuaSocket in non-blocking mode, but programming with that is a nightmare. Every read may fail, may return any number of bytes, etc. It's much easier to use event-based IO.
Also, this could unify the API and the objects used inside the server itself. We could use high-performance epoll-based implementation etc.
Reply
Thanks given by:
#7
Could we also implement the ddp protocol used by meteorjs and enable a little mongodb?
Reply
Thanks given by:
#8
DDP seems to use TCP as the underlying transport protocol, so once this API is in place, it can be implemented on top of it.
MongoDB doesn't seem to be a protocol at all.
Reply
Thanks given by:
#9
okay, yeah I know, that mongoDb isn't a protocol XD i just wanted to throw that in. Because with a nosql database the programming of plugins with a data storage would be really easy.
Reply
Thanks given by:
#10
This looks great, finally there will be a way to implement irc bridge! Smile
Reply
Thanks given by:




Users browsing this thread: 16 Guest(s)