NetWyrm’s Litte Corner

In my last post I talked about patching LuaSocket to make it work with LuaLanes. The patch is there: lua socket acceptfd patch.

So what does it do?

It adds a new method “acceptfd” which works just like “accept” but instead of returning a socket object it returns  the underlying “file descriptor” (a number). A new optional parameter to “socket.tcp()” allows to create a socket object from a file descriptor.

How does it help us ?

Lua Lanes handles multithreading by running a new lua state in each thread, they do not share data and instead communicate using Lindas (see the docs on Lanes site). Linda is great but it cannot pass such userdatas as sockets. Si we use this patch to accept an incomming connection, getting its  file descriptor. Then pass it onto a new thread which creates its socket object from it.

Easy, fast and grooovy!

A little exemple:

require ‘lanes’
require ’socket’

local linda = lanes.linda()

function handler(fd)
require’socket’

local sock = socket.tcp(fd)
local line = sock:receive(”*l”)
linda:send(”order”, {line=line})
sock:send(”line: “..tostring(line)..” \n”)
sock:close()
return “done”
end

function main_thread()
while true do
local order = linda:receive(3, “order”)
if order then
print(”order”, order.line)
end
end
end

function run()
local s = socket.bind(”0.0.0.0″, 2525)
local threads = {}

lanes.gen(”*”, main_thread)()

while true do
local fd = s:acceptfd()
threads[lanes.gen("*", handler)(fd)] = fd

print(”threads:”)
for h, s in pairs(threads) do
print(” * “, h, ” :=: “, h.status)

local v, err = h:join(0)
if not v and err then
print(err)
elseif v and v == “done” then
threads[h] = nil
end
end
end
end

run()

(Badly formated sorry, I have not yet mastered the art of posting code with wordpress )

Ever wanted to make a network server in Lua?

You have come to the right place, after much experimentation on Tethys and Mailcatch.

When it comes down to how you can do a sockets server you have those options:

• A single process with lua coroutines. The main thread will handle the server socket, when accept’ing connections it starts a new coroutine to handle it. A coroutine scheduler is necessary to yield while waiting on sockets. Such a scheduler could be Copas or LOOP. I will write about it later but I have found this way to choke when handling many connections/seconds
• A forking server. The main process handles the server socket, when accept’ing connections it forks a new process to handle it. This is quite easy to write using lua posix or some more specialized libs (I will release one that wraps it up nicely someday).
• A single process server with multithreading. Lua does not do multithread but there are modules out there to do it. The most prominent one being Lua Lanes.

I have experienced with all three (in this order) and settled for multithreading using Lanes with a twist (it does not spawn a new thread for each connection it uses a pool of pre-existing ones) because it gave me, by far, the best performance.

In my next posts I will explain those three methods and how to implement them easily in Lua.