** This area is for discussion, please see This Wiki for notes on layout. ** - Andre Carregal
WSAPI and frameworks
In particular I'm very curious to see if WSAPI would be usable by Javier Guerra's solutions for Xavante, or if we are still talking about two different views of the web content generation. - Andre Carregal
Orbit goes Camping has a first stab of WSAPI, with a wsapihandler for Xavante. Each Orbit application honors the WSAPI protocol. I am also separating query string parsing and postdata handling in a
wsapi.requestmodule, and setting cookies in awsapi.responsemodule. - Fabio MascarenhasI like the interface proposal; but some things are still unspecified, most importantly about the isolation guaranties. Fabio's wsapihandler uses Rings; with all the performance implications it has, i hope it's not required!. -Javier Guerra
I'm working on the details as the feedback is coming. Concerning Orbit use of Rings that is their decision (as for CGILua for example). WSAPI does not assume any isolation whatsoever. It just provides a simple communication channel with the underlining Web server. My question about your use of it concerns the continuations based on coroutines. What would you need in order to be able to construct a filter based on WSAPI that would offer such feature? Assuming things work out as planned, do you think your solution could potentially be used with other Web servers or just Xavante? What would be missing? - Andre Carregal
I am not assuming isolation, either. I am using Rings because the CGILua handler used Rings, but Orbit applications are isolated from each other even without Rings,
orbit_app.run(env)runs a fresh copy of the application each time._ - Fabio MascarenhasI like the server type as "The server calls a registered application passing an Environment." It is most like java servlets, isn´t? The concept of lightweight container, IMO , will provide more flexibility to develop web application and frameworks. I work with java]]/J2EE a long time and a hope, one day, I can develop this applications using Lua but with the actual ( or more advanced ) concepts/frameworks like java comunity. During my experience (not much) using kepler, I missed a well defined application concept, not so connected with the server. What you guys think about it? Guilherme Martins
Apparently I have to work a lot more on the details... :o) WSAPI is not a framework, but the foundation for different frameworks. WSAPI could be used for a servlet like implementation, sure, but it is not a servlet view of web processing. What I have in mind is to offer a library that would be useful for CGILua, Orbit, "lualets" or maybe even continuation style frameworks. It may sound too bold, but it is just the opposite. Please check how Python's WSGI and Ruby's Rack handle the same problem we have here. - Andre Carregal
If WSAPI doesn't promise any isolation, then the current call interface proposal is very similar to Xavante's handler interface. The main differences are RFC3875 CGI-style variables (instead of just reading all request headers into a table), and returning an iterator for output (instead of the 'res' object). I'd like to note, however, that the 'res.content' field can also be a function, so it's not so different after all. - Javier Guerra
I'm thinking about not writing a WSAPI adaptor for Xavante, but making it the 'native' API. So far, my main objection is the RFC3875 CGI-style table for headers and extras. Is there any motivation for adhering to this standard?. Currently Xavante just puts the headers into a table, with lowercase keys, maybe reserving uppercase for any non-client data?. - Javier Guerra
This is exactly what I was thinking too. WSAPI could be the native API for Xavante and a thin layer over mod_wombat for example. Concerning the RFC3875 issue, I didn't get what you mean. Are your concern about the RFC itself or the field names in the Lua table? - Andre Carregal
- I just think it's ugly. All 'metavariables' share a single namespace, vagely segmented by prefixes, everything is in SHOUTING UPPERCASE, and there's room for interpretation of some defined vars (SCRIPT_NAME might the CGILua launcher, or the .lua file). I like to use two tables: one for the request's HTTP headers (lowercase), and another for server-handler parameters. Of course, if there's any real advantage to use the standard, so be it. - Javier Guerra
What about a compromise of using a single table for the values but splitting the values into fixed fields? So we could have something like
env = {
cgi = {...} -- meta variables
ext = {...} -- server parameters
wsapi = {...} -- wsapi attributes
input -- request input stream
errorlog -- error output stream
}
Concerning the confusion (present even in the current CGILua version) about SCRIPT_NAME and PATH_INFO, the rule suggested would be that SCRIPT_NAME refers to the application virtual path (how do we got "here") and PATH_INFO is the application target virtual path (what is left). Note that the definition of target is application dependent. - Andre Carregal
- I've been thinking that giving an iterator to return content is very Lua-like... for the WSAPI server, but not for the 'client' (that is, the app, or a new framework). How about this: ?
env = {
req_headers = {...}, -- request headers
wsapi = {...}, -- WSAPI-defined attributes
ext = {...}, -- server-specific extensions
input, -- function to read the request content
res_headers = {...}, -- response headers
output, -- function to write response content
}
Where calling env.output(data) for the first time would send the headers, so they should be filled before the first output. I'm not sure about other parts of the response (mainly the status line), they could be on env.wsapi{}, or on it's own 'return values' subtable.
Not sure how this would affect the stackability... maybe defining some functions to help a client manage it's own clients. - Javier Guerra
Right, this is a typical "Helper" module for those who prefer this style of content generation (CGILua uses it). The point of not assuming an output() function is that the server side should no worry about buffering the status and headers, that's up to the application side. Just remember that this could be an intermediate module.
I don't get why overload the env with both the input and output details. Why not just return the status, headers and then the output itself? And in this model how do I get something like PATH_INFO? - Andre Carregal
Javier, my wsapi.response helper exposes a more procedural format to an application (Orbit actually uses it). You do:
local res = wsapi.response.new() res.headers["X-Foo"] = "bar" res.status = "200 OK" return res:finish() -- returns the status, headers and an iterator to the content
You can also set cookies with this response object. I also have a wsapi.request that does GET/POST processing, and gives nicer (lowercase) names to some of the CGI metavariables. Both modules are at Lablua's SVN server. - Fabio Mascarenhas
My point (please prove me wrong) is that iterators are easy and nice to use, but more error-prone to create. Usually that means that the WSAPI client would have to use a coroutine-based loop. Don't you believe that WSAPI should be easier for the client than for the server?. And about loading env with both request and response... it's just because I fee that the client app.run(env) function shouldn't return until the response is finished. With the current iterator-based approach, it does finish but leaves a callback behind; Lua's scoping and GC makes it work very well, but might not be so straightforward for newcomers (hum... i usually don't care about that...).
Fabio, i saw your wsapi.response() helper, but didn't understand it. Would you document it a bit? looks very promising! - Javier Guerra
I will document it, but for now the best documentation is how Orbit uses it. From orbit.lua:
function app_instance_methods:run(wsapi_env)
local req = wsapi.request.new(wsapi_env)
local res = wsapi.response.new("200 OK", self.headers)
function self:set_cookie(name, value)
return res:set_cookie(name, value)
end
function self:delete_cookie(name)
return res:delete_cookie(name)
end
self.input, self.cookies = req.params, req.cookies
self:dispatch(req.path_info, string.lower(req.method))
res.status = self.status
res:write(self.response)
return res:finish()
end
Javier, keep in mind that WSAPI itself is supposed to be very low level. No newcomers would use it directly, only developers like Fábio and Tomás who would offer higher abstractions like Orbit and CGILua. What Fábio is currently proposing are a set of Helper functions for WSAPI that would handle details like URL processing, Cookies etc. While I like the idea of sharing that code amongst the frameworks, I'm still checking how this would be done in a way that both Orbit and CGILua (and eventual others) would have enough functionallity. - Andre Carregal
Even more so. Setting up a callback is a lot more overhead than calling output functions. Especially if you use a coroutine loop to get control inversion. - Javier Guerra
But if you only have an output function, how would WSAPI be able to garantee that no output would be sent before the status and the headers? This is exactly SAPI's current problem. By forcing the application to return either the complete contents or an interator, WSAPI can always send the status and headers before the contents. - Andre Carregal
Good point. Currently, what Xavante does is: the handler can set any res.header[] field at any time; but the sendresdata() calls sendheaders() function before sending data and only if it hasn't been called before.
IOW, the handler sets the headers, but they're ignored after the first output. - Javier Guerra
Exactly what CGILua and Orbit do. My point is that is a job for the application side, not for the server side. Remember that the server side will have to be implemented for every launching model, so this code would be replicated in every launcher. A trivial way to use it like this is to use Fábio's helpers (Request and Response), part of the WSAPI package, but not part of the filter API. - Andre Carregal
Can I ask if there's any technical advantage? As I see it, using an output function is less code (on both sides) and lower overhead. I really love iterators; but i feel it's not the be best pattern now. - Javier Guerra
API and dataflow
Although Fabio's implementation uses a different approach, I still think we can model WSAPI as a non buffered API, so instead of waiting for the full render of the response, the application could return the response in chunks.
This would be the "sequence diagram" for the interaction between a server and a application. Remember that any of them can be a filter.
Server App | | | App.run(env) | |----------------------------------->| | | (processes env and sets status and headers) | return status, headers, response | |<-----------------------------------| | | | response() | >|----------------------------------->| | | | (processes the next chunk) | | return chunk | -|<-----------------------------------| | | V V
So the API would be something like:
App.run(env)
Processes a request on an application App. env is the mentioned environment table. Generates the HTTP response by returning an HTTP status code, the HTTP headers and the response value.
If the response value is a string it is supposed to be the complete response and is sent in one piece to the server (and thus the request cycle is considered complete). If the response value is a table it can be iterated or concatenated before being sent to the server. If the response value is a function it is assumed to be an iterator and would be used called using response() to get each chunk.
One point that is still not clear is error handling. When an application detects an error before returning the status and headers, it can simply change the answer accordinigly. But what about when the error is raised after the status and headers had been sent? - Andre Carregal
- As shown above, i'm proposing a more 'procedural' flow, where the client calls some env.output() function to return content. I feel it's easier for the client than setting up a coroutine-based callback. (of course, setting an env.contents to a string or table should also work) - Javier Guerra
But you can return the whole content in a single string, wouldn't that be 'procedural' too? The difference is that any buffering would be done by the application, not the server side. - Andre Carregal
WSAPI as a Standalone Project
Another issue is that it may be better to someday move WSAPI to a standalone project on LuaForge in order to let it clear that it is not intrinsically part of Kepler. - Andre Carregal