Thursday, September 10, 2009

Removing warnings from ct_expand

I use ct_expand alot (can you tell?). The released version outputs warnings about unused variables, which are harmless, except ... that it conditioned me to ignore unused variable warnings from the compiler. Then the other day I had a real bug which the compiler was warning me about (via an unused variable) that took me quite a bit of time to find. Lesson learned: make sure that a correct compile is as quiet as possible, so problems stand out.

Ulf gave me some pointers on how to prevent ct_expand from emitting these warnings. Basically, ensure that the temporary variables created by ct_expand are prefixed with an underbar. Here's a patch:

--- src/ct_expand.erl 5 Jun 2009 21:07:32 -0000 1.1
+++ src/ct_expand.erl 10 Sep 2009 04:12:07 -0000
@@ -139,6 +139,9 @@
{Fname, Arity} = erl_syntax_lib:analyze_function(Form),
VarNames = erl_syntax_lib:new_variable_names(
Arity,
+ fun (N) ->
+ list_to_atom ("_V" ++ integer_to_list (N))
+ end,
erl_syntax_lib:variables(Form)),
{Form, [{function, Fname},
{arity, Arity},

Tuesday, September 8, 2009

Application Variables during Upgrade

A quick note related to my previous post: it turns out the application controller gives "old values" for application variables during the code upgrade process. Therefore the call to

element (2, application:get_key (drurly, modules))

returns the old list of modules, i.e., is missing new modules. ct_expand to the rescue! I get the list of modules at compile time from the application specification via

Modules =
ct_expand:term (
begin
{ ok, [ { application, drurly, PropList } ] } =
file:consult ("drurly.app"),
case proplists:get_value (modules, PropList) of
Y when is_list (Y) -> Y
end
end
),

For this to work, you must ensure the application specification is generated prior to compiling the module with this function in it.

Wednesday, August 26, 2009

Dynamically Loading Webmachine Resources

I've been using webmachine lately, which is fabulous for REST server development. It has a modular concept of resources and the design could be summarized as a "RESTlet container". Because Erlang has hot code loading I was interested in doing this dynamically; in other words, I want to write a module which provides a webmachine resource, hot code load it into the system, and have webmachine start dispatching requests to it.

The good news is, you can change the dispatch list at any time, via
application:set_env (webmachine, dispatch_list, NewDispatchList).
This does not upset webmachine and it will see the change immediately. So for a single application leveraging webmachine, this seemed as simple as having a code_change handler execute something like
application:set_env
(webmachine,
dispatch_list,
lists:foldl (fun (Mod, Acc) ->
case catch Mod:dispatch_rules () of
{ 'EXIT', _ } -> Acc;
X -> X ++ Acc
end
end,
[],
element (2, application:get_key (drurly, modules)))).
(drurly is the name of the application in this case). Now my modules export a function like

dispatch_rules () ->
[ { [ "clip" ], ?MODULE, [] },
{ [ "clip", id ], ?MODULE, [] }
].
if they are webmachine resources.

The problem is that order is important: the webmachine_dispatch_list is consulted in order, and the first match is executed. To solve this I created a sort function which orders the dispatch rules by "specificity"
path_spec_priority ('*') -> 3;
path_spec_priority (X) when is_atom (X) -> 2;
path_spec_priority (X) when is_list (X) -> 1.

dispatch_specificity ({ PathSpecA, _, _ },
{ PathSpecB, _, _ }) ->
case erlang:length (PathSpecA) - erlang:length (PathSpecB) of
X when X > 0 ->
true;
X when X < 0 ->
false;
_ ->
PrioPathSpecA = [ path_spec_priority (X) || X <- PathSpecA ],
PrioPathSpecB = [ path_spec_priority (X) || X <- PathSpecB ],

case PrioPathSpecA =< PrioPathSpecB of
false ->
false;
true ->
FullPathSpecA = [ { path_spec_priority (X), X } || X <- PathSpecA ],
FullPathSpecB = [ { path_spec_priority (X), X } || X <- PathSpecB ],

FullPathSpecA =< FullPathSpecB
end
end.
Basically:
  • Longer dispatch paths come first.
  • If two dispatch paths have equal length, the more specific one comes first, where specificity is defined by examining the elements left-to-right, with
    • string literals are most specific
    • atoms except '*' are the next most specific
    • '*' is the least specific
  • If two dispatch paths have equal length and specificity, sort by Erlang term order (effectively, break ties arbitrarily)
The code change handler now becomes:
application:set_env
(webmachine,
dispatch_list,
lists:sort
(fun dispatch_specificity/2,
lists:foldl (fun (Mod, Acc) ->
case catch Mod:dispatch_rules () of
{ 'EXIT', _ } -> Acc;
X -> X ++ Acc
end
end,
[],
element (2, application:get_key (drurly, modules))))).
This has proven sufficient for a single application running webmachine.

For multiple applications that want to run under the same webmachine, I suspect the right way to go is to have a gen_server which
  • contains webmachine dispatch configuration "fragments" per key, where the key is intended to be the application name;
  • accepts commands to replace or delete a particular fragment by key;
  • for any replace or delete, rebuilds the complete dispatch list by concatenating all fragments together and sorting by specificity, and then updates the webmachine application env var.
For multiple applications that want to run under different webmachines, well unfortunately webmachine uses some global application settings under the fixed atom webmachine and thus currently, like the highlander, there can be only one. (In fact, you can only listen on one ip/port combination with webmachine right now. I might have to patch it to accept multiple ip/port combinations to listen to, since a standard trick of mine is to have nginx handle both regular and ssl connections and connect to a different back-end port to indicate whether or not the connection is secure.)