metacall()()()™ API and application architecture pattern

callstream

metacall means “call of calls”

I am one of those people who think that JavaScript should not be changed. No new keywords should be added. If anything, some should be removed, like ‘new’ for example. But there is one concept and one keyword which is missing: interface. And without interface API is very difficult to make simple.

Originally published 2010 Mar 05

Class inheritance is evil. Interface inheritance is a “good thing”.

That is the key concept which delivers high degree of decoupling and encapsulation. Two corner stones, in the foundations of application Architecture concepts. To actually implement them , interfaces are necessary. And by this I do not mean classes pretending to be interfaces.I mean pure interfaces.

I like JavaScript prototypical nature. The concept of “class” is unnecessary to write great code. Keep that in mind. Generic programming and interfaces is all that is needed. Maybe you did not know that, but even C++, does not enforces classes onto you. Standard C++ library (aka “STL”) is prime example what narrow and precise interfaces + generic programming can do. JavaScript is much “poorer” language, than C++, but still extremely expressive. I am using it a lot, and I always miss interface concept, implemented directly in a JavaScript language.

So, I suppose this new JavaScript pattern, I am toying with, might be important. This is closest I ever come to having interfaces in JavaScript. It is delivering an decoupled communication, or “interfacing”, with some state “in the back”. Simply by providing a mechanism for creating a stream of function calls, “anchored” and conceptualized as one call.

It is important so I shall emphasize it : this is not an Interface, that is missing from JavaScript. Rather this is communication mechanism that I propose to be used ubiquitously, to achieve the highest degree of decoupling between implementations and their callers. The interface here is in the mechanisms and communication kind used which is: messaging. I propose non-trivial JavaScript programs to be entirely composed in this way.

I have very high hopes for this mechanism and communication pattern, and I shell describe it here, step-by-step. Let’s perform an “deep plunge” first :

// metacall

// previously known as CallStream

function metacall ( ) { return metacall; }

That is it. The pattern I am describing here I shall call: “metacall”. It is presented here as an JavaScript mechanism. In its simplest and purest form. Just a concept, described in code. A very simple coding idiom. A function whose call, returns that same function itself.

With the snippet and function above we can write this most exciting, piece of javascript.

1 2	// stream of calls. All under one "metacall" metacall ()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()()() ;

Besides the fact that above code is useless, it actually is legal JavaScript and it works. We can write an endless “stream of calls” , just by calling this crazy “metacall” first. We can compose the chain of calls, starting with this one call first. Hence the name “metacall”. Of course, we can pass any number of any kind of arguments to any of these calls. As naturally as any other JavaScript code does.

// stream of calls. All under one "metacall"

metacall (1)(true)(new Date)([4,5,6])({a:1})(document.body)

(navigator)(true)(function(){})(false)()()()() ; // ad infinitum

Remember. All of this call’s the same function: metacall(). So we can collect the arguments and do something with them, before next call comes in.

function metacall ( ) {

// here do something with arguments

call_handler ( arguments ) ;

// return metacall for next call

return metacall;

}

Above call_handler(), is an gateway to anything. Anything that can do something with some arguments passed to the metacall(). Different metacalls can be made just by using different call_handler’s.

// different meta callers are made by using different call_handler's

function metacallOne ( ) {

call_handler_one ( arguments ) ; return metacallOne;

}

function metacallTwo ( ) {

call_handler_two ( arguments ) ; return metacallTwo;

}

I think that by now I can prove that metacall pattern decouples interface from implementation. It delivers the interface as a separate entity. And. it provides interface with no methods.
It is a kind-of-a interface. And very flexible one indeed. Specific metacall needs almost never be changed. It is very resilient to change in the underlying state to which it passes arguments. It does not want to know anything about arguments. It just want’s to pass them.

I think here I have an pattern with well defined, narrow and single purpose of existence. To allow metacalling (aka chaining), and nothing else. Nice. Perhaps. Still a bit useless. How do we make this useful? Maybe by finding an very good “counter” example? jQuery call chaining perhaps?

These days “call chaining” is very popular and fashionable JavaScript sport. jQuery crowd can’t have enough of call chaining. Perhaps I can use my metacall to devise some new and “awesome” call chaining mechanism, better than jQuery does? Although jQuery uses what some might call “simpler” and more “normal looking” form :

1	$("selector").find("div").attr("A",1).css("color", "#fff") ; // and on, and on ...

Function $() returns an object ,where each public method of that object returns, again the same object. Above chain calling syntax is actually very efficient and leads to smaller and more efficient code. It is also logical. Because all calls in a chain, naturally are working on the same underlying state (dom nodes) , which is defined by the first call to the jQuery itself. After seeing this for the first time, and an initial “JavaScript cultural shock” it is easy to understand that jQuery chain deals with the same instance of an jQuery object. As defined with the first call, which has selected some dom node set.
Ok, but how is “metacall” performing when compared to this “traditional call chaining”? This is how. Imagine jQuery version which is based on “metacall” , and let us try and repeat the chain above, shall we ?

1 2	// jQuery interface based on metacall uasge $("selector")(find,"div")(attr,"A",1)(css,"color", "#fff") ; // chain proceeds here...

What do I see above? First of all everything becomes much more flexible. How so? Simple: there is no ‘.’ (dot) operator. Functions used can come from anywhere. Functions used can be changed in a totally free manner. They are not attached to jQuery. “find”, “attr” and “css” might be jQuery “aware” functions or they might be any other functions which “know” how to achieve the required result. They are “truly functional”! therefore they can be 100% re-usable. They might work on two, or more , similar , but different “metacalls”. It is just a question of “contract” they have to oblige to. These functions can even be used at run-time, on several wildly different instances of metacalls. They can become “Stream Visitors”. Generic Functions re-used (aka “called”) on many different metacall instances. Example is due.

// StreamVisitor re-usability

metacall("#container")(find,"div")(attr,"A",1)(css,"color", "#fff")(logger) ;

// logger is a StreamVisitor

metacall("#toolbar")(css,"color", "#fff")(logger) ; // logger re-used

Function “logger” above is a “StreamVisitor”. Without any change, one single generic function is used to send logs from any jQuery instance. The “StreamVisitor” requirements are in the “contract”. They need not follow any dependancy on some another functions or objects. And all of that without any extra “plugin” mechanism. Every function in this stream of calls is a “plugin”.

And this is the second “good thing” about this pattern. No methods. Every function capable of operating on the hidden state of the metacall is a stand alone function. It is not a method. There are no methods. Just functions. No special “plugin” mechanism is needed. Everything that can be used as a StreamVisitor is a “plugin” in a traditional meaning of the term “plugin”.

But wait, there is more ?! And this makes me dizzy I think. We can go one step further in this decoupling “jamboree”. We can write a metacall so that we even do not use function objects. We can aim for OO “nirvana”: just objects and messages. We can encapsulate everything inside metacall. Let me show you this form of metacall on the example above.

// metacall pattern with no function objects

metacall("selector")("find","div")("attr","A",1)("css","color", "#fff") ;

// calling chain with no functions

metacall does not prescribe anything about types of arguments, of course. The form above would give us even more freedom and flexibility. Above is almost an “jQuery mini language”. Just strings. jQuery commands, vs functions (StreamVisitors) or methods on jQuery Meta Caller object.

This is the third good thing. metacall allows for total encapsulation of operations on some state. Combined with stream of calls. And still in a controlled manner. metacall actually does not have any state, it does not know nor it does care what call_handler does. Consider an (imaginary) SVG encapsulation :

// metacall pattern for svg simple wrapper

// create svg canvas

var canvas = svgmetacall('svg width="5cm" height="4cm" viewBox="0 0 500 400"')

('polyline points="100,200 100,100"')

('polyline points="250,100 250,200"') ; // break here

// proceed somehwere else

canvas('polyline points="250,200 250,300"')

('path d="M100,200 C100,100 250,100 250,200 S400,300 400,200"')

('circle cx="100" cy="200" r="10"')

('circle cx="250" cy="200" r="10"') ;

No functions. Just a “language” like API. In this case we use a well defined , exsiting SVG syntax, plus metacall, to deliver an non obtrusive javascript syntax which allows the user to concetrate on the SVG but still stay inside the javascript code. Minus, tiresome SVG XML syntax of course.

function svgmetacall ( canvas_declaration ) {

// called once , at the metacall beginning

initialize_svg_canvas( canvas_declaration ) ;

return function metacall () {

// called each time

handle_svg_calls( arguments ) ;

return metacall;

}

Here I have made a bit more realistic example and introduced one more attribute of this pattern : an metacall Conctructor. Which is kind-of-a interface constructor. With the same purpose as any other classical “constructor” : to provide initialization opportunity. An initial input to aid object creation and preparation. To do the above in a traditional way, we would need to have some traditional “object with methods” :

// traditional svg wrapper

// create svg canvas

var canvas = new Svg({ width:"5cm", height : "4cm", viewBox:"0 0 500 400"});

canvas.polyline([100,200 100,100]) ;

canvas.polyline([250,100 250,200]) ;

canvas.path({ d : "M100,200 C100,100 250,100 250,200 S400,300 400,200"});

canvas.circle(100,200,10); //x,y,r

canvas.circle(250,200,10) ;

Summary

Perhaps I could formalize this meta-pattern in its (for the time being) final JavaScript shape.

// called once on the first call

function metacallConstructor ( ) {

// pass initial arguements to the state

metacallBridgeInitiator ( arguments ) ;

// callers use this function for "metacalling" to the state behind

return function metacall () {

// A bridge to the state behind

metacallBridge ( arguments ) ;

return metacall ;

}

metacall meta-pattern (mechanism) gives us additional degree of flexibility in composing concrete patterns and designs. By basing designs on this meta-pattern and concepts, we are comprehensively decoupled from “no-interface” issues, while still handling the underlying state effectively. metacall is simply encapsulating pure interfacing activity. An simple yet powerful mediator concept, that decouples users from implementations.

Also, it delivers method for brokering the calls between user and underlying state handler. In a totally decoupled manner. Opposite of this, in every traditional objects-and-methods universe, interfacing mechanism and implementation is unfortunately interlaced with actual implementation of the logic of handling the object state.

Let me repeat once more : metacall pattern decouples user from interface interface from implementation. It delivers the interface as a separate mediating entity. And. It provides an message based interface, with no methods ;)

20 thoughts on “metacall()()()™ API and application architecture pattern”

20 Comments

Balazs Endresz
2010-03-30, 13:16
Interesting idea, let me reflect on some other aspects of this:

First, I think calling this an interface pattern is slightly misleading, it’s more like sealing an object, where you can’t add methods to it any more (in case of passing strings). Of course, compared to prototypal code you can have much more control over the properties of the underlying object but on the other hand you can’t really extend it, only pass functions to it. For example, how would jQuery plugins look like without having a shared prototype object, where they can be organized?

Moreover, you can use arbritary functions with prtotypal code as well with an extra helper method. A naive implementation in jQuery:

1
2
3
4

$.fn.apply = function(fn){
fn.call(this);
return this;
}

where you can use the above like this:

1

$().apply(function(){ this.toggle(); });

What I like about your “callstream” pattern though, is that it indeed hides the logic of state management/mutation from the user, which might be a “good thing” in some use-cases.
Speaking about state, you don’t even have an object here, but only functions, and when you assign a function to a variable, then unlike an object, it won’t create a reference but will rather create a copy of that function. This might have some performance implications. And you won’t be able to determine if two callstreams are the same (though I’m not sure if that would be meaningful at all), not to mention that since one might expect it to work as an object it could even lead to errors if not used accordingly.

The last issue, which might need some more explanation as it’s a bit far reaching, is that you’ve actually created a new notation to write (sort of) monadic code in javascript:
The last two links above are two examples showing how this could have been done until now. Basically you could simulate Haskell’s “do notation” in javascript by creating an OO wrapper, where indeed, passing standalone functions isn’t quite comfortable. Or by passing higher order functions as arguments to a “sequence” function. So I think your “callstream pattern” might be a really nice alternative for these. Though I’m not sure I would use it in place of existing prototypal code :)
Dusan
2010-03-30, 14:44

Thanks Balazs.

First of all terminology is extremely important when inventing new patterns. “interface” is indeed an heavily and unfortunately overloaded term. Therefore, I think that I should use the word: Interfacing. So it is not a name any more, it is an Activity .

This is dramaticaly different and I think (dramatically) better term, for describing the concept it is naming. As indeed, this is all about interfacing activity and/or communicating in a decoupled manner with some implementation which is behind a “CallStream”.

Much lake communication through REST end points. Completely message based and decoupled from the server side components. Messaging system and concept but on the micro level, of the JavaScript programs.

With CallStream “in mind”, I should obviously revisit my Moment := micro + component idea, and produce something more useable, than the sum of these two articles.

Monadic point, you made is very interesting too. I shall try and engage in productive exchange of ideas with authors behind the links you provided.
Also you said : “…Though I’m not sure I would use it in place of existing prototypal code…”
Which renders me confused: what can be more “prototypal” than CallStream ;)

–DBJ
Bertrand Le Roy
2010-03-30, 22:38

Very interesting idea and definitely true to JavaScript. You should submit that on Ajaxian to get more feedback and generally increase exposure of the idea.
I was also confused by the “interfacing” name. Call stream is pretty clear on the other hand.
The pattern is similar to jQuery’s each (with a terser syntax) or more accurately Balazs’ apply plug-in in the comment above. Ruby blocks are also kind of similar, as are some Lambda parameters in some C# patterns.
But the syntax is definitely novel and fun. I liked the crazy CallStream()()()()()…
I also like that it provides a non-plugin extensibility device.
But I stopped agreeing when you replaced the function with strings: you would need something to handle the commands and removing functions also removes the main appeal of the pattern, which is frictionless extensibility.
Plus, it then becomes a sort of DSL expressed in string form, and you’re losing a lot by doing that (ability to analyse the code, provide auto-completion, etc.)
I’ll try to find a way to express the same idea in C#. Should be fun.
HB
2010-03-30, 23:07

Very interesting post. The “logger” example made me wonder, how would you introspect the current “stream”? For example, you might want to make a series of calls, then log the results all at once as the final call in the chain.

Since (currently) each call returns the same CallStream object, is there a simple way to trace streams without affecting other streams?
Bertrand Le Roy
2010-03-30, 23:40

He he. I have a working C# implementation… Blog post pending.
Scott Koon
2010-03-30, 23:59

Nice, my first thought was “maybe you could nest calls to CallStream?”

(CallStream(CallStream(“foo”, CallStream(1024, CallStream()))));

Oh my, we ran smack into LISP again. ;)
Dusan
2010-03-31, 00:33

@Bertrand : C# will be fun, especially using C#4 and its dynamic/functional extensions. Variant with strings instead of functions is just an example of the flexibility of this pattern. Heck, maybe I can, or should call it : “meta pattern”, since it obviously can be a base of the whole family of more concrete patterns.

@HB : I think you dived into more concrete usability issues, which is good. Although I think “stream introspection” should be decoupled from a CallStream. As “anything” else. It is separate mechanism, that is functional but not part of the CallStream, and still serves the purpose. Remember: CallStream just dispatches the function arguments, one after another. Where and how and why is part of the concrete design and implementation. CallStream is just an meta pattern. A conceptual thinking framework for building concrete frameworks.

@Scott K. : Ah, I have to admit now that I was “raised” on Auto Lisp. And some of it was non-trivial indeed ;)
Bertrand Le Roy
2010-03-31, 01:36

C# implementation can be found here: http://j.mp/cz7GSJ
Fluxtah
2010-04-01, 12:55

Nice, I guess you could also defer execution so you can do funky things like execute calls in reverse or randomly or by some interesting algorithm. :)

Dusan
2010-04-01, 15:20

@Balazs: I was always wondering do you understand your JavaScript potential ? Seriously. Your CallStream curry is great.

Balazs Endresz
2010-04-17, 10:05

Thanks, but the CallStream curry is actually wrong! Try this:

1
2
3
4
5
6
7

var fn = curry(add);
fn(1, 2);
console.log(fn(3));

var fn2 = curry_old(add);
fn2(1, 2);
console.log(fn2(3));

What happens is that you call the same function twice, but with the callstream curry you have only one shared state where the variables are stored. So the functions behaves quite strangely, if you pass any parameters then all the other ones will receive it as well.
Dusan
2010-04-17, 10:13

Idea is great. Implementation needs work ;)

Balazs Endresz
2010-04-21, 18:49

Well, I might implement this because it allows you to reference parsers that will be defined only later on, so you can’t refer to them by plain javascript variables (just by adding function wrappers). Other that it doesn’t make anything easier but this would be still a useful thing.

Another use case is to keep the ParseState objects (which contain the string to be parsed, and the cache – see “packrat parsing”), so if you run a parser with the same string it could automatically reuse the stored ParseState object. But you rarely need this feature and the state can be saved manually as well.

I don’t use it for initialization? Without this feature, why would I do it?

The interesting part is that you can’t know which call will be the last, but if it receives a different type of argument, only then will it be evaluated (and cached). During that you can easily replace use("p1") with the actual parser, which can be even itself. The logic is pretty much decoupled I think, as this is just a layer on top of the do_ function and the operator resolution thing – it’s exactly what I needed to unite the two.

Comments are closed.