Higher-Order

Conj-labs Clojure lessons part i

July 4, 2010 – 9:43 am

I attended the first conj-labs in Brussels, June 23-25. It was three days of intense Clojure learning with Clojure profiles: Christophe Grand and Lau Jensen. I must say it exceeded my expectations! Everyone was great and I really feel it took my Clojure to the next level.

This post describes some mistakes I’ve made in the Clojure circuit breaker project, and how I managed to fix them, mostly due to guidance from Christophe.

The following is an (embarrassingly long) list of issues that my previous version of circuit breaker had. I think others can learn from my mistakes.

Platform speed protocol functions. In order for a protocol function reaching a datatype to run at platform call-speed, the datatype must inline the protocol implementation. In the old version, I extended the state transitions protocol to the circuit breaker states using (extend …): this is usually 3-4x times slower than platform-speed method calls (but can be much slower when cache misses). I now inline the protocol extension to the states.

This inlining gave a new problem. I had two data types, say A and B, where A creates an instance of B and B creates an instance of A in the implementation of the inlined protocol functions. You can’t have this type of “mutually recursive data types” since they are compiled on the fly to Java classes. The fix was to…

Create factory functions for your data types. There are at least two reasons for this. First a factory function can do processing of its arguments (e.g. pre condition), say for validation. The data type constructors can’t have any logic. Second, as noted by Alex Miller on his blog: to use data types defined in another namespace you must import them. But you can use require/use to bring the factory functions into your namespace.

I thought I knew: protocols aren’t interfaces (and Java interop).. I used a protocol: CircuitBreakerTransitions to model the states. The protocol contains functions needed for state transition on various events. I extended the protocol to my states, e.g.,

(defrecord ClosedState [policy fail-count]
  CircuitBreakerTransitions
  ...)

It inlines the protocol so the generated class implements the protocol’s backing interface: It’s fast and works great.

The problem appeared when I wanted to expose a Java API for the circuit breaker: one of the API methods getCurrentState returns the current state – now, the question is what is its signature? My first attempt was public CircuitBreakerTransitions getCurrentState(). Now what is wrong?

The problem is that my protocol can be extended to reach other types. Those types may or may not inline the protocol functions, and hence it is possible for the protocol to reach a type which doesn’t implement the CircuitBreakerTransitions interface… So should I just return “Object” – that isn’t really useful.

I haven’t implemented a solution yet, but this is what I am thinking. Keep the type: CircuitBreakerTransitions getCurrentState(), but in my Java-interop (gen-class), the function that implements getCurrentState should grab the state s from the circuit breaker. Now I know that (satisfies? CircuitBreakerTransitions s) is true, but (instance? (:on-interface CircuitBreakerTransitions) s) may be false. If it is true, I simply return it. Otherwise: reify CircuitBreakerTransitions by delegating to the protocol functions on s. This may sound a bit wierd at first, but I think of it like creating a new object that satisfies the interface by dynamically dispatching to s.

Type-hits are for removing reflection, not enforcing type safety My first iteration tried to type-hint the records, e.g.,

(defrecord ClosedState [^TransitionPolicy policy ^int fail-count])

This is just plain terrible First, those hints are ignored (but may be used in later versions of Clojure). Second, int isn’t suported in the current version of Clojure. Third, TransitionPolicy is a protocol – and, yes, protocol’s aren’t interfaces (see above): problem being again if I supply an object that satisfies the protocol but doesn’t implement the backing interface. Ugh.
The reason I put the hints wasn’t to remove reflection, it was for the generated constructor to have types (which is a valid thing to want, but unsupported now). But it is the wrong approach for the reasons above.

The right thing is to remove hints and expose a factory function which checks that the first argument satisfies the TransitionPolicy protocol and that the second argument is non-negative.

Premature optimization. My circuit breaker implementation tried to optimize a corner-case for the swap! function: suppose you (swap! state f) and f applied to the current state is identically the same state. Then there is no reason to do a compare-and-set!. My transition-by! function implemented this optimization. However, this was a premature optimization that changes the semantics of time (wow, Clojure really is powerful ). The fundamental question is this: [s] -f-> [s] (where s is a state and f is a side-effect free function that advances time), is this transition really advancing time? (it isn’t observable, is it?…)

Anyway, I’ve never benchmarked the code so it was a premature optimization leading to less understandable code. Also, as Jacob and I discussed in the plane home: it actually skews the behavior of the circuit breaker to prefer switching states rather than staying in the same state – something that you may or may-not want.

Finally an off-topic: It is now possible to build the Clojure circuit breaker including AOT compilation of Java interop classes. I ran into a problem with lein where the order of namespaces in :namespaces isn’t preserved. This is a problem when you are gen-interfacing in one file and gen-class :implements that interface in another.

I’ve created a very simple patch to lein to preserve the order.

http://blog.higher-order.net/wp-content/uploads/2010/07/0001-don-t-convert-namespaces-aot-to-a-set-as-it-does-n.patch.txt

Phil is applying this patch so it should be out soon. Until then, if you don’t want to patch lein, there is a step-by-step guide to repl AOT compilation in the README.

Those interested can grab the latest code.

http://github.com/krukow/clojure-circuit-breaker

Posted in General | No Comments »

Is it just me or is this important

June 16, 2010 – 2:22 pm

for all JVM-based languages
http://www.infoq.com/news/2010/06/azul_ori?

Posted in General | No Comments »

clj-ds: Clojure’s persistent data structures for Java

June 11, 2010 – 3:45 pm

One of the appealing features of Clojure is the pervasive use of (efficient!) persistent data structures. (In previous posts I’ve shed some light on how PersistentHashMap and PersistentVector are implemented, although some of that information is slightly dated now).

There are many advantages to programming with persistent data structures (which implies immutability) but that isn’t the topic of this post… Currently the Clojure data structures are implemented in Java, so in principle they should be usable also outside of Clojure, say from Java. However, in practice it is inconvenient (see below).

I’ve created the project clj-ds to make Clojure’s data structures available in a more practical form to other JVM languages than Clojure. The README file from the clj-ds GitHub project explains the motivation:

Advantages of clj-ds when constrained to working with Java (as opposed to just including clojure.jar)

* Currently the Clojure data structures are implemented in Java. In the future,
all of Clojure will be implemented in Clojure itself (known as “Clojure-in-Clojure”).
This has many advantages for Clojure, but when it happens the data structures will
probably be even more intertwined with the rest of the language,
and may be even more inconvenient to use in a Java context.

The clj-ds project will maintain Java versions of the code, and where possible attempt
to “port” improvements made in the Clojure versions back into clj-ds. Thus keeping maintained
versions of the Java data structures.

* In the current Clojure version, calling certain methods on PersistentHashMap requires
loading the entire Clojure runtime, including the bootstrap process. This takes about one second.
This means that the first time one of these methods is called, a Java user will experience a
slight delay (and a memory-usage increase). Further, many of the Clojure runtime
Java classes are not needed when only support for persistent data structures
is wanted (e.g., the compiler).

* The clj-ds library is not dependent on the Clojure runtime nor does it run any
Clojure bootstrap process, e.g., the classes that deal with compilation have been removed.
This results in a smaller library, and the mentioned delay does not occur.

* Clojure is a dynamically typed language. Java is statically typed, and supports
‘generics’ from version 5. A Java user would expect generics support from a Java
data structure library, and the Clojure version doesn’t have this.
clj-ds will support generics.

* Finally, a slight improvement. Certain of the Clojure data structure methods use Clojure’s ‘seq’ abstraction. In the implementation of the Java ‘iterator’ pattern. It is possible, to make
slightly more efficient iterators using a tailor made iterator. clj-ds does this.

Code: http://github.com/krukow/clj-ds

Posted in Clojure, General | 3 Comments »

Stomple RC1: Combining WebSockets and Reliable Messaging

June 1, 2010 – 10:30 pm

I’ve finished Release Candidate 1 for Stomple, my JavaScript Stomp library which uses WebSockets to access compatible Stomp Message Brokers (which includes JMS providers via StompConnect). Stomple is heavily inspired by Jeff Mesnil’s great stomp-websocket, but aims to be feature complete and to provide a high-level API which is more customizable, robust and easy to use.

What? From the Stomp website:

Stomp provides an interoperable wire format so that any of the available Stomp Clients can communicate with any Stomp Message Broker to provide easy and widespread messaging interop among languages, platforms and brokers.

As mentioned, this includes JMS. Stomple is a JavaScript library which enables using browsers that support WebSockets as messaging clients, e.g., subscribing to an JMS topic in a JavaScript program running inside Chrome.

Websocket communication has less overhead than HTTP when HTTP semantics aren’t needed. More importantly, WebSockets allow the server to push updates to the browser (as opposed to the usual HTTP request-response paradigm). For a simple example, see the screen cast of the “transactional chat example” where chat messages are pushed from the server to participating clients.

Why release Stomple when there is already stomp-websocket? First of all, Jeff did an excellent job with stomp-websocket. However, I wanted additional features and more production maturity (e.g., support for timeouts, various low- and high-level callbacks and automatic receipts).

A few notable features:

• Customizability and Callbacks. A key value of Stomple is that it shouldn’t cut off access to any low-level functionality. Hence everything is customizable and accessible to the user code. All core functions support success and failure callbacks. Further, there are low-level callbacks: socketOpen, socketMessage, socketClose, socketError corresponding to the low-level websocket events. These can be used to customize behaviour for special use-cases. At a higher-level, there are callbacks onConnect, connectFailed, onError, onReceipt correspoding to the Stomp events.
• Automatic receipts. If client.autoReceipt is true (which it is by default), then all Stomp messages sent to the server include an automatically generated ‘receipt’ header. In the Stomp protocol, this causes the server to send a receipt for each message it receives, hence reassuring that the message has been received. If not enabled, the client can’t be sure the message is actually received.
• Timeouts. By default all messages sent to the server are associated with a (configurable) 8 second timeout. So for example, suppose you subscribe to a destination, but do not receive a server receipt within 8 seconds: then your failure callback is called with reason ‘timeout’ and information about the message that failed. This way you are guaranteed that either your success or failure callback will always be called (but not both ). These time-outs are pervasive and configurable per API call.
• Transaction management. In Stomp, transactions are started using the ‘BEGIN’ frame, corresponding to a begin function in Stomple. When you call begin, you don’t have to specify a transaction-id: if you don’t, one will be generated automatically for you. Further, any API calls made between a begin and a commit/abort automatically use this transaction identifier, hence grouping the calls in the same transaction (unless, of course, you explicitly provide a different or no transaction id). Further, this nests arbitrarily, and works in a stack-like manner (e.g., begin, begin, send, associates send with inner-most begin).
• Feature complete. Stomple aims to be feature complete. For example, it supports the recommended Stomp ‘content-length’ header, where the current stomp-websocket version does not.
• Correctness. Stomple is tested manually and has a growing automatic test-suite written in YUI3-test. Stomple passes JSLint.

I decided not to build on Jeff’s library because when APIs become more complex, I prefer using configuration-objects since it makes code more readable and easy to maintain (most Stomple functions take a single object literal parameter which names the function’s “many arguments”).
For example to create a Stomple client and to configure it to use destination ‘jms.topic.chat’ as default use:

var client = Stomple.create_client({
    url : "ws://localhost:61614/stomp",
    destination : "jms.topic.chat",
    login : "guest",
    passcode : "guest"
});

A different example of using single-argument configuration objects would be subscribe:

client.subscribe({
    success: function(frame) {//called if subscribe succeeds within timeout-period
        //subscribe succeeded... do something
    },
    failure: function(spec) {//called if subscribe fails or times out
        console.log(spec.reason);
    },
    handler: function(msg) {//called when a message arrived ('this' is scope - see below) 
        this.received = msg.body;//"this" is "someObject"
    },
    scope: someObject
});

Note by the way that if client.autoConnect is true (which it is by default), then you don’t have to explicitly connect. Stomple will automatically connect upon your first action. The subscribe function supports an optional destination parameter in-case a default wasn’t supplied when the client object was created (or in case you want to subscribe to a non-default destination).

How?
For each Stomp client frame (command): SEND, SUBSCRIBE, UNSUBSCRIBE, BEGIN, COMMIT, ABORT, ACK, DISCONNECT there are corresponding lower-case functions on the client object, e.g., send, subscribe… Defaults are specified when creating the client object. For example.

var client = Stomple.create_client({
    url : "ws://localhost:61614/stomp",
    destination : "jms.topic.chat",
    login : "guest",
    passcode : "guest",
    autoConnect: true,//which is actually the default
    timeout: 3000,
    autoReceipt: true//which is actually the default
    //...
});

See Stomple.ClientPrototype for all overrides.

Once a client object is created, the user can issue commands. For example, in the chat example a typical sequence would be.

client.subscribe({
    handler : function(msg) {//update screen on chat msg receiption
	    var c = document.getElementById('content');
	    c.innerHTML = c.innerHTML + '<br>' + msg.body;
	},
	scope : anObjectIfNeeded,
	success : function() {// did subscription succeed?
        //send a test message
        client.send({
            body: "User krukow joined chatroom #stomple",
            success: function() {
              console.log("sent welcome message...");          
            },
            failure: function() {
              console.log("Failed...");
            }
            
        });
	},
	failure : function() {// did subscription fail?
		console.log("sub fail");
	}
});

Things are pretty straightforward from here (if you get Stomp ). For more advanced examples see the automated test-suite. Here is an example of a transaction.

this.client.begin({
    failure: function() {
        Y.Assert.fail("Begin transaction failed.");
    },
    success: function(){ 
		for (i=0;i<N;i+=1) {
			this.client.send({
				body: ""+msgBodies[i],
				failure: sendFailed
                //we don't wait for receipts here... Socket provies order..
			});
		}
        this.client.commit({
            failure: function() {
                Y.Assert.fail("Commit transaction failed.");
            },
            success: function(){//...
            }
        });
    }
});

Posted in General | 4 Comments »

My son

May 24, 2010 – 11:32 am

Posted in General | No Comments »

Stomple: JMS via WebSockets

May 9, 2010 – 10:22 pm

I’ve spent a couple of days working on Stomple (pronounced “stumble”). Stomple aims to be an easy to use, full-featured and robust Stomp client for JavaScript.

You can watch a screencast of the transactional chat example.
Screencast of transactional chat.

What is Stomp? Stomp provides an interoperable wire format so that any of the available Stomp Clients can communicate with any Stomp Message Broker to provide easy and widespread messaging interop among languages, platforms and brokers.

You can use any of the Stomp Clients to work with any JMS compliant message broker, for example HornetQ. There are Stomp clients for many languages, and Stomple adds JavaScript via WebSockets. This means that you can send and receive JMS messages directly in the browser (when it supports WebSockets).

Stomple is inspired by Jeff Mesnil’s cool stomp-websocket, but I wanted more customization, feature-detection, a nicer API, and more robustness (e.g. timeouts) (and I wanted to write it myself ). Just like Jeff’s client, Stomple is not directly a Stomp client since it runs over WebSockets which requires a handshake between the browser’s client and the server. This is supported by HornetQ and ActiveMQ (details in Jeff’s docs: http://www.jmesnil.net/stomp-websocket/doc/).

This library is not a pure Stomp client. It is aimed to run on the Web Sockets protocol which is not TCP. Basically, the Web Sockets protocol requires a handshake between the browser client and the server to ensure the browser’s “same-origin” security model remains in effect.

This means that this library can not connect to regular Stomp brokers since they would not understand the handshake initiated by the Web Socket which is not part of the Stomp protocol and would likely reject the connection.

I’ll post more documentation soon, but stomple should be quite simple to use.

if (Stomple) {//feature detection: is stomple available?
    Stomple.debug = true;//enable debug logging of messages
    var client = Stomple.create_client({
        url: "ws://localhost:61614/stomp",
        destination: "jms.topic.chat",
        login: "guest",
        passcode: "guest"
    });
    client.subscribe({
        handler: function(f) {
            console.log('received frame');
            console.log(f);
            console.log("Message: "+f.body);
            console.log("'this' is bound to 'scope'");
            console.log("Session:"+this.session);
            client.send({
                success: function() {},//handle successful send
                failure: function() {},//handle failed send
                timeout: 5000//wait max 5 seconds before failing
            });
        },
        scope: client,
        success: function() {//did subscription succeed?
            console.log("sub ok..");
        },
        failure: function() {//did subscription fail?
            console.log("sub fail");
        }
    });
} 

Check it out on Github: http://github.com/krukow/stomple.

More info to come

Posted in General | No Comments »

Circuit Breaker: a small but real-life example of Clojure protocols and datatype

May 5, 2010 – 10:59 am

(Update July 2nd 2010:
I’ve cleaned up the code and git repo.
Inlined the protocol function definitions in the state records for native-platform speed.
The policy can now specify which exceptions should be considered errors: this is really useful in real life when you don’t want to trip the circuit breaker say on security exceptions.
Now builds with lein 1.1.0.
…]$ lein jar
…
…]$ javac -cp lib/clojure-1.2.0-master-20100623.220259-87.jar:circuit-breaker.jar src/C.java
…]$ java -cp lib/clojure-1.2.0-master-20100623.220259-87.jar:circuit-breaker.jar:src C
)

Michael Nygards stability pattern “Circuit Breaker” is useful for failing fast when calling integration points that are unstable (which is every integration point I’ve ever dealt with!). This is done by detecting when integration points fail, and subsequently cutting off access for a time-period. Use the circuit breaker to

… preserve request handling threads in the calling system. Very often, when you make a call to an external integration point that’s broken, it will tie up a thread in a blocking synchronous call for an indefinite period of time. [Michael Nygard, QCon interview]

I’ve written a fast, non-blocking functional implementation of the Circuit Breaker in the 1.2 branch of Clojure which will be released shortly. It uses the new Clojure constructs protocols and datatypes for modeling states, for interop and to obtain platform-speed polymorphic calls.

The implementation exposes a simple Java interface which makes it usable from Java, Scala, JRuby et al.

Why?

Design Patterns are a disease, and Clojure is the cure. (the smiley is mine!)
[http://www.nofluffjuststuff.com/blog/stuart_halloway/2009/10/the_case_for_clojure]

I believe this implementation has a couple of advantages compared to these Java and Scala implementations that use the GoF “State Machine” pattern.

First, I find the functional version simpler (see examples below). Second, this version guarantees that only a single call is made to the integration point when the circuit breaker decides to re-test if it is working again (the other versions seem to allow an unbounded number of calls if more threads concurrently try to access the integration point calling “invoke”). Finally, this version encapsulates the (immutable) state in a single Clojure atom (corresponding to a Java AtomicReference), whereas the GoF implementations use at least three atomics for various counters. Why does that matter? Well, it gives you the ability to obtain a consistent (immutable) snapshot of the state of circuit breaker at any given time which can be used to e.g. logging and analysis – this isn’t possible when you have several atomics in play.

These benefits come naturally from following Clojure’s programming model and concurrency constructs. Let me illustrate the Clojure features that I’ve found useful for this problem.

Protocols and records.
I’m using pure polymorphic functions on-success, on-error, on-before-call as transition functions mapping a state to the next state for an event (successfull call, error call and before a call is initiated). A pure function proceed is a predicate on states that decide whether or not the state allows calls to go though to the integration point.

Together these functions form a Clojure protocol (which is similar to a Java interface, but has additional benefits).

(Show with JavaScript: for non-JS User agents, see http://gist.github.com/390485)

Apart from the definition of the protocol, we define a default implementation of the protocol functions that our states can use. The default transition functions are simply the identity function and proceed defaults to false.

We now define datatypes corresponding to each type of state: closed (calls go through, count failures), open (calls fail-fast, stores a time-stamp when IP failed), initial-half-open (a single call goes through), pending-half-open (waiting for a probing call to return). The datatypes are parameterized by a “transition policy” defining how many failures are “needed” to transition to the open state, and how long to wait in the open state.

(For non-JS User agents, see http://gist.github.com/390492 )

This simply defines the states as datatypes. Note that we use defrecord not deftype. This makes our datatypes work like persistent clojure maps which is extremely useful – for example our states can be destructured (see, e.g. defrecord at http://clojure.org/datatypes).

We can make our new types participate in our CircuitBreakerTransitions protocol

(http://gist.github.com/390494)

A couple of notes:

- We use merge to take the default implementations and “override” with the implementations given (this would correspond to an abstract super-class in Java but is more flexible).
- We use destructing in the function definitions for easy access to the “ClosedState” data, e.g., in the body of (fn [{f :fail-count p :policy, :as s}] ... the states fail-count is available as f and similarly for the policy. The :as s clause makes the state itself available as s.
- Finally, we can construct new instances of the states since they are simple dynamically compiled classes, e.g., (ClosedState. p 0) creates the initial state with a policy p.

Pure functions. A clear advantage of the functional approach is the ease of testing. Consider this simple test of some states and transition functions.

(http://gist.github.com/390505)

This is the core of the circuit breaker itself:

(http://gist.github.com/390529)

A circuit breaker is simply an atomic reference to a state. The function wrap-with takes a function, f to wrap – this represents a function that will call an integration point, and a circuit breaker, named state. It then returns a “wrapped” function which is guarded by the circuit breaker. It uses the “transition-by!” function which makes a state-transition from the current state.

An example usage:

(http://gist.github.com/390535)

Notice that the snapshot of the state is available simply with a deref, e.g., as @cb.

A Java-interface

To expose the functionality to Java I have used Clojures gen-class to create a class that exposes two methods given by this protocol which lets you wrap a function and look at the state:


(defprotocol CircuitBreaker
(#^clojure.lang.IFn wrap [this #^clojure.lang.IFn fn])
(#^net.higher_order.integration.circuit_breaker.states.CircuitBreakerTransitions current-state [this]))


Then using gen-class I generate a class that implements the interface corresponding to that protocol. This gives the possibility of using the circuit breaker from Java:

import clojure.lang.IFn;
import clojure.lang.RT;
import net.higher_order.integration.circuit_breaker.AtomicCircuitBreaker;
import net.higher_order.integration.circuit_breaker.CircuitBreaker;

public class C {
	public static void main(String[] args) {
		CircuitBreaker atomicCircuitBreaker = new AtomicCircuitBreaker();
		IFn wrap = (IFn) atomicCircuitBreaker.wrap(new clojure.lang.AFn() {
			public Object invoke(Object arg0) throws Exception {
				if (arg0 == null) throw new IllegalArgumentException("null arg");
				System.out.println("Invoked with: "+arg0);
				return arg0;
			}
		});
		succeed(atomicCircuitBreaker, wrap);
		fail(atomicCircuitBreaker, wrap);
		fail(atomicCircuitBreaker, wrap);
		fail(atomicCircuitBreaker, wrap);
		fail(atomicCircuitBreaker, wrap);
		fail(atomicCircuitBreaker, wrap);
		fail(atomicCircuitBreaker, wrap);
		sleep(1000);
		status(atomicCircuitBreaker);
		fail(atomicCircuitBreaker, wrap);
		sleep(5000);
		succeed(atomicCircuitBreaker, wrap);
	}

	
	private static void sleep(long howlong) {
		try {
			Thread.sleep(howlong);
		} catch (InterruptedException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
	}

	private static void succeed(CircuitBreaker atomicCircuitBreaker, IFn wrap) {
		try {
			System.out.println(wrap.invoke("KARL"));
			System.out.println(wrap.invoke(42));
		} catch (Exception e) {
			System.out.println(e.getMessage());
		} finally {
			status(atomicCircuitBreaker);
		}
	}

	private static void status(CircuitBreaker atomicCircuitBreaker) {
		System.out.println(RT.printString(atomicCircuitBreaker.current_state()));
	}

	private static void fail(CircuitBreaker atomicCircuitBreaker, IFn wrap) {
		try {
			System.out.println(wrap.invoke(null));
			System.out.println(wrap.invoke(42));
		} catch (Exception e) {
			System.out.println(e.getMessage());
		} finally {
			status(atomicCircuitBreaker);
		}
	}
}

This is getting long. I’ll save the comparison for the next post

Github: http://github.com/krukow/clojure-circuit-breaker

Posted in Clojure | 3 Comments »

Clojure: Rich Hickey and Stuart Halloway at JAOO

March 25, 2010 – 8:44 pm

See Danish Clojure Users Group posting.

Posted in General | No Comments »

VersionManager

January 22, 2010 – 5:10 pm

[update: First, my apologies to jdalton, my post was not meant to derail Fusebox, merely to show a different approach to similar problems. ]

I read a post on Ajaxian about Fusebox, a JavaScript library which is described as:

[...] The problem is that frameworks / libraries / third-party scripts may overwrite native methods or each other’s custom methods resulting in unpredictable outcomes. Fusebox, a limited version of the sandboxing component found in FuseJS, avoids these issues by creating sandboxed natives which can be extended without affecting the document natives.

I think Fusebox is a good project. The code in this blog post shows a different approach which in some situations may be more useful and in others, it may not.

With Fusebox script developers avoid polluting the globals by writing their scripts using FuseBox and its wrappers. E.g, you write code with a “Fusebox prefix” like this:

  var fb = Fusebox();
  fb.Array.prototype.hai = function() {
    return fb.String("Oh hai, we have " + this.length + " items.");
  };
  fb.Array(1,2,3).hai(); // "Oh hai, we have 3 items."
  typeof window.Array.prototype.hai; // undefined

// like the native Array constructor the sandboxed constructor will return [ , , ]
  var a = fb.Array(3);
  // equiv to square-bracket notation [3]
  var b = fb.Array.create(3);
  // converting a native array to a sandboxed array
  var c = fb.Array.fromArray([1, 2, 3]);

That immediately reminded be of some work I’ve have been doing with colleague Jimmy Junker at Trifork (jju at trifork com) motivated by the following problem:

In portal environments where multiple portlets want to use different JavaScript libraries or different versions of the same JavaScript libraries, you are very likely to run in to problems since almost all libraries (except later YUI versions) are designed to live in the global namespace.

After a brainstorming session we developed a simple prototype (no pun intended), uninspiringly named “VersionManager”, with which we succeeded in loading two different versions of the PrototypeJS library. With VersionManager you can write code like this:

with (VersionManager.version("1.6.1")) {
//"1.6.1" refers to prototype which must have been loaded
  console.log("abcdaba".gsub("a","42"));// console.logs '42bcd42b42'
  console.log(typeof Object.extend);// console.logs 'function'
  $A([1,2,3])._each(function(x){
    console.log(x);
  });// console.logs '1','2','3'
}
VersionManager.clear();//otherwise previous version is lingering...
try {
  console.log(typeof "".gsub);
//console.logs 'function' <= this is generic delegating proxy function
  console.log("abcdaba".gsub("a","42"));//throws error
} catch (e) {
  console.log(e);
  //TypeError: "attempt call to delegate with no version defined (see documentation)"
}
console.log(typeof Object.extend);// console.logs 'undefined'
console.log(typeof $A);// console.logs 'undefined'

So version manager lets you load several libraries on the page, and sandboxes changes made to globals. By telling VersionManager which version you want to use for a particular block you code, you can use each of those libraries, even if they define the same global variables or properties on the prototypes of objects, e.g., Array.prototype.

VersionManager also creates sandboxes, but as opposed to Fusebox, what is sandboxed is "changes made to" the built-ins, and global variables defined. Also, the goal is not to avoid extending the prototypes, but almost the opposite: to allow several scripts/libraries that would otherwise conflict to live in the same page without seeing each other, even if they define the same global names or want to extend built-in prototypes in different ways.

With VersionManager you simply write regular user code, but wrap it in a with statement. This means that existing code can easily be rewritten.

The loaded "sandboxed" libraries must adhere to a few syntactic and semantic constraints.
For an example, see a sandboxed version of prototypejs here

Transformed prototypejs

The transformation applied to prototypejs is very simple and can be done automatically (i.e., it is easy to write a program that performs this script-transformation for you). It is a bit technical, but it results in is adding a declaration with-statement "header" at the beginning and a "footer" and the end. In fact, we are writing a program that transforms any JavaScript program/library into an equivalent one that adheres to these constraints.

An alpha version is on github. It is a proof-of-concept that shows that the fundamental approach seems to work.

jdalton pointed out a couple of things :

• You may run into problems with minifiers given that your code is wrapped in 'with'. However, you can minify the code first, and then wrap it.
• It doesn't track changes to event or elements.

So far, we've tested it only in latest Firefox and Safari, but I see no reason why it couldn't work in other recent browsers. We have sucessfully loaded prototypejs in a sandbox, and managed to run all prototypejs unit tests successfully inside the sandbox.

Github project: http://github.com/krukow/versionmanager

/Karl

Posted in JavaScript, namespacing, with | No Comments »

The Joy of Clojure

January 14, 2010 – 12:54 pm

In case you haven’t noticed there is a very interesting Clojure book coming out, titled “The Joy of Clojure,” written by two very interesting authors that anyone hanging out in the Clojure community should know: Chris Houser and Michael Fogus.

As an appetizer, the first chapter is available for free:

Clojure—A Lisp for the Java Virtual Machine

I’ve read the first chapter and the book looks very promising! To quote the last paragraph of chapter one:

We’ve talked a little about how this book will go beyond what Clojure is to why it’s designed the way it is and how that design can be exploited through idioms that will help you think in Clojure. So lets stop talking about what this book will do and get on with the doing.
Fasten your seat belts.

I’ve fastened my seat belt and ordered my copy

Posted in Clojure, General | 3 Comments »