Decorators - The Python Saga - Part 2

Python introduced a short-hand for the adapter pattern on functions. You can "decorate" a function with another function. This is a neat tool you can use to factor out some common code from a bunch of functions. You can fiddle with the arguments, return values, or intercept exceptions thrown by any function you decorate.

The canonical example is a memoize decorator. The idea is to generalize the notion of memoization so you can simply subscribe to it in any function you want to memoize.

def factorial(n):
	if n == 1: return 1
	return n * factorial(n - 1)
factorial = memoize(factorial)

You accomplish this by writing the memoize decorator. A decorator is a function that accepts a function and returns another. Python virtuously provides a shorthand for taking the function, decorating it, and assigning it to a variable with the same name.

@memoize
def factorial(n):
	if n == 1: return 1
	return n * factorial(n - 1)

In the imagined normal case of decorators, the returned function accepts the same arguments and returns the same kinds of values as the accepted function. However, a decorator does have the liberty of extending or restricting that interface, like accepting additional arguments or raising an exception if the arguments are of the wrong type. It might also perform some common computation on the original arguments and pass the result to the original function as an additional argument. In any case, you can use some closures to create a decorator:

def memoize(function):
	cache = {}
	def decorated(*args):
		if args not in cache:
			cache[args] = function(*args)
		return cache[args]
	return decorated

Of course, that's too simple. A lot of things you put after the "@" symbol are just functions that return decorators so that they can be configured with arguments. For example, you probably want to make a memoize decorator that lets you specify your own cache object. So, you need another layer of deference.

def memoize(cache = None):
	if cache is None: cache = {}
	def decorator(function):
		def decorated(*args):
			if args not in cache:
				cache[args] = function(*args)
			return cache[args]
		return decorated
	return decorator

@memoize({})
def factorial(n):
	if n == 1: return 1
	return n * factorial(n - 1)

Since, in Python, functions, objects, and types are indistinguishable to the casual observer, you can do the exact same thing with a class, although I shudder to think that you might want to forgo the simplicity and elegance of closures. After the transform, the previous code might look like this:

class memoize(object):
	def __init__(self, cache = None):
		self.cache = cache
	def __call__(self, function):
		return Memoized(function, self.cache)

class Memoized(object):
	def __init__(self, function, cache = None):
		if cache is None: cache = {}
		self.function = function
		self.cache = cache
	def __call__(self, *args):
		if args not in self.cache:
			self.cache[args] = self.function(*args)
		return self.cache[args]

@memoize()
def factorial(n):
	if n == 1: return 1
	return n * factorial(n - 1)

So now you can use a Least Recently Used Cache, assuming it is a dictionary-like-object (a duck-dict, if you will):

from lru_cache import LruCache
@memoize(LruCache(max_size = 100, cull = .25))
def factorial(n):
	if n == 1: return 1
	return n * factorial(n - 1)

Download decorators.zip.

Variadic Positional and Keyword Arguments - The Python Saga - Part 1

Python supports "variadic" arguments. Variadic arguments are the man behind the curtain for C's printf function. The idea is that a function can accept a variable number of positional arguments, the values to put in your format string. In C this is accomplished with an ellipsis, ..., and some VA macro-linked-list-stuff that I always have to look up. Python goes a couple steps further with variadic arguments and the results are stunning, orthogonal, and actually useful almost every day. With Python, you get both "positional" arguments, like C, and keyword arguments: those arguments that conceptually map, in any order, to the names of the arguments in your function's declaration. The magic symbols are "*" and "**" for positional and keyword arguments respectively. With one "*", you can declare a function that accepts any number of arguments as the declared list object:

def foo(*args):
	return args
assert foo(1, 2, 3) == [1, 2, 3]

You can also pass an array of positional arguments to a function with very similar syntax:

def foo(a, b, c):
	print [a, b, c]
assert foo(*[1, 2, 3]) == [1, 2, 3]

And you can do the same thing with keyword arguments except you use dictionaries:

def foo(**kwargs):
	return kwargs
assert foo(a = 10, b = 20, c = 30) == {'a': 10, 'b': 20, 'c': 30}

Likewise, you can pass keyword arguments:

def foo(a, b, c):
	print [a, b, c]
assert foo(**{'a': 10, 'b': 20, 'c': 30}) == [10, 20, 30]

You can use them in combination, along with default arguments to provide beautiful, orthogonal, reusable abstractions:

def foo(a, b = None, c = None, d = None):
	return [a, b, c, d]
assert foo(*[1, 2], **{'c': 3}) == [1, 2, 3, None]
def bar(a, b, c, *args, **kws):
	return [a, b, c], args, kws
assert bar(1, 2, 3, 4, 5, f = 6) == ([1, 2, 3], [4], {'f': 5})

It Shtarts with a Bloody Esh.

The light saber application for the iPhone was recently "upgraded" to a naggy advertisement application for a game that I will not give the benefit of calling out by name. If, like me, you need to "downgrade" your iPhone application, fear not, there is a way.

If you upgraded the application directly on the phone, the old version may be backed up on your main iTunes library. If you have docked since you got the application originally, and have not docked since you upgraded, just delete the application from your phone then dock and sync. The old version will be restored.

If you have already docked and synced your iPhone with your iTunes library, it will appear that the new application is both on your phone and your computer with no sign of the old one. Check the trash—it might still be there. Copy the .ipa (iPone App) file for your application to your iTunes/Mobile Applications directory and open it in iTunes. It will prompt you for whether you would like to replace the newer version. Just say, "YES!".

JavaScript Module Standard

The purpose of this document is to propose a contract between a collection of JavaScript module loaders and modules. The specification describes the environment that module loader implementations provide, and the environment that modules may depend upon. In particular, compliant module systems:

• map one file to one module (while leaving room for implementation-specific multi-module bundling for website performance),
• cache singleton module objects before executing the corresponding module file,
• execute modules with a feature-rich context,
• resolve module URLs relative to a module path, and
• conform to a domain-name-based site-package naming convention and leave a name-space for a centrally managed standard library.

The specification is intended to be suitable for client- and server-side JavaScript module loader implementations.

The specification is intended to provide insights and an easy migration path to future versions of JavaScript.

The specification is intended to narrow the domain in which JavaScript modules can universally depend to maximize portability.

The specification encourages modules to adhere to a strict subset of the JavaScript environment in which they may be loaded. In spirit, this is a theoretical version of the JavaScript language that provides the intersection of behaviors provided by Class-A browsers and server-side run-times including Rhino, plus this system for loading modules.

Module Execution Context

The singleton module object MUST be declared and cached BEFORE the corresponding module file is executed. The module file MUST only be executed ONCE for the duration of a page-view or JavaScript program.

In a module file's execution context, the context object, represented by this, MUST be the module object.

The scope chain, from global to local, of a module file's execution context MUST consist of:

• builtins
• moduleScope
• module
• <anonymous>

builtins

Rules for module systems:

• The builtins object MAY be frozen.
• All objects in the transitive closure of builtins on item selection MAY be frozen.
• The builtins object MAY contain more values than specified.
• The builtins object MUST include:
• String
• Number
• Boolean
• Object
• Array
• Date
• RegExp
• Error
• EvalError
• RangeError
• ReferenceError
• SyntaxError
• TypeError
• The module loader MAY enforce these invariants at any time. In some environments, verifying these invariants will not be possible or pragmatic.
• All objects in builtins MUST conform to the JavaScript subset described in the introduction: one that consists of the intersection of behaviors of the respective objects in all Grade-A browsers and server-side JavaScript environments.

Rules for modules:

• Modules MUST NOT write items to the builtins object.
• Modules MUST NOT modify any object in the transitive closure through references on the builtins object.
• Modules MUST NOT access any items in the builtins object not herein specified.
• Modules MUST NOT use non-standard features provided by builtins.

moduleScope

The moduleScope is a module's private name-space for module loader functions and imported values.

The moduleScope MUST provide:

• builtins
• module
• moduleUrl
• moduleRootUrl
• require
• include
• foreignModuleBind
• log

The moduleScope MAY provide:

• register
• publish

Modules MAY augment the moduleScope with additional items.

Modules MUST NOT overwrite the items specified here.

Module Loading

require(<moduleUrl>, [<structure>])

The require function returns an object with items from a foreign module. The required module is referenced with a URL. By default, all items from the foreign module are copied into the returned object. The returned object MAY be frozen. Modules MUST NOT modify the returned object. If a structure is provided, a subset of the items from the foreign module will be returned, the result of destructure(<module>, <structure>). If a function in the foreign module was declared with the foreignModuleBind decorator, the corresponding item in the returned object is the result of <foreignModule>.moduleScope.moduleBind(<name>, <value>).

If the URL begins with a dot, ("."), the fully qualified URL for the requested module is resolved relative to the fully qualified URL of the current module. This would be the result of urlJoin(moduleRootUrl, moduleUrl, foreignModuleUrl). Otherwise the fully qualified URL is urlJoin(moduleRootUrl, foreignModuleUrl).

Regarding module file names:

• Directory components and file names in modules MUST be in camelCase.
• Modules MUST have a ".js" extension if they are provided by files.
• Modules MUST not have an extension if they are provided by the module loader but are not backed by real files. This might include a "window" module in a particular browser implementation.
• The module root is reserved for a cross-browser JavaScript standard library.
• Modules provided by entities other than the standard library MUST exist in a subdirectory of the module root corresponding to a domain name controlled by the author, or a subdirectory thereof.

Module authors are encouraged to use module relative URLs to increase the mobility of entire directory trees of modules.

include(<moduleUrl>, [<structure>])

The include function defers to require, both its arguments and its return value. However, include also copies all of the items from the object returned by require to the moduleScope object.

foreignModuleBind(<function>)

A function decorator that denotes that the module loader guarantees that, when the decorated function is called, it will receive the module object for the module file in which it was called as its context object: this. foreignModuleBind MAY return the same Function it was passed. The returned function MUST be usable in the module in which it was declared as if it were the function passed to foreingModuleBind. foreignModuleBind MAY modify properties of the given Function.

For example:

this.foo = foreignModuleBind(function () {
	log("foo called from: " + this.moduleScope.moduleUrl);
});

destructure(<object>, <structure>)

For the purpose of this specification, the "destructure" function has the following semantics. If the structure is an Array, the returned Object contains the items from the given <object> corresponding to the keys provided in the given Array structure. If the structure is an Object, the returned object contains items where each key corresponds to a value in the structure, and the value is a value from the object corresponding to the key in the structure. For example:

• destructure({"a": 10, "b": 20}, ["a"]) == {"a": 10}
• destructure({"a": 10, "b": 20}, {"a": "A"}) == {"A": 10}

module

The module object is a module's public interface. Adding items to the module object makes them available for export to other module scopes. To that end, the module object is mutable. The module object MUST provide a moduleScope. Modules MAY augment the module object.

moduleRootUrl

moduleRootUrl is the fully qualified URL of the JavaScript site-packages directory: the module root.

moduleUrl

moduleUrl is the relative URL of the current module file relative to moduleRootUrl.

log(<message>, [<label>])

log is a simple console logging function.

The module loader MAY ignore calls to log. The module loader MAY ignore the label argument.

The optional label MAY be any string, and MUST be suitable for use as a CSS class-name (preferably lower-case delimited by hyphens) of which the following MAY be significant:

• info
• warn
• error
• module
• help
• pass
• fail

Afterword: Browser Implementations

This specification outlines the process of requiring modules from within other modules. However, in a browser's global context, JavaScript execution blocks are not modules. To that end, this specification does not require that the module loader be invoked in any particular fashion. A particular implementation might hook an initial module to be loaded from within a script tag. Another implementation might scan the DOM for script tags with an alternate language attribute and execute them as modules with the current page's URL as their module URL.

Afterword: Future ECMAScript import semantics.

A future version of the ECMAScript standard might specify new syntax and semantics for importing modules. Current discussions about this feature trend toward having new syntax that "desugars" to native JavaScript. To that end, I propose the following syntax and desugaring transformations in the context of this specification:

• import "<moduleUrl>" as <moduleName>;
• module.<moduleName> = require("<moduleUrl>");

• from "<moduleUrl>" import *;
• include("<moduleUrl>");

• from "<moduleUrl>" import <a>;
• include("<moduleUrl>", ["<a>"]);

• from "<moduleUrl>" import <a> as <a'>;
• include("<moduleUrl>", {"<a>": "<a'>"});

Tale Update

I'm not exactly on schedule for Tale (I intended to have a playable alpha this month), but here's your consolation prize: art updates.

I recently added Fire Drakes and ducks to the Tale managerie. You'll encounter varous breeds of Fire Drake Elementals and Alloyals throughout the eastern face of the world, Oria. The brave and, more importantly, resilient will eventually encounter FrancisDrake, the drake leader.

Additionally, those daring to travel south to Austra will eventually encounter first patrols, squadrons, platoons, companies, battalions, regiments, brigades, divisions, corps, and eventually hordes of penguins. The penguin collective recently added an air-force to their numbers with propellor-beanie sporting aerial grenadier penguins, Aero-pengins, and ballistic-missile ground forces, Rocket-penguins.

The penguins appear to be gearing up for war with Borea, however, the red mage of the North will not be caught with his snow-pants down. The elves have recruited the not-so-friendly poler-bears to introduce penguin to their rich baby-seal diet. The narwhal claims neutrality and you're not likely to find him unless you dive in the frigid Boreal Sea.

Plans are under-way to populate all of the evil faces of the world with popular open-source mascots. You can expect to see the BSD Daemon in the hell-world of Dysia, and maybe even Fire-foxen in Oria. Drop me a line if you think of an appropriate mascot for the /East, perhaps involving fire or ducks, especially if it continues the theme of rougue operating systems.

On a completely different topic, there is new art for Tale accessories including a morning-star, the brick-of-power, and the iTome for manic fighters. These items will be available on the ever-popular face 5, Occia.

That gives me a good an idea. Perhaps there should be an operating system for every face! Faces 6, 5, and 4 (good) would be commercial operating systems while faces 1, 2, 3 (evil) would be open-source. Face 6 will obviously be blighted with drafts; does anybody have an idea for Borea, the north pole?

Also, there's a new item for your initial inventory when you start the game in Euia, a spork.

Got Style?

There's a new JavaScript library from Google on the block. So, I'm of course out to absorb the good bits for Chiron. I've got to wonder some times whether people have started to use ideas from my code.

Compare this bit from Google's JavaScript library with this one in Chiron.

I don't hear much about people using Chiron. Could this be a sign that people have been reading it and absorbing the good bits? There are other parallels. For example, some of the function names and file names match up with Chiron too, like base.js and inherits (instead of Doug Crockford's recommended term, begets).

Re: Recommendations for the Fourth Edition of the ECMAScript Programming Language Specification

Doug Crockford has posted some recommendations to improve JavaScript.

I have some comments in the context of my experience with live-patching JavaScript issues with libraries.

New Features

Function Reflection

Doug recommends that function objects contain a "name" property. Some versions of JavaScript actually have this already. This could be handy but I haven't needed this and would not be likely to use it. I was surprised that FireFox had this when I used a Function in a with block. I've learned my lesson.

However, Doug also recommends the addition of an arguments member to Function objects. This is a critical feature. The arguments array would contain the names of all of the functions declared argument variables. Function objects already have a length member that corresponds to the length of this hypothetical arguments array. There was some buzz about using this value for method overloading and it permitted me to create an elegant partial application decorator. If we had access to the names of a function's argument declarations, I could implement a decorator for supporting Python-style positional and named arguments. Decorated functions would accept an array/list of positional arguments and an object/dict/map of keyword arguments and the decorator would translate them to an array for a wrapped Function.apply. That decorator or further decorators could support default values and automatic argument length assertions.

typeOf

Doug notes that JavaScript's typeof "function" is broken because it returns the wrong strings for null and arrays. All that and he hints that using camelCase instead of lowercase would have been more in keeping with the language's aesthetic. He recommends that we deprecate typeof and replace it with a fixed version called typeOf. I would go a step further. typeof returns a string. I propose that it ought to return the type object. This would jive well with instanceof, instanceOf, or isInstance. This means there should be a Null type that would construct null and an Undefined type to construct undefined.

Object Methods

Don't Enum

I don' particularly care whether Object has a dontEnum method since I don't augment base types and wrap them when I need reliable collection types, but this would help ameliorate integration woes with prototype so it's a good idea.

Array of Keys

Doug wants a Object.arrayOfKeys method. I want a keys method that returns a Set. I would also like keysIter and, less emphatically, keysArray. I don't want the name arrayOfKeys because it introduces an unnecessary Of and doesn't sort as well as keys*.

Array of Values.

Doug also wants Object.arrayOfValues. I want values and valuesIter. These should return Array and Iter object respectively.

toJsonString

Object.toJSONString(memberName). The first thing that comes to mind to improve this idea is to rename it toJsonString; I prefer to consider acronyms words and I would have also called XMLHttpRequest XmlHttpRequest. This name jives well with toString, but I would have called that string. But, I actually would call this repr and made it a global function. repr would work on arbitrary objects, a subset of which would be serializable JSON. The behavior of repr would be overridable by providing a repr method on an object.

Beget Object

Doug recommends that Prototype inheritance be a little more lucid by providing a beget function that returns a new, empty Object that inherits from the previous. I like this idea. I called it inherit in Chiron.

isEmpty

isEmpty would return whether an object has any properties, other than those inherited. Sounds fine to me, although I probably would have just used number and boolean casting. This is a good idea too though, because in JavaScript, it's hard to distinguish an Object that is being used as the base of the inheritance tree and an Object that is being used as a hash-map. To that end, I recommend the addition of a Map type. Map would have isEmpty, number, and boolean functions and would implicitly cast to Number and Boolean based on their length that would be reflected publically by the getLength function.

Array Methods

Doug recommends the addition of some of Mozilla's JavaScript 1.6 Array methods. I support all of them. However, some nomenclature refinements and specifications:

indexOf should be augmented by a find function that throws a KeyError instead of returning a negative index.

lastIndexOf should be augmented by a findLast function similar to find.

every should be called all. This function should accept an iterator or iterable and short circuit on the first failure.

filter should be called where. Filtering implies the opposite of finding. To filter something is to remove it from a stream if it passes a particular condition. where implies that the outgoing stream should contain only those elements from the original stream that affirm the guard.

forEach is good as is. I would like to specify that it should return this so that forEach calls can be chained. This has implications on iterations that are partially consumed by a forEach call that throws a StopIteration exception in its continuation.

map is good as is. As a member of an array, each should be a synonym. These functions should also be declared in global scope with opposite argument order: map(function, collection, [context]) vs each(collection, function, [context]).

some should be called any. This function should accept an iterator or iterable and short circuit on the first success.

String Methods

trim is good as is. There should also be trimBegin and trimEnd or trimFirst and trimLast.

string.eval is a good idea, but I've got my own ways to launder the scope chain for eval.

Date Methods

Date.toJSONString and Date.toISOString are both great ideas; not having them has been a problem for me in the past. I recommend calling them toJsonString and toIsoString.

Corrections

Reserved Words

Doug argues that identifiers in object literal notation and member dot notation should allow reserved words. I concur. It's a small syntax shortcut that identifiers used in member selection and object literal notation are not required to be enquoted; the language should be equally permissive for both syntax forms.

Doug also argues that the list of reserved words is too long. I have mixed feelings. On one hand, having a long list of reserved identifiers leaves open the door for future advances in the language (some of which include type annotations, which I find dubious). On the other hand, they muddy the name space for current code. On the latter note, it's important that all browsers are equally strict. Safari, at the moment, is much more strict than other browsers, so I've been surprised.

Object Literal Notation

I agree that commas should be more regular throughout the language. They should be permitted after any value, including the last in an Object or Array literal without affecting the length of either.

arguments

arguments should definitely be an instance of an Arguments type that inherits from Array.

Inner Functions and the Context Object (this)

I agree that this should not be the global object (window) in inner or anonymous functions. this should be acquired from the scope chain in such closures. Doug claims that this is not the standard. I recall being corrected on this point, but I also recall having been under the same impression. I leave this as an exercise to the reader to determine the current state of affairs in various browsers.

Tail Recursion

Tail recursion would be nice.

Deprecation

Primitive wrappers should be eliminated. Boxing is almost never necessary. In fact, the use of the new keyword could be completely obviated and code would become much more reusable since there would be no distinction between a factory method and an object constructor.

I actually use the with statement in a couple cases that are really important. I don't recommend using it for its designed purpose, but I really do need it to stay in the language for my module system and my JavaScript shell to keep working. FireBug, in particular, depends on this feature.

Semicolon insertion was silly. Off with its head.

I don't particularly care about arguments.callee.

typeof has to go.

Again, eval is a necessary part of my perverse world. I can do my own laundry. There are, however, irresponsible uses of eval that I do not condone.