CommandLoop module in python

http://littlelanguages.com/web/software/python/modules/cmdloop.py

Here's a python module which provides a base class for writing simple interactive command loop environments. It makes use of python 2.4 decorators to describe the command functions.

CommandLoop provides a base class for writing simple interactive user environments. It is designed around sub-classing, has a simple command parser, and is trivial to initialize.

Here is a trivial little environment written using CommandLoop:

import cmdloop

class Hello(cmdloop.CommandLoop):
  PS1='hello>'

  @cmdloop.aliases('hello', 'hi', 'hola')
  @cmdloop.shorthelp('say hello')
  @cmdloop.usage('hello TARGET')
  def helloCmd(self, flags, args):
    '''
    Say hello to TARGET, which defaults to 'world'
    '''
    if flags or len(args) != 1:
      raise cmdloop.InvalidArguments
    print >> self.OUT, 'Hello %s!' % args[0]

  @cmdloop.aliases('quit')
  def quitCmd(self, flags, args):
    '''
    Quit the environment.
    '''
    raise cmdloop.HaltLoop

Hello().runLoop()

Here's a more complex example:

import cmdloop

class HelloGoodbye(cmdloop.CommandLoop):
  PS1='hello>'

  def __init__(self, default_target = 'world'):
    self.default_target = default_target
    self.target_list = []

  @cmdloop.aliases('hello', 'hi', 'hola')
  @cmdloop.shorthelp('say hello')
  @cmdloop.usage('hello [TARGET]')
  def helloCmd(self, flags, args):
    '''
    Say hello to TARGET, which defaults to 'world'
    '''
    if flags or len(args) > 1:
      raise cmdloop.InvalidArguments
    if args:
      target = args[0]
    else:
      target = self.default_target
    if target not in self.target_list:
      self.target_list.append(target)
    print >> self.OUT, 'Hello %s!' % target

  @cmdloop.aliases('goodbye')
  @cmdloop.shorthelp('say goodbye')
  @cmdloop.usage('goodbye TARGET')
  def goodbyeCmd(self, flags, args):
    '''
    Say goodbye to TARGET.
    '''
    if flags or len(args) != 1:
      raise cmdloop.InvalidArguments
    target = args[0]
    if target in self.target_list:
      print 'Goodbye %s!' % target
      self.target_list.remove(target)
    else:
      print >> self.OUT, \ 
        "I haven't said hello to %s." % target

  @cmdloop.aliases('quit')
  def quitCmd(self, flags, args):
    '''
    Quit the environment.
    '''
    raise cmdloop.HaltLoop

  def _onLoopExit(self):
    if len(self.target_list):
      self.pushCommands(('quit',))
      for target in self.target_list:
        self.pushCommands(('goodbye', target))
    else:
      raise cmdloop.HaltLoop

HelloGoodbye().runLoop()

Articulated Keywords

http://littlelanguages.com/web/languages/articulated-keywords/keywords.html

Firefox's Bookmark Keywords

In Firefox, you can set a keyword for a bookmarked page. This keyword can be typed in as a shortcut to the location bar. If the bookmark link has a "%s" in it, then the text after the keyword will be substituted in.

For example, if the user had a bookmark (with the keyword 'k') of:

http://foo.bar.org/search?q=%s

And the user typed in:

k foo bar baz

Then the browser would construct:

http://foo.bar.org/search?q=foo bar baz

Creating articulated keywords

Given Firefox's bookmark keywords, and javascript's abaility to access the query string of the url, it is possible to write bookmarks which jump to pages which parse the query as a command sequence. Such command dispatch systems usually treat the first variable token as a command, and dispatch to functions registered on those names.

Note: it is also possible to bookmark 'javascript: ...' links, called bookmarklets. These can be quite useful, but require updating, while the articulated keywords method described here need users to set it once, and can be updated in a suite, and all users will see the updates.

A sample implementation is provided in keywords

Polymorphism in Shell

Polymorphism is a very useful programming language feature, so that code can deal with different, but related types of data in uniform manner. In languages like Java or C++, the types must be related vis inheritance or in the case of Java, as interface implementors. In Smalltalk, no such restriction exists. Any class can implement a method of a particular name. The reason I mention Smalltalk is because its semantics most closely resemble the mechanisms used for implementing polymorphism in shell. Let's dive into the details. In Java or C++, we can use statements of the form:

obj.method(5)

and which method is executed depends on which class obj is an instance of. This is called "dispatching on the type of the receiver", as the method chose to execute the "dispatch", is based upon the type of the receiver "obj". In shell, as there are no objects, we have to dispatch on something other than the type of an object. In evaluating an expression i shell, one of the first actions performed is variable expansion. For example, the expression:

rls=ls
$rls *.c

will list all the C source files in the current directory. the expansion of the variable rls into ls occurs before the search for the command to be executed. To get useful polymorphism, additional pieces are needed. A setting from a set of values needs to be provided to set the environment variable:

case $REMOTEOS in
win32) rls="DIR";;
   *) rls="ls";;
esac

will choose the appropriate command for listing files. Polymorphism is most useful when thare are groupings of functions. Again we use variable expansion to form the complete function name:

function win32_ls()
{
   rsh winHost DIR $*
}
function unix_ls()
{
   rsh unixHost ls $*
}
function win32_cat()
{
   rsh winHost TYPE $*
}
function unix_cat()
{
   rsh unixHost cat $*
}
case $REMOTEOS in
win32) rls=win32_ls
      rcat=win32_cat
      ;;
   *) rls=unix_ls
      rcat=unix_cat
      ;;
ease

Update: (crutcher) Alternatively, you could structure the alias construction as something like this:

case $REMOTEOS in
  win32)
    function rls() { rsh winHost DIR $*; }
    function rcat() { rsh winHost TYPE $*; }
    ;;
  *)
    function rls() { rsh unixHost ls $*; }
    function rcat() { rsh unixHost cat $*; }
    ;;
esac

Page Compiler

http://littlelanguages.com/web/languages/page2html/

Here's a little page compiler, it isn't much, but it points the way towards larger systems. I've used variations on this theme in many projects, and it works quite well, especially when used atop a version control system.

So we take a page written in an XML language which supports a superset of HTML, such as this one:

<page title="Example Page">
<ol>
  <li>Apple</li>
  <li>Banana</li>
  <li>Pear</li>
</ol>
<w page="Monkey"/>
</page>

And we write a transform which can resolve this page, expanding only the new elments we define (such as <w page="NAME"/>):

<xsl:transform version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<!--
 ! This transform demonstrates how we can transform
 ! page documents which contain supersets of html
 ! into pure html.
 +-->

<xsl:template match="page">
<html>
<head>
  <title><xsl:value-of select="@title"/></title>
  </head>
<body>
  <h1><xsl:value-of select="@title"/></h1>
  <xsl:apply-templates mode="resolve-page"/>
  </body>
  </html>
  </xsl:template>

<xsl:template match="node()" mode="resolve-page">
  <!--
   ! This is an identity template in XSLT. However,
   ! other templates with higer specificity will
   ! override this where appropriate.
   +-->
  <xsl:copy>
    <xsl:for-each select="attribute::*">
      <xsl:copy/>
      </xsl:for-each>
    <xsl:apply-templates mode="resolve-page"/>
    </xsl:copy>
  </xsl:template>

<xsl:template match="w" mode="resolve-page">
  <!--
   ! Here we resolve <w page="Name"/> elements into
   ! wikipedia links.
   +-->
  <a style="cursor: help"
     href="http://en.wikipedia.org/wiki/{@page}">
    <xsl:value-of select="@page"/>
    </a>
  </xsl:template>

</xsl:transform>

Applied to our earlier example, we get (example.html):

<html>
<head>
<meta http-equiv="Content-Type"
    content="text/html; charset=UTF-8">
<title>Example Page</title>
</head>
<body>
<h1>Example Page</h1>
<ol>
  <li>Apple</li>
  <li>Banana</li>
  <li>Pear</li>
</ol>
<a style="cursor: help"
    href="http://en.wikipedia.org/wiki/Monkey"
    >Monkey</a>
</body>
</html>

Now, all we need is a script to compile all pages in our directory tree, and we can stop writting HTML boilerplate!

#!/bin/bash
STYLESHEET="$(dirname $0)/page2html.xsl"
echo -n "Compiling pages ";
find . -name '*.page.xml' | while read PAGE; do
  echo -n "." # A nice little progress message.
  PAGE_DIR=$(dirname "$PAGE");
  PAGE_BASENAME=$(basename "$PAGE" .page.xml);
  xsltproc \ 
     -o "$PAGE_DIR/$PAGE_BASENAME.html" \ 
     "$STYLESHEET" \ 
     "$PAGE";
done;
echo
exit 0

Tail Call Optimization as Python Decorator

http://littlelanguages.com/web/languages/python-tail_call_optimization/

This is a bit different, its tail call optimization implemented as a python decorator (decorators are available in 2.4).

#!/usr/bin/env python2.4
# This program shows off a python decorator
# which implements tail call optimization. It
# does this by throwing an exception if it is 
# it's own grandparent, and catching such 
# exceptions to recall the stack.

import sys

class TailRecurseException:
  def __init__(self, args, kwargs):
    self.args = args
    self.kwargs = kwargs

def tail_call_optimized(g):
  """
  This function decorates a function with tail call
  optimization. It does this by throwing an exception
  if it is it's own grandparent, and catching such
  exceptions to fake the tail call optimization.
  
  This function fails if (and only if) the decorated
  function recurses in a non-tail context.
  """
  def func(*args, **kwargs):
    f = sys._getframe()
    if f.f_back and f.f_back.f_back \ 
        and f.f_back.f_back.f_code == f.f_code:
      raise TailRecurseException(args, kwargs)
    else:
      while 1:
        try:
          return g(*args, **kwargs)
        except TailRecurseException, e:
          args = e.args
          kwargs = e.kwargs
  func.__doc__ = g.__doc__
  return func

@tail_call_optimized
def factorial(n, acc=1):
  "calculate a factorial"
  if n == 0:
    return acc
  return factorial(n-1, n*acc)

print factorial(10000)
# prints a big, big number,
# but doesn't hit the recursion limit.

@tail_call_optimized
def fib(i, current = 0, next = 1):
  if i == 0:
    return current
  else:
    return fib(i - 1, next, current + next)

print fib(10000)
# also prints a big number,
# but doesn't hit the recursion limit.

Caseless Python!

http://littlelanguages.com/web/languages/python-caseless/

Here's the same trick as before, this time producing a python environment which is caseless. In these interactive environments, a good deal can be done by overriding the get/set/del methods.

#!/usr/bin/python2.4

import types
class CaselessDictionary(dict):
  def __setitem__(self, key, value):
    if isinstance(key, types.StringTypes):
      key = key.lower()
    dict.__setitem__(self, key.lower(), value)

  def __delitem__(self, key):
    if isinstance(key, types.StringTypes):
      key = key.lower()
    dict.__delitem__(self, key.lower())

  def __getitem__(self, key):
    if isinstance(key, types.StringTypes):
      key = key.lower()
    return dict.__getitem__(self, key.lower())

import code
code.interact(local=CaselessDictionary())

A Managed Environment in Python

http://littlelanguages.com/web/languages/python-managed-environment/

Here's a more interesting example. This works fine, unless the variables fall through to the global dictionary, or some code marks them global.

#!/usr/bin/env python2.4

import code

class ManagedVariable:
  def get(self):
    return None

  def set(self, value):
    pass

  def delete(self):
    # Return false to stop the delete.
    return True


class ManagedEnvironment(dict):
 def managing(self, key):
   if not self.has_key(key):
     return False
   return isinstance \ 
     (dict.__getitem__(self, key), 
      ManagedVariable)

 def __setitem__(self, key, value):
   if self.managing(key):
     dict.__getitem__(self, key).set(value)
     return
   dict.__setitem__(self, key, value)

 def __getitem__(self, key):
   if self.managing(key):
     return dict.__getitem__(self, key).get()
   return dict.__getitem__(self, key)

 def __delitem__(self, key):
   if self.managing(key):
     if not dict.__getitem__(self, key).delete():
       return
   dict.__delitem__(self, key)


class RangedInt(ManagedVariable):
  def __init__(self, value, (low, high)):
    self.value = value
    self.low = low
    self.high = high

  def get(self):
    return self.value

  def set(self, value):
    if value < self.low:
      value = self.low
    if value > self.high:
      value = self.high
    self.value = value


class FunctionValue(ManagedVariable):
  def __init__(self, get_func = None,
               set_func = None, del_func = None):
    self.get_func = get_func
    self.set_func = set_func
    self.del_func = del_func

  def get(self):
    if self.get_func:
      return self.get_func()
    return None

  def set(self, value):
    if self.set_func:
      self.set_func(value)

  def delete(self):
    if self.del_func:
      return self.del_func()
    return True


class Constant(ManagedVariable):
  def __init__(self, value):
    self.value = value

  def get(self):
    return self.value

  def delete(self):
    return False

def dynamic(str):
  return FunctionValue(eval('lambda:' + str))

d = ManagedEnvironment()
d['RangedInt'] = RangedInt
d['FunctionValue'] = FunctionValue
d['Constant'] = Constant
d['dynamic'] = dynamic
d['ranged'] = RangedInt(1, (0, 100))
import time
d['time'] = FunctionValue(lambda: time.time())
d['constant'] = Constant(42)

code.interact(local=d)

A Type Coerced Environment for Python

http://littlelanguages.com/web/languages/python-type-coercion/

Here's a fun little language. This only works in python 2.4, and it has problems there (it breaks if you invoke globals). The idea here is that python permits general mappings (as opposed to dictionaries) to be used for the local() namespace (though not the global() namespace). Here we are merely starting an interactive environment using a modified dictionary with an overridden __setitem__, and performing type coercion there.

#!/usr/bin/env python2.4
import code
class TypedDictionary(dict):
def __setitem__(self, key, value):
 if self.has_key(key):
   t = type(self[key])
   if t != type(value):
     try:
       value = t(value)
     except Exception:
       raise TypeError, \ 
             "illegal assignment to '%s':" \ 
             " %s cannot be coerced to %s" \ 
               % (key, type(value), t)
 dict.__setitem__(self, key, value)
code.interact(local=TypedDictionary())

This simple little environment provides basic (and very incomplete) type coercion to python, as the sample session below shows:

>>> a = 1
>>> type(a)
<type 'int'>
>>> a
1
>>> a = '23'
>>> type(a)
<type 'int'>
>>> a
23
>>> b = '4'
>>> type(b)
<type 'str'>
>>> b
'4'
>>> b = 23
>>> type(b)
<type 'str'>
>>> b
'23'
>>> c = 2.3
>>> type(c)
<type 'float'>
>>> c
2.2999999999999998
>>> c = 1
>>> type(c)
<type 'float'>
>>> c
1.0
>>> c = [1,2,3]
Traceback (most recent call last):
  File "<console>", line 1, in ?
  File "./t.py", line 11, in __setitem__
    raise TypeError, TypeError: \ 
       illegal assignment to 'c': <type 'list'> \ 
       cannot be coerced to <type 'float'>