Sluift Interactive Console

Sluift, our Swiften-based XMPP script tool, comes with an interactive mode that lets you type your commands directly in a console. Until now, this console was actually the standard one that comes with the Lua distribution, a very bare bones “REPL” loop. However, we recently replaced this simple implementation with our own, which allowed us to do some usability enhancements such as tab completion, integrated help, and much more. These enhancements make it easier to execute commands, play around with Sluift, and help you writing Sluift scripts. In this post, we describe these new improvements in more detail.

If you want to try any of these new Sluift features yourself, just check out a development version from the Swift Git repository, or get it from homebrew using brew install --HEAD libswiften.

Tab completion

The first improvement is that the console now supports tab completion. This not only makes it easy to quickly type long commands such as sluift.new_client, but it also helps to quickly discover which commands or fields are available on a given object:

> sluift.<TAB>
base64     disco      get_help   hexify     jid        new_uuid   sleep      tprint    
copy       from_xml   help       idn        new_client sha1       to_xml     unhexify

> client = sluift.new_client('alice@wonderland.lit', 'MyPass')
> client:<TAB>
add_contact           for_each_message      get_next_message      messages              send_presence        
async_connect         for_each_presence     get_next_presence     process_events        set                  
cancel_subscription   for_each_pubsub_event get_next_pubsub_event pubsub                set_caps_node        
confirm_subscription  get                   get_software_version  pubsub_events         set_command          
connect               get_contacts          get_vcard             query_pubsub          set_disco_info       
disconnect            get_disco_items       get_xml               remove_contact        set_options          
events                get_dom               is_connected          send                  set_version          
for_each_event        get_next_event        jid                   send_message          wait_connected       

Print and store command results

A second improvement is that we print the results of each command invoked in the console. You therefore no longer have to remember to put = in front of a command to actually see the result:

> sluift.base64.encode("Hello world")
SGVsbG8gd29ybGQ=

> client
userdata: 0x7fd76b45a568

> client:get_software_version{to = 'wonderland.lit'}
{
  ['os'] = 'Linux',
  ['version'] = '0.9.0',
  ['name'] = 'Prosody'
}

Apart from printing each result, the console also stores each result of each command in the variables _1, _2, …, allowing you to reuse results easily in subsequent commands:

> sluift.new_client('alice@wonderland.lit', 'MyPass')
userdata: 0x7fd76b735fc8

> client = _1
> client:connect()

Integrated help

The console now comes with integrated help functionality. If you want help for any object, just call help(), and pass the object or function you want help for. For example, getting help for a function:

> help(sluift.new_client)

Creates a new client.

Returns a `Client` object.

Parameters:
  jid: The JID to connect as
  passphrase: The passphrase to use

Getting help for an object:

> client = sluift.new_client('alice@wonderland.lit', 'MyPass')
> help(client)

Client interface

Methods:
  add_contact: Add a contact to the contact list.
  async_connect: Connect to the server asynchronously.
  ...
  pubsub: Returns a `PubSub` object for communicating with the PubSub service at `jid`.
  ...

In case you don’t have an object, but want to see the methods available on a class of objects, you can pass a string to help:

> help("PubSub")

Interface to communicate with a PubSub service

Methods:
  events
  for_each_event
  get_affiliations
  ...

All the help provided in the interactive console is now also available on-line.

Changing context

Finally, we made it easier to do multiple operations on an object, without each time having to specify the object on which to call the operation. You can now change the “context” of the console by calling with with the target object as an argument. Any global function or field requested will first be searched in the target object, after which it falls back to the normal behavior. Moreover, any function called and found in the target object will be passed self implicitly. For example, suppose you want to do multiple operations on a client:

> with(sluift.new_client('alice@wonderland.lit', 'MyPass'))
alice@wonderland.lit> connect()
alice@wonderland.lit> get_software_version{to = 'wonderland.lit'}
{
  ['os'] = 'Linux',
  ['version'] = '0.9.0',
  ['name'] = 'Prosody'
}

Notice that the prompt of the console reflects the current context.

Resetting the context can be done by calling with():

alice@wonderland.lit> with()
>

The with function is also available for scripts to use (as sluift.with). You can pass a dynamic scope to only temporarily change the context. For example, a variant of the EchoBot that uses with would look like this:

client = sluift.new_client('alice@wonderland.lit', 'MyPass')
sluift.with(client, function ()
  connect()
  set_version{name = "EchoBot", version = "0.1"}
  send_presence("Send me a message")
  for message in messages() do
    send_message{to = message["from"], body = message["body"]}
  end
end)

Swiften gets PubSub support

We have added support for XEP-0060 (Publish-Subscribe) to Swiften, our C++ XMPP library. This allows you to create applications that create, manage, subscribe to, and publish to nodes on a PubSub service. Although we currently don’t expose any of the PubSub functionality in Swift yet, you can already do all PubSub tasks from Sluift, our Lua scripting interface built on top of Swiften. In this post, I will walk through performing a couple of common PubSub tasks in Sluift.

Common PubSub tasks

Before we start working with PubSub, we need to connect to an XMPP server, and choose a PubSub service:

== Sluift XMPP Console (20130901) == 
Press Ctrl-D to exit  
> client = sluift.new_client('alice@wonderland.lit', 'MyPassword')
> client:connect()
> client:send_presence()
> pubsub = client:pubsub('pubsub.wonderland.lit')

Using the pubsub handle, we can now perform all tasks on pubsub.wonderland.lit, our PubSub service of choice. For example, we can get a list of all the available nodes on our service:

> = pubsub:list_nodes()
{
  ['items'] = {
    [1] = {
      ['jid'] = 'pubsub.wonderland.lit',
      ['node'] = 'hatters_riddles',
    },
    [2] = {
      ['jid'] = 'pubsub.wonderland.lit',
      ['node'] = 'queen_quotes',
    }
  }
}

The result of list_nodes is a table with a list of nodes in the items field.

Let’s have a look what’s available on Queen’s node of quotes, and retrieve all items from her node:

> quotes = pubsub:node('queen_quotes')
> = quotes:get_items()
{
  [1] = {
    ['data'] = {
      [1] = {
        ['_type'] = 'body',
        ['text'] = 'Off with her head!'
      }
    },
    ['id'] = 'quote2'
  },
  [2] = {
    ['data'] = {
      [1] = {
        ['_type'] = 'body',
        ['text'] = 'Off with his head!'
      }
    },
    ['id'] = 'quote1'
  },
  [3] = {
    ['data'] = {
      [1] = {
        ['_type'] = 'body',
        ['text'] = 'Off with their heads!'
      }
    },
    ['id'] = 'quote3'
  },
  ['node'] = 'queen_quotes'
}

The result is a list of items, each having an id and a data field, the latter containing the actual data of the item. The data field in turn consists of a list of payloads (although typically only one), each corresponding to a Payload from Swiften. In case the payload is recognized, the _type field contains the name of the Swiften Payload class (converted to snake case), and the other fields correspond to the members of the specific Payload class. If the type of the payload is not recognized, _type is dom, and the fields will have a DOM-like representation of the XML payload. In the case above, the payload of the PubSub item is a standard Body payload, which has a text member with the text of the body.

The node we inspected doesn’t look very interesting, so let’s subscribe to the other node, and wait for an update for a while:

> riddles = pubsub:node('hatters_riddles')
> riddles:subscribe()
> = riddles:get_next_event()
{
  ['_type'] = 'pubsub_event_items',
  ['type'] = 'pubsub',
  ['from'] = 'pubsub.wonderland.lit',
  ['node'] = 'hatters_riddles',
  ['items'] = {
    [1] = {
      ['data'] = {
        [1] = {
          ['_type'] = 'body',
          ['text'] = 'Why is a raven like a writing desk?'
        }
      },
      ['id'] = ''
    }
  }
}

get_next_event blocks until an event comes in on the riddles node. Again, the event contains a _type field that corresponds to the Swiften type of the PubSub event payload, in this case PubSubEventItems, which consists of a list of published items.

We can also listen to all pubsub events for a few seconds:

> for event in client:pubsub_events{timeout = 5000} do print(event.item) end
{
  ['_type'] = 'body',
  ['text'] = 'Off with her head!'
}
{
  ['_type'] = 'body',
  ['text'] = 'Off with his head!'
}

Note that the item field of event is actually a convenience shortcut to the first item’s first payload.

Let’s now create a node of our own, to publish our blog updates to.

> blog = pubsub:node('alices_blog')
> blog:create()

Before publishing to it, we can have a look at the default configuration of the node:

> = blog:get_configuration()
{
  ['node'] = 'http://swift.im/Swiften/QA/PubSub',
  ['data'] = {
    ['fields'] = {
      [1] = {
        ['value'] = 'http://jabber.org/protocol/pubsub#node_config',
        ['type'] = 'hidden',
        ['name'] = 'FORM_TYPE'
      },
      [2] = {
        ['value'] = '',
        ['type'] = 'text-single',
        ['name'] = 'pubsub#title',
        ['label'] = 'Node Title'
      },
      [3] = {
        ['type'] = 'list-single',
        ['name'] = 'pubsub#access_model',
        ['label'] = 'Access Model',
        ['value'] = 'presence',
        ['options'] = {
          [1] = {
            ['value'] = 'open',
            ['label'] = 'Open'
          },
          [2] = {
            ['value'] = 'presence',
            ['label'] = 'Presence Sharing'
          },
          [3] = {
            ['value'] = 'roster',
            ['label'] = 'Roster Groups'
          },
          [4] = {
            ['value'] = 'whitelist',
            ['label'] = 'Whitelist'
          }
        }
      }
    },
    ['type'] = 'form'
  }
}

The configuration of a node is actually a Data Form description, together with a list of the current values of each variable. If we would for example like to make the node only viewable by a specific list of people, we can change the configuration to use a different access model by submitting a form with the pubsub#access_model variable changed:

> blog:set_configuration{configuration = {['pubsub#access_model'] = 'whitelist'}}

Now that the node’s set up, we can publish our first post to it:

> rabbit_hole_post = [[
  <entry xmlns='http://www.w3.org/2005/Atom'>
    <title>Down the Rabbit Hole</title>
    <summary>
      I was beginning to get very tired of sitting by my sister on the
      bank and of having nothing to do: once or twice I had peeped into the
      book my sister was reading, but it had no pictures or conversations in
      it, "and what is the use of a book," I thought, "without pictures
      or conversations?"
    </summary>
    <link rel='alternate' type='text/html' 
      href='http://www.gutenberg.org/files/11/11-h/11-h.htm#link2HCH0001'/>
    <id>tag:gutenberg.org,2008:entry-1234</id>
    <published>2008-06-25T18:30:02Z</published>
    <updated>2008-06-25T18:30:02Z</updated>
  </entry>
]]
> blog:publish{id = 'rabbit_hole', item = rabbit_hole_post}

Since Swiften doesn’t support parsing or serializing of Atom payloads, we construct the XML ourselves as a string, and pass it directly to publish. Publishing known payloads can be done using a structured format, as we will show in the PEP section below.

In case we change our mind about publishing the item, we can delete the item again (and optionally notify subscribers of the action):

> blog:retract{id = 'rabbit_hole', notify = true}

We can also decide to delete the node altogether:

> blog:delete()

Personal Eventing Protocol

The more recent ‘extended presence’ XMPP protocol extensions rely on XEP-0163 (Personal Eventing Protocol), which provides one PubSub service per JID for publishing personal events on, such as a change of mood, avatar picture, location, and much more. Since this is just PubSub underneath, we can control PEP information from Sluift as well.

For example, assume we want to publish our current location via PEP:

> geoloc = client:pubsub():node('http://jabber.org/protocol/geoloc')
> geoloc:publish{item = {
    _type = 'user_location', latitude = 50.3314, longitude = 4.7578}}

Notice that we omit the JID of the PubSub service, which is equivalent to the PubSub service of our bare JID, which is where each user has a personal PubSub service. In PEP, the node is identified by the URI of the protocol. Contrary to publishing raw XML as we did in the previous section, we now pass a table representation of the user_location payload to publish.

If we want to get updates of the location of our contacts via PEP, we have to let the other servers know we are interested in user location information by sending out presence with extra information:

> client:set_caps_node('http://wonderland.lit/my_client')
> client:set_disco_info{features = {'http://jabber.org/protocol/geoloc+notify'}}
> client:send_presence()

If we now listen for a while, we will start getting information about our contacts whereabouts:

> for event in client:pubsub_events{timeout = 5000} do 
    print(event.from .. ': ' .. tostring(event.item)) end

turtle@wonderland.lit: {
  ['longitude'] = -4.1345181465149,
  ['latitude'] = 50.366630554199,
  ['_type'] = 'user_location'
}
rabbit@wonderland.lit: {
  ['longitude'] = -4.7591938972473,
  ['latitude'] = 50.332908630371,
  ['_type'] = 'user_location'
}
hatter@wonderland.lit: {
  ['longitude'] = -4.2007088661194,
  ['latitude'] = 50.376739501953,
  ['_type'] = 'user_location'
}
queen@wonderland.lit: {
  ['longitude'] = -4.1416540145874,
  ['latitude'] = 50.551124572754,
  ['_type'] = 'user_location'
}

An example script that uses this is the ContactsMap.lua script, which puts all your contacts with their picture on a map:

Contacts Map

For a simpler script that just dumps all the PEP events on your contact list, see PEPListener.lua.

More

Although we only showed a handful of common PubSub commands, Sluift (and of course Swiften) supports many more. While we finish documenting Sluift, you can have a look at our PubSub test suite to get an idea of some other supported commands an how to use them.

Swift 2.0 Released

Just in time for a 2012 release: Swift 2.0 is out! This new stable release has many enhancements and bugfixes, so head over to the Release Notes page to find out what’s new.

Happy New Year!

Swift 2.0 Release Candidate Released

That’s right, we’re almost there: we just put up the release candidate of Swift 2.0. If no critical bugs are reported, this will become the final release. As always, we encourage everyone to get the new build and try it out, and tell us about any bugs they should come across.

Swift 2.0-beta2 Released

We have just released the second (and final) Swift 2.0 beta. Apart from several bugfixes, highlights include the possibility to set custom connection options, getting detailed information about certificates upon connection errors, and improved room invitations. We encourage everyone to get the new build and try it out, and tell us about any bugs they should come across.

Swift Hackathon Roundup

Last sunday, we finished our week-long Swift Hackathon, and it was a great success, leading to Swift 2.0-beta1! Here’s a list of the things we achieved during that week.

First of all, the goal of the week was to find and fix as many bugs as possible. This is what our ‘hackathon bug count dials’ were displaying at the end of the week:

Hackathon Week Bug Counter

In only one week, we found 19 bugs, and fixed 64! Not a bad result for our first hackathon, don’t you think? As you can see from the trend, we put a big dent in the list of open bugs:

Hackathon Week Bug Trend

And if fixing all these bugs wasn’t enough, we found the time to do some other things on the side as well during the week:

Thanks again to all the people who have helped us during this excellent week!

Swift 2.0-beta1 Released

After another year of development, we’re happy to announce that we released our first Swift 2.0 beta! We encourage everyone who is interested in helping us with testing to try out this new release, as it has many bugfixes and enhancements (see the release notes for more details).

Thanks to the hackathon week (of which details will be posted shortly), we believe this first beta to be pretty stable. Nevertheless, should you find some bugs, please come and tell us about it!

Swift Hackathon Update

We’re just halfway through our Swift Hackathon, so we thought we'ld update you about the progress we’ve made so far. In fact, a screenshot of our live hackathon week bug counter sums this up quite well:

Hackathon Week Bug Counter

That’s right: in merely a couple of days, we managed to fix 50 (more than half!) of the open bugs, and found 17 new bugs. And what’s more: we still have the whole weekend ahead of us, so you still have a chance to join us in fixing, testing, and improving Swift!

A big thank you to all the people who have been helping us out so far!

Google Summer of Code 2012

It’s that time of year again: Google announced which students they are going to sponsor for contributing to open source projects. This year, we have the pleasure of welcoming 3 students at Swift, who will be working on some very exciting projects.

This summer, you’ll see the following new faces hanging around the Swift room:

Since Kevin and I could only mentor Cătălin and Mateusz, and we really wanted to have Yoann join us as well, we decided to bring in some extra help this year. Tobias, who not only has participated in GSoC 4 times as a student, but also is a top Swift contributor, and authored practically all of the Swift Jingle code during GSoC last year, will be mentoring Yoann in the screen sharing project.

As you can see: great times ahead!

Swift Hackathon

All the cool kids are doing it, and so are we: starting Monday April 23rd, we’re holding a week long Swift hackathon! We will be focusing for a whole week on bugfixes, and at the end of that week release the first beta of Swift 2.0, the next major Swift release. Everyone is invited to join us online in our chatroom at swift@rooms.swift.im, and start hacking with us. And if you can’t or don’t want to fix bugs, we also need plenty of people to help us with testing Swift extensively that week.

(Thanks to Tobias for suggesting this).