Recently in Angular Category

I returned from Varaždin, where first Razmjena Vještina u Varaždinu was held. I decided to present on topic which can be summarized as the right problems from non-relational stack on the web.

I fully understand that presentation is packed with information, and I did skip some angular-mojolicious examples which are documented in README so take a look there also.

I have first CouchDB application in production from beginning of this week. This also means that I have new challenges, and this time task is simple: I want to send confirmation e-mail when someone enters data into my CouchDB.
Let's see how that can be done.... Easiest solution would be to send e-mail from web server which I have in the middle, but that would introduce additional latency to requests which require sending e-mail, so this isn't best solution. It would be much better to somehow extend CouchDB with ability to run custom code.

There are several ways to extend CouchDB in any language:

• View servers are useful if you want to write custom code to massage your data
  Since we don't really want to map over all data in our database, this approach isn't applicable here
• External processes allows you to write peaces of code which are executed on HTTP request, similar to CGI, but using JSON for passing data
  This might work for sending e-mails, but this would require special CouchDB installation which has installed external process, and I would need to make two requests (one to save data, and another to trigger sending of e-mail)
• HTTP database API has _changes feed which allows external process to connect to CouchDB and monitor all changes
  This seems like a perfect fit. I can have another process (possibly running on another machine) which will recognize documents which need e-mail notification.

I knew about _changes feed in CouchDB for a while, but Asynchronous architectures with the CouchDB _changes feed webcast pushed me into right direction. There are several things in that webcast which will be useful for this example:

• _changes?feed=continuous allows us to get instant notifications about document changes (with optional server-side filters) - only requirement is to have HTTP client which can understand CouchDB's chunked JSON responses.
• Documents in CouchDB can be used as state machine. In our example, we need to somehow note that we are working on sending e-mail (so that other workers won't pick it up while we are sending it) and note that sending is finised.

So, I decided to write couchdb-triggers.pl which implements queue using trigger field within document. In current implementation, you can put shell command in trigger.command and it will be executed on back-end and result will be returned in trigger.output. This requires three document updates, which are tracked using trigger.active array which contains timestamps of each step (with 0 indicating finished trigger). So, how does it look like?

We can submit document with trigger.command field to CouchDB:

$ curl -X PUT http://localhost:5984/monitor/free -d '{"trigger":{"command":"free"}}'
{"ok":true,"id":"free","rev":"1-511080b02d975944ea493eb5e9ef8783"}

If we try to get document a moment later, we will see that it containes trigger.output with results from our command:

$ curl http://localhost:5984/monitor/free
{"_id":"free","_rev":"3-a486faee4d416cd6ecc94183fe88a3bb","trigger":{"active":[1290521633.16487,1290521633.5196,0],"output":"             total       used       free     shared    buffers     cached\nMem:       3076500    3036388      40112          0     289572    1550120\n-/+ buffers/cache:    1196696    1879804\nSwap:      1949688     330296    1619392\n","command":"free"}}

However, as you can see, document was updated two more times (from _rev field). We also see that trigger.active has three elements, last one being zero. This means that trigger was started (first time-stamp), executed (second time-stamp) and successfully stored result back to CouchDB (final zero).

Separating trigger functionality from CouchDB itself also has benefits with replication: we can insert documents in any CouchDB, replicate it against one which has Internet connection and couchdb-trigger.pl running, and then just run replication again to get results.

Scheme with zero as last time-stamp also allows me to write server-side filter function which returns document for particular key only if trigger.status has zero as last element (meaning that trigger finished successfully). This can be used to implement synchronous triggers: first PUT document, and that call _changes?feed=continuous using filter on just that document until trigger fires (to refresh user interface or start replication back to remote database).

It seems that this trigger pattern will be useful for various tasks in the future, and my relational brain is slowly beginning to move into NoSQL land of replicated, schema free databases, but this time with external triggers :-)

A week ago, I stumbled by accident on <angular/> via Angular: A Radically Different Way of Building AJAX Apps. I was hooked. Making html ultimate template language by annotating it with few attributes and having two-way data-binding between input forms and objects which can be stored directly in the cloud make a lot of sense for me.

First a bit of history. Back in 2003 I wrote a small tool called wopi (Web Online Poll Interface) to generate on-line polls easily. It was basically html template with few magic fields which would be parsed by perl script to produce php code and database schema for deployment.
I was aware that this approach simply doesn't make sense any more, since browsers can do much more than in 2003, and we have flexible storage systems like CouchDB which doesn't require us having pre-defined data schema.

First I decided to try out hosted service at getangular.com to see if I can create working application in an afternoon. And I could.
Then I tried examples which worked on API at getangular.com against angular.js checkout on github, and I was at first surprised that things like ng-entity didn't exists in checkout. However, since storage API is part of hosting offer, that made sense. I will have to implement my own REST storage anyway, and REST API for angular hosting service is documented.

Another interesting thing was that source on github is newer version than one on hosting service, so I had a problem figuring out how to make $resource call which maps objects to REST work and developers on mailing list where very helpful in figuring it out.

Now, I needed to write some kind of REST storage against which I can test my angular controllers which I was writing.

angular-mojolicious is in-memory data store using documented REST API based on mojolicious lightweight perl web framework. In current stage, it provides ability to replicate data from shared storage and provides support for $resource call in angular.

Application which I was writing was simple conference submission app. Basically, one work per submission, but with possibility to group a bunch of works in symposium (single topic which have additional data).

I decided to duplicate symposium work inside each work to make this simple (and store number of work within symposium) and simply copy symposium data from first work to second work within same symposium.
However, I wanted to display all works within symposium and my REST back-end doesn't have any query capability yet (aside from returning all objects). At first, I just added additional javascript object symposium which stores all works within singe symposium in order, mostly so I can display list of works which are part of same symposium.
And again, implementing something visible on screen (list of works within symposium) I created something I can simply wrap into another $resource and store it on server, creating a data view which I needed.

All this duplication of data still seems a bit strange to my relational brain, and pure size of POST request to store symposium data with all works included could be prohibitively expensiveness for hundred of works. However, all data needed for symposium is stored within Work documents, so I could just write server side code which would update Symposium objects directly when Works are updated (something like stored procedures in relational lingo).

In the process, I also learned to pull upstream angular code often, because developers are very active, and any bug you might hitting now might already be fixed in upstream git. I learned it hard way, trying to figure out why my app isn't working while fix was already at angular's github repository. Kudos to fast release cycle.

Any sufficiently advanced technology is indistinguishable from magic. [Artur C. Clarke]

In summary, <angular/> is really good way to build web applications, so take a look. angular-mojolicious can get you started. It includes examples from angular cookbook and my simple conference submission application to get you started.

Update: My message about first week of development with angular got quoted on angular wiki testimonial page.