Pomme::TAB

I use vim as my text editor.
Practicing TDD for creating perl applications usually imply the following workflow:

the perl code (*.pm) and the test code (*.t) open in the same terminal view where the vim window is split into two horizontal halves. That’s on my left display.
On the right display is a shell where I do run the test (prove my_test.t and its variations). The right display is also used for perldoc (and the web browser).
On the left display I obviously switch often between the two halves, writing some tests then writing the code, going back to the test code to write some more. Between each switch I always run the test.
Because of the convenience of having the test code and the application code on the same window, the ease of switching with vim and the rhythm of TDD I found TAB-switching to the other terminal window on the other screen counter productive.

I know that some people run a !prove mytest.t in command mode under vim, but that’s still too long and you have to type the name of the file.
Granted you do that only once and the other times you can just type !! to re-exec the last command.
This is screwed, if you do run other commands in the same way as the last command may not be the one you think it is anymore.

vim as a built-in command make that does different things depending on the mode. The usual set up for perl is for make to do syntax checking (perl -Wc) on the current file. This can be setup in the vim configuration file (.vimrc) in a user home directory (on UNIX) using the makeprg option.

If make could run the test for me instead of syntax checking that would be a better solution than what I described above. However I don’t want to loose the ability to syntax check as it is faster and less confusing to find stupid typos that way rather than seeing a test failing and trying to find the reason amidst the jungle of error messages.

Also, ideally, I’d like to syntax check and test run to be triggered by a press of a key rather than having to type :make.

So I’ve added these lines in my .vimrc:

map <F4> :w<cr>:set makeprg=perl\ -c\ %<cr>:make<cr>
map <F8> :w<cr>:set makeprg=prove\ -v\ %<cr>:make<cr>

After switching to the test code, pressing F8 will run the test while pressing F4 will do a syntax check.
F4 will work on any perl script, so you can syntax check both the application code and the test code.
Also you don’t have to save the file as the file will be saved before syntax check or tests is run.

You have to switch to the test code before you can press F8 to run the test.
Pressing F8 when the application code is the current file will attempt to run prove on it which will fails of course.
Ideally, in this situation, I’d like vim to find and run the corresponding test based on location/name of the application code but I don’t know how to do that.

The fact that all the main activities I do are now done on one split window, leaves the second screen for documentation (perldoc and web browser), running the whole test suite, doing other tasks, …

The benefit of the split window is also very useful when doing TDD in a Coding Dojo (or if you have only one screen) as there’s only one screen projected on a wall for the audience to see and everything is happening on this screen.

Curtis Poe has written an article on ONLamp titled “TAP vs XML”.

TAP (Test Anything Protocol) is a specification for the test report format as generated by Test::More and family in perl.

Very popular in the perl world, TAP is language agnostic and you can find test frameworks generating TAP in many languages.

There is supposedly one in python but I couldn’t find when I started my python project, so I fell back to unittests, ex-PyUnit, port of the jUnit test framework (the perl member of the xUnit family being Test::Unit).

The other thing I like about TAP is its simple and predictable format that makes its easily machine readable. That triggered a plethora of perl modules that “do things” with a TAP report.

Thanks to a module like Test::TAP::HTMLMatrix, we have been able to generate report like that:

It is based on Test::TAP::Model which I’ve seen being used to store the test reports in a SQLite database to facilitate a fined grained control of the evolution of tests (for which revision of the code they are passing, which ones are failing, if there is a pattern when a group of tests starts all to fail, …)

The ONLamp article presents TAP as opposed to XML.

TAP is good as long as you can build an ecosystem around it like we did but there’s a whole world out there where XML is deeply anchored for messaging and configuration.

It just so happens that I’ve been observing this world for a while from my perl island. What have I seen? I’ve seen tools, great tools grown up to support modern software engineering practices.

my options were:

• moaning about these tools being unfriendly to perl
• try to recreate these tools for perl projects
• subvert these tools by using them with perl

I’ve tried the first one, it was for good for me but didn’t make things moving.
The Second one has been tried too: too much code that needs maintain (and we are not paid for working on such things) and not good enough to be open sourced (because not being paid to work on such things, we rushed them).

We’ve started to look at the third option which seems to be the One.

In the case I’m interested in, it’s to fully make use of Cruise Control to support continuous integration and automated testing for my perl project.

As I am in a position where I’ve got the basic functionalities working along with some nice perl-CC integration (our HTMLMatrix test reports are now generated and archived within CC), I want to further integrate the perl testing framework to Cruise Control reporting facilities.

And this is where the TAP vs XML ends for me.

I need my TAP based perl framework to talk nicely to the Cruise Control build report.
For example, I want him to stops telling me that there’s no test run when I know that are, because CC doesn’t understand the format test output.

I don’t mind the unreadability of the xml, as XML is not for human to read.
It will be transformed by XSLT (after merging it with CC build xml log) in a human-readable form to be part of the CC build report.

I don’t mind xml generation being prone to errors by human (remember xml is not for human to use) as xml syntax can normally be validated by machines using Schemas.

I found a possible helper module on CPAN in the shape of Test::TAP::XML that needs to be investigated.

So it’s more like TAP+XML then?

For me, yes, but I just found someone who also tries to bury XML in the world of project automation

I’m mainly a perl programmer, but last week I’ve decided to explore the python language.

To keep my motivation high I’ve decided to use python for something useful to me: download every day my favorite web comic into the Pictures directory of my home directory on Mac OS X and keep it uncluttered by removing older episodes.

In the working of the project I’ve learned how in python to scrap web sites, parse XML with DOM, handling exceptions, manipulating the filesystem, doing time calculations, using regular expressions, …

Python is quite cool, and I’m planning to further improve the application.
I’m still using the default Mac OS X python, the 2.3.5 without readline
I’ll try to install a more recent version with readline support.
Python 2.5 is tempting by I don’t know if it is well supported yet by Trac, an open-source python project I’m intending to hack at some point.

I’ve added in my del.icio.us bookmarks a bunch of links that I’ve found helpful.

The other day I was discussing with software testing expert Antony Marcano about the issues encountered in introducing Test driven development in a team. Especially we were talking about resistance from the developers in introducing automated acceptance testing.

They see them as redundant: they implemented the feature and they know it works, so why do they have to write additional tests?

I reckon what has not been perceived by the developers in such situation is:
For one, a developed feature will have a life after its first release, it often has to stay part of the product for a long time. Sometimes the feature will be improved or changed as driven by clients requirements (more on that later). During all that time the client need to be sure that the feature is working. It’s obvious why the fact that the developer knows it’s working at one point in time is not enough.
Even if the feature is not being change for a long period, there may be other features that are added or changed that may break that stable feature because of interdependencies.

Secondly, what has been completely missed is the fact that an acceptance test in TDD is not really a “test” in the meaning that is commonly understood by developers.
this look like a language barriers issue (This is where Antony mentioned to me the existence of behavior driven development - or BDD). Acceptance tests are to represents the specification for a feature defined by the customer/business owner.
This specification takes the form of an automated test so that developers can code the feature against it. From a developer’s perspective it is a tangible goal to work against. Before the feature is created the test fails, when it’s done the test is passing. The acceptance tests “drive” developers effort.

This way of expressing requirements reduces the risk of ever-changing requirements as the business owner would have to update the acceptance test each time making him/her think twice when creating/changing the specification for a feature. That’s a win for the developers.
Because the business owner has written the test (maybe with help of a tester), when the test passes , they know that the feature that has been developed is what they wanted and it’s working. That’s a win for business owner.
The business owner and the developer know that at whatever stage of the life of a product, there is an automated test to assert that a particular feature is working. That’s a win for both.

On the same subject, The Register has published an excellent article last night:

Driving on the Right Side of the Code: “

Test Driven Development

“

(Via The Register.)

A couple of years ago I played with something called Camel Bones.
It is a Cocoa framework with a perl module an Xcode template.
It allow a programmer to use perl to write desktop Cocoa application for Mac OSX
It was fun, although the application built are not the fastest in the world.

I was thinking of that when I read this blog article about dynamic language to write Mac OSX desktop application. I’m particularly looking forward to Objective-C 2.0 and it’s new garbage collector.
That will make a big change, as I remember that when writing a Cocoa application with objective-C, a programmer has to handle manually memory allocation and de-allocation.

I don’t entirely agree with Scott Stevenson when he says that Obj-C 2.0 will obviate the need of using a dynamic language to write Mac OSX application: One of the reason of doing so is to give practitioner of a popular scripting language an entry point to Mac OSX programming. Also it facilitates cross-platform reusability.
If you’ve written a perl application with CLI on Solaris, you can make it a GUI application on Mac OSX by adding a GUI layer using Camel Bone or using TK to get it work on Linux. Additionally, outside Mac OSX, Obj-C hasn’t got a huge user base as perl and python have.

These bridged applications won’t be as fast and as integrated to Mac OSX as a native Objc-C application though.

Using Dynamic Scripting Languages for Mac OS X Application Development: “

There’s been a lot of interesting discussion over the last week regarding the use of dynamic scripting languages for programming desktop applications. Here are a few links that caught my eye.

“

(Via Daring Fireball.)

I use vi daily. I switched from Emacs 4 years ago. Sometimes I reflect with a smile on the fact that I’m still happy and productive with a software written before I was born.

The Birth of vi: “lanc writes ‘Bill Joy, co-founder of Sun, tells the story of how he wrote the vi editor. The article at The Register delves into his motives, who instigated the project, and some of the quirks of leaving a ‘gift to mankind’. From the piece: ‘9600 baud is faster than you can read. 1200 baud is way slower. So the editor was optimized so that you could edit and feel productive when it was painting slower than you could think. Now that computers are so much faster than you can think, nobody understands this anymore. The people doing Emacs were sitting in labs at MIT with what were essentially fibre-channel links to the host, in contemporary terms. They were working on a PDP-10, which was a huge machine by comparison, with infinitely fast screens. So they could have funny commands with the screen shimmering and all that, and meanwhile, I’m sitting at home in sort of World War II surplus housing at Berkeley with a modem and a terminal that can just barely get the cursor off the bottom line.”

Seen in a subversion commit message. Made me smile for some obscure reasons:

“Possible fix for forthcoming bug” 

And that’s why automated testing of the integration points are very important!

I’m currently involved in a huge project made of multiple components.
To each component corresponds a project team.
My direct upstream dependency is a web API.

In addition to our own unit tests and system tests, we’ve got a set of test scripts that monitor the features we are using. That includes:

• validating the supplier API against their own XML Schemas (Relax-NG in our case) for the API calls used in our system
• If both components use a common set of reference data, a test to check that the data set is the same in the API and in our component

There’s trouble only if the XML Schemas are not updated when changes are made to the API.

As the tests are run regularly we discover changes in supplier’s API quickly. If we don’t have time to make the change in our side (e.g: 5 minutes before a demo) we switch to another instance of the API with the old code (the supplier has a set of rolling instances that guarantees that the previous version of the API is always available on an instance). This is not really applicable with public APIs.

An additional step we have taken is to mock up the results of API calls into static XML. These are normally used for our internal system testing, but if the integration tests with the supplier fails and there isn’t a workable instance of their API, we can switch to the static XMLs (this situation happened once in our project).

This approach will also work with public APIs that output XML like Flickr’s API and Amazon REST service.

Doing a Demo? Don’t Rely on an API: “

Eurobuddy Matt was doing a presentation when there was an API change at GoogleMaps. He’s a quick fella, but I’ve been caught with my pants down.

“

(Via ONLamp.com.)

The first time I was presented Selenium I had the wrong understanding of it.

Few months ago, a group of client side developers organized a talk in our department about testing javascript code. The focus was mainly on jsUnit, but it was also the first time I heard of Selenium. It was presented as an alternative to jsUnit to test client side javascript in a cross-browser way.
For that purpose Selenium acts as a browser plugin that allows one to record his actions on the browser on a particular site and play them back later.
You can also finetune the generated scripts.

In the same time I was starting to think about acceptance testing and automation testing.

Until the Continuous Integration and Testing conference in London, I didn’t realise there was a link between the two.

During that Open Space conference, there was a talk by a guy from Germany.
He was sharing his experience with Selenium for acceptance testing.
Obviously the product he was developing is a web application.
It means that most of features (or business value) are accessible from a web GUI.

This how it became really clear to me that Selenium can work in two way: in addition of the plugin mode for manual record/playback, there’s a driven mode that allow a programmatic control of Selenium. Now we are getting closer to automation
Before going that way, I’ll dwell a bit more on the acceptance testing part of it.
The acceptance tests are the tests that prove the product owner/customer that the software is doing what he has specified. For a web applications, most of the business features are visible on a web interface. Even if for some reasons the business value is invisible (backend features of some sort) it’s often possible to make it exposed on a web interface:

• On an earlier project one of the requirement was that the system should process data within a ceration amount of time. We build a small web application to expose the cycle duration on a web interface. We used Nagios to monitor this value but we could easily imagine that this information can also be captured for acceptance tests.
• Let’s say that you’ve been asked to syndicate your data as RSS feeds. You can use Firefox as an RSS aggregator and test the feeds from there.
• On another project the feature was to import a continuing feed of complex XML documents in a database. The team setup a smart logging and exposed import successes and failure on a web interface.

Now, we can go back to automation.
Selenium can be driven programmatically through a number of platform (java, python, ruby and perl). It uses a component called Selenium Remote Control which is a java server that run on the same platform where a web browser is available.

There’s also a platform dependent API that sensd command to the Remote control using the language of choice.

In perl you can use a CPAN module called Test::WWW::Selenium.

The whole thing becomes really interesting when you combine that approach with the use of FIT (Framework for Integration Testing, which is more than a framework actually), but that’s another story.

This afternoon one of my developers told me:

“There is a certain elegance to xml”