↓ Archives ↓

Category → dev

Jumping to JavaScript code definitions on click in vim with tern.js

If you develop JavaScript a lot and do not know tern.js, you should definitely check it out: Tern.js anaylzes your code and enhances your various editors with code completion, function argument hints, refactoring and more goodies.

If you use vim and want to use Tern’s ability to jump to a variable or function definition by simply Ctrl-clicking on it, just create a file called ~/.vim/ftplugin/javscript/ternclick.vim and add this line to it:

:nnoremap <buffer> <C-LeftMouse> <LeftMouse>:TernDef<CR>

By limiting this functionality to buffers with filetype javascript you can still use other ctrl-click plug-ins (or built-ins) like ctags.

Remove diacritics (Umlauts, Accents, Special characters) in JavaScript

So I recently found myself generating permalinks in JavaScript again which can be fun and painful. It seems to be less painful if you just ignore anything that’s not [a-zA-Z0-9] and replace it with a hyphen -.

However, this starts looking ugly rather quickly if you’re from Germany or France for instance, where use of umlauts and accents is very common. Something really nice like
J'ai montré les éléphants à ma sœur
becomes something really ugly like
j-ai-montr-les-l-phants-ma-s-ur.

So as Holger pointed out, I needed a diacritics table which I found here. After some modifications for the German language (e.g. ä -> ae, ß -> ss), I came up with this.

It’s still heavily based on what lehel built, so thank him, not me. I just wanted to put my improved version here, so I don’t forget it.

Update: I have created a Gist for this over at Github so we can continue to update it there…

Switch from Bazaar to Git

Recently I had to switch a project from Bazaar to Git. Fortunately this has become quite easy using the right plugins. Just in case anyone out there needs to do the same: these are the steps if you want to keep your version history.

Git comes with an import plugin, bzr has an export plugin available. On debian based distros you might need to install the latter using

# available in lenny-backports, squeeze and ubuntu since 9.10
sudo apt-get install bzr-fastimport

Assuming you have  Bazaar repository in the folder ./bzr-repo just follow these steps to clone it to a git repoitory in the folder ./git-repo:

BZR=./bzr-repo
GIT=./git-repo

mkdir $GIT
cd $GIT
git init
bzr fast-export ../$BZR/ | git fast-import
git checkout
cd ..

The best part is: If you missed some revisions of Bazaar, e.g. because you forgot to merge an important branch, you can easily repeat the line bzr fast-export ../$BZR/ | git fast-import and Git should be smart enough to only import missing revisions.

Sparkleshare

Today a release candidate for sparkleshare 0.2 came out. Thanks to @jan for making me notice this great software a few days ago. It’s kind of a dropbox or ubuntu one clone but completely open source. You can use it to store your local data on a public server, your own server, your intranet or wherever you trust to put our data. All you need is a git server, ssh and a certificate to access it. No matter if it’s your local data storage, github or plan.io.

Comparing it to dropbox of course there still are a lot of downsides given its early state. There is no sophisticated encryption, only the data transfer is encrypted using ssl certificates. There is no GUI for easy sharing of data: whoever has access to your git repository can access your data, but no one else. However there are already a few nice features working quite well. Among them the easy GUI-based setup, the automatic syncing and the nice way to access old versions o data: right-click on a file, choose a version and a new file appears with the old date in its file name.

There is the source code for linux and mac versions available, binaries and a windows version are expected at a later point of time. In case you are on Ubuntu or Debian I recommend the following steps for installation after getting and unpacking the source from hbons’ github:

# install dependencies for the build process
sudo apt-get install gtk-sharp2 mono-runtime mono-devel monodevelop \
libndesk-dbus1.0-cil-dev nant libnotify-cil-dev libgtk2.0-cil-dev \
libwebkit-cil-dev intltool libtool python-nautilus libndesk-dbus-glib1.0-cil-dev

# using the prefix you enable the nautilus extension
./configure --prefix=/usr

# compile
make

# use checkinstall instead of make install to install it as a clean debian package
# that can be easily uninstalled using your favorite package manager
sudo checkinstall

# start the service
sparkleshare start

# You possibly have to restart nautilus to enable the plugin
nautilus -q

So go try it out!

Bazaar over chrooted sftp

How to set up bzr for chrooted sftp users

How to restrict a user’s access to sftp://.../var/bzr

For the prototyping editor itself and for a lot of our clients’ projects we are heavy Bazaar users at pidoco° to manage our distributed workflow. When we started some years ago we just installed bzr on one of the test servers where all of the developers had ssh access anyway. We put the repositories in /var/bzr and used sftp to checkout/push/pull source changes. This was handy as a sftp server comes with openssh installed.

As the team grew over the years we got to a point where we wanted to give new developers access to the bzr repositories without giving them full ssh access. However we did not want to have to change all the urls for existing repositories. Luckily this can be achieved easily since Debian Lenny.

Per Server Settings

On our scm server we have a user group called bzr that grants read/write access to most of the repositories (except of some personal or release branches) to all users with bzr access. And now we added the group sftponly. All users in this group will be restricted to sftp access only instead of a full shell.

sudo addgroup sftponly
sudo addgroup bzr

You probably have to add ‘/usr/lib/sftp-server’ to /etc/shells to make it a valid shell, eg. like this:

root@host # echo '/usr/lib/sftp-server' >> /etc/shells

The following settings in /etc/ssh/sshd_config force the internal sftp server to be used by openssh and change the root directory for all users in the group sftponly to /var/chroot. Make sure to restart sshd afterwards.

Subsystem sftp internal-sftp
Match Group sftponly
    ChrootDirectory /var/chroot
    AllowTCPForwarding no
    X11Forwarding no
    ForceCommand internal-sftp

Up to now our repositories have been in /var/bzr. These need to be moved to /var/chroot/var/bzr to let the sftponly users access them. /var/chroot needs to have root:root as owner for openssh to work correctly. For the existing ssh users we add a symbolic link to keep the old paths working:

sudo mkdir /var/chroot
sudo chown root:root /var/chroot
sudo mkdir /var/chroot/var
sudo mv /var/bzr /var/chroot/var
sudo ln -s /var/chroot/var/bzr /var/bzr

Per User Settings

giving the user username sftp access, but nor ssh access:

USERNAME=username                                  # give the user a name
sudo adduser ${USERNAME}                           # add user and data to system
sudo usermod -s /usr/lib/sftp-server ${USERNAME}   # disallow ssh/bash, allow ssh/ftp (sftp)
sudo adduser ${USERNAME} bzr                       # allow group access to most bzr folders
sudo adduser ${USERNAME} sftponly                  # disallow access to /, allow access to /var/bzr

This changes user’s shell to sftp-server.

Result

As a result of these settings both normal ssh users as well as restricted users in the sftponly group can use the same url for their shared repositories
bzr checkout sftp://my.domain/var/bzr/my_repository. By using chroot however users in the group sftponly are restricted to using sftp and can only access the folders in the bzr subdirectory.

Sources

In the Debian Administration Weblog you can find information on how to setup an OpenSSH SFTP chroot() with ChrootDirectory and on how to restrict users to SFTP only instead of SSH.

JavaScript Testing and Continuous Integration Part II

In my previous post I described how to use the YUI Test Framework to write JavaScript tests for your web application. This part continues with how to run the tests in different browsers and how this can be used with a continuous integration server.

Executing your tests in different browsers

A framework that is very famous when it comes to browser testing is Selenium. It automates the execution of click streams in different browsers using two major components: the selenium IDE and the selenium remote control. The IDE is a Firefox plugin for recording the click streams. Originally, the click streams are written as plain HTML tables, but over the time selenium added drivers for the most popular programming languages. This way, you may record a click stream and have it transformed into e.g. Java / JUnit, where you then can add several assertions or high level programming constructs. A simple selenium JUnit test may then look like this:

public class MyTest extends SeleneseTestCase {
	protected Selenium selenium;

	public void setUp() {
		selenium = new DefaultSelenium("192.168.1.111", 4444,
                                       "*firefox", "http://you.web.server/");
		selenium.start();
	}

	public void testSomething() {
		selenium.open("http://you.web.server/path/to/your/tests");
		selenium.windowMaximize();
		selenium.windowFocus();
		// wait for...
		selenium.isVisible("unittestsFinished");
	}
}

Having the test, the selenium remote control come into play. It is used to open a browser, inject some JavaScript code that triggers all the desired events, and click through your app. In our case we use selenium to open a firefox, load the HTML page containing our YUI tests, and wait for them to be finished. (The wait is not shown in here, you have to write it yourself.) When our tests finish, we write an invisible <div> containing the string ‘unittestsFinished’ which indicates selenium that the browser may be closed.

Iterate over your browsers

Now that we have selenium to trigger our JavaScript tests we still need to iterate over all browsers. We also need to trigger selenium somehow. Since the continuous integration server at pidoco is a hudson server, and hudson can execute an ant build script, we use ant to do the job:

<property name="browsers" value="firefox,safari,iexplore"/>
<for list="${browsers}" param="browser" delimiter=",">
	<sequential>
		<junit haltonfailure="false" dir="antbuild" fork="yes"
                               showoutput="yes">
			<sysproperty key="browser" value="@{browser}"/>
			<formatter type="xml" />
			<test name="tests.selenium.SeleniumYUITestsLauncher"
                              todir="tests"
                              outfile="@{browser}-selenium-result.xml"/>
		</junit>
		<get src="http://your.test.server/path/to/junitxml"
                     dest="tests/TEST-@{browser}-yui-result.xml"/>
		<replace file="tests/TEST-@{browser}-yui-result.xml"
                         token="testsuite name=&quot;"
                         value="testsuite name=&quot;@{browser}."/>
	</sequential>
</for>

Ant iterates over a list of browsers that we want the tests to be executed in. It then triggers the selenium test via JUnit. The test result in the *selenium-result.xml is not very interesting since it is only one test which opens the yui tests. Using a system property ant tells selenium which browser it should open.

Next, we download the reported YUI test result from our test server to the integration server. You may also use selenium to read the result from the HTML page directly and save it to a file within the JUnit test. However, we had a test report server from the beginning, so we used that to get the results from the browser to our integration server. Once we have the xml file we use a regular expression to add a fake package to the test suite such that the integration server can tell us in which browser a test may be failing.

Continuous Integration

The only thing that is left for the continuous integration server is to call the ant target and read all the xml files in the ‘tests’ folder as JUnit test results. This is pretty easy but allows you to analyse your JavaScript tests, which were executed automatically in different browsers, with all the power of the continuous integration server of your choice.

JavaScript Testing and Continuous Integration Part I

Adding your Tests into a continuous integration system for test automation is quite common among professional software development. However, when it comes to testing JavaScript code, many people lack the experience and best practices to set up a productive infrastructure. We at pidoco have invested quite some time to solve this problem and came up with the setup I describe in this blog post.

The JavaScript Testing Framework

There are countless frameworks out there to use for testing JavaScript code. And I have to admit it is quite confusing to find the best for one’s project. From what I saw up to now, many frameworks look very similar to each other. They do differ in some details, which make people like one or the other. To us, it was important to have a comprehensive documentation of the frameworks capabilities. Therefore, we chose the Yahoo UI test framework. It comes with an extensive documentation, just as you know from the Yahoo UI framework. You can transform a JavaScript Object into a test case, which makes it looking quite similar to JUnit tests:

var testCase = new Y.Test.Case({

    name: "TestCase Name",

    //traditional test names
    testSomething : function () {
        //...
    },

    testSomethingElse : function () {
        //...
    }
});

Within your test functions you have a broad variety of assertions that you can use. In addition, you can use the wait() and resume() functions to interrupt a test and wait for asynchronous events to happen. This is very useful if you need to wait for some UI components to be rendered or some server calls to return. Next, you can also trigger dom events using the YUI Event object:

Y.one("body").simulate("click");

Once you have written your first test case you want to execute the test and review the results. YUI Test comes with a nice test result viewer that you may include into your test site. On the other hand, the tests should be executed in several browsers and you may want to review the results on just one page. Therefore, the framework offers a function to send the result to a URL where a server may collect and show the results. In our first attempt we sent the result as a JSON string to the server, who transformed the collected results in a simple table with the tests in rows and the browsers in columns.

var reporter = new Y.Test.Reporter("http://www.yourserver.com/path/to/target",
                                   Y.Test.Format.JSON);
reporter.report(results);

Since many continuous integration servers like to read test results in an xml file, YUI allows for several formats to get your results:

  • Y.Test.Format.XML (default)
  • Y.Test.Format.JSON
  • Y.Test.Format.JUnitXML
  • Y.Test.Format.TAP

Oh, and if you asked yourself ‘where the heck is this Y coming from?”, here is the solution:

var Y = YUI();
Y.use('test');

Being able to write a JavaScript unit or integration test the next question is coming up:

How to test your web app

Following a test driven development a lot of JavaScript has to be written just for the tests. When deploying the web app into the productive environment, we don’t want to include the testing code. But how to separate testing code from productive code? We found two solutions to be possible. First, we use a plugin that loads, triggers, and reports the tests, but is only included into the system during development and test. This way, we simply disable the plugin and have a productive system without any testing code. However, not every web app has a plugin system that supports this solution. Therefore, we have thought of a second way to test your app: deploy a second app that contains a simple HTML page which loads the testing code plus an iFrame containing the web app to be tested.

Please have in mind the same origin policy for the HTML page containing the iFrame and the content of the iFrame. If both are delivered from the same domain, the outer page may access the JavaScript code of the iFrame content. This enables us to call the productive code from the testing code. In case your code either of the application or the test checks for e.g. instances of Array, it fails if the array was created inside the iFrame and checked outside or vice versa. You may create an array using new window.frames[0].Array() and test for an array with a instanceof window.frames[0].Array.

Using the iFrame approach you may enhance the HTML page such that it loads you app and the test code, automatically runs the tests, and listening to the Y.Test.Runner.COMPLETE_EVENT to upload the test result to your reporting server.

How the tests are triggered in different browsers and how this goes with continuous integration is subject of the second part of this blog post.

Detecting Safari 5.0 with browscap.ini

There are many ways to identify browsers on the server side. Gary Keith offers a quite complete mapping of user agent strings to browser/os versions enhanced with information about their capabilities. I guess it’s impressive how fast Gary was for all these years in adding new browsers to his browscap.ini and how reliable in keeping it up-to-date. Unfortunately, according to his website, he is currently facing health issues and cannot maintain the project for a while. So if you are using php_browscap.ini but missing Safari 5 or Chrome 6 support you can apply the patch listed below. Until then: Get well soon, Gary!

UPDATE:
Gary has uploaded a newer version of browscap.ini which now includes Safari 5 and Chrome 6. You don’t need to apply the patch below anymore.

--- browscap.ini_old    2010-06-22 21:50:30.000000000 +0200
+++ browscap.ini_new    2010-06-22 22:31:55.000000000 +0200
@@ -5504,6 +5504,57 @@
 Win32=false
 Win64=true

+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Chrome 6.0
+
+[Chrome 6.0]
+Parent=DefaultProperties
+Browser="Chrome"
+Version=6.0
+MajorVer=6
+Win32=true
+Frames=true
+IFrames=true
+Tables=true
+Cookies=true
+JavaApplets=true
+JavaScript=true
+CssVersion=3
+supportsCSS=true
+
+[Mozilla/5.0 (Macintosh; U; Intel Mac OS X*; *) AppleWebKit/* (KHTML, like Gecko) Chrome/6.* Safari/*]
+Parent=Chrome 6.0
+Platform=MacOSX
+
+[Mozilla/5.0 (Windows; U; Windows NT 5.1; *) AppleWebKit/* (KHTML, like Gecko) Chrome/6.* Safari/*]
+Parent=Chrome 6.0
+Platform=WinXP
+
+[Mozilla/5.0 (Windows; U; Windows NT 5.2; *) AppleWebKit/* (KHTML, like Gecko) Chrome/6.* Safari/*]
+Parent=Chrome 6.0
+Platform=Win2003
+
+[Mozilla/5.0 (Windows; U; Windows NT 6.0; *) AppleWebKit/* (KHTML, like Gecko) Chrome/6.* Safari/*]
+Parent=Chrome 6.0
+Platform=WinVista
+
+[Mozilla/5.0 (Windows; U; Windows NT 6.1; *) AppleWebKit/* (KHTML, like Gecko) Chrome/6.* Safari/*]
+Parent=Chrome 6.0
+Platform=Win7
+
+[Mozilla/5.0 (Windows; U; Windows NT 7.0; *) AppleWebKit/* (KHTML, like Gecko) Chrome/6.* Safari/*]
+Parent=Chrome 6.0
+Platform=Win7
+
+[Mozilla/5.0 (X11; U; Linux i686*; *) AppleWebKit/* (KHTML, like Gecko) Chrome/6.* Safari/*]
+Parent=Chrome 6.0
+Platform=Linux
+
+[Mozilla/5.0 (X11; U; Linux x86_64; *) AppleWebKit/* (KHTML, like Gecko) Chrome/6.* Safari/*]
+Parent=Chrome 6.0
+Win32=false
+Win64=true
+
+
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Google Code

 [Google Code]
@@ -9604,6 +9655,47 @@
 Parent=Safari 4.0
 Platform=Win7

+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Safari 5.0
+
+[Safari 5.0]
+Parent=DefaultProperties
+Browser="Safari"
+Version=5.0
+MajorVer=5
+Frames=true
+IFrames=true
+Tables=true
+Cookies=true
+BackgroundSounds=true
+JavaApplets=true
+JavaScript=true
+CssVersion=3
+supportsCSS=true
+
+[Mozilla/5.0 (Macintosh; ?; *Mac OS X*; *) AppleWebKit/* (KHTML, like Gecko) Version/5.0* Safari/*]
+Parent=Safari 5.0
+Platform=MacOSX
+
+[Mozilla/5.0 (Windows; ?; Windows NT 5.1; *) AppleWebKit/* (*) Version/5.0* Safari/*]
+Parent=Safari 5.0
+Platform=WinXP
+
+[Mozilla/5.0 (Windows; ?; Windows NT 5.2; *) AppleWebKit/* (*) Version/5.0* Safari/*]
+Parent=Safari 5.0
+Platform=Win2003
+
+[Mozilla/5.0 (Windows; ?; Windows NT 6.0; *) AppleWebKit/* (*) Version/5.0* Safari/*]
+Parent=Safari 5.0
+Platform=WinVista
+
+[Mozilla/5.0 (Windows; ?; Windows NT 6.1; *) AppleWebKit/* (*) Version/5.0* Safari/*]
+Parent=Safari 5.0
+Platform=Win7
+
+[Mozilla/5.0 (Windows; ?; Windows NT 7.0; *) AppleWebKit/* (*) Version/5.0* Safari/*]
+Parent=Safari 5.0
+Platform=Win7
+
 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Lunascape 5.0

 [Lunascape 5.0]