nip2 for nerds - an example of adding a nips2 tool

I wanted to try the effects of different types of edge filters with nip2. I had decided to use nip2 because I had already used vips for an image processing program but I did not want to waste time programming everything in detail.

For my edge filter trial, I needed to have an image and several matrices and then effectively call the vips2 function vips_conv(). nip2 does have the facility to do this directly (Toolkits/Filter/Convolution/Custom Convolution) but I was not aware of this at the start.

Lets now start the example:

1. We first need a file to process: Add the file via File/open (file type jpeg). Your nip2 workspace should now look like:

2. Save the workspace! In my experience, this should be done often! When it comes to writing your own tool, nip2 is likely to crash without explanation if the tool code is not correct.

3. We now add a matrix which we will use for edge filtering. Add via Toolkits/Matrix/New/Convolution. Make sure that the new matrix item is completely shown by clicking on the down arrow next to the matrix item. Your nip2 workspace should now look like:

4. This matrix is not much used for a convolution, as it will just copy the image. I wanted to try a 5x5 matrix with particular values. We therefore make a right-click on the A2 button to change the matrix size to 5x5. We also click on the A2.value button to update the contents of the matrix. After this, the workspace should now look like:

5. Now we come to the tricky bit - defining your own tool. Before going further however, I would very much recommend that you read the nip2 manual (http://www.vips.ecs.soton.ac.uk/supported/current/doc/html/nipguide/nipguide.html). Sections 2.3 and 6 are particularly relevant. As it says in the nip2 manual:

The insides of nip2 are built with nip2’s own programming language. It’s a pure lazy functional language with classes. It’s C’s expression syntax (more or less) plus approximately Miranda/Haskell function syntax, plus some basic class stuff. nip2’s main window is a class browser for this programming language. 

 I also found the following links which could help: https://www.haskell.org/tutorial/index.html and https://en.wikipedia.org/wiki/Haskell_(programming_language).  Functional programming is something different for most programmers like me, who are used to C, C++ or Java!

6. The first step is to open the toolkit editor using the menu option Toolkits/Edit. The toolkit  editor will now be displayed. Within the toolkit editor, select View/Definition browser. This is extremely useful as it allows searches for functions with only partial knowledge of the name to be carried out. As an example, type conv into the search window. The following is displayed:

There are two entries here that could interest us: im_conv and conv. im_conv is the direct interface to the vips im_conv() function (declared in the libvips/deprecated/ subdirectory of the libvips installation directory). If possible, we should avoid calling such functions directly. It is much easier to use the equivalent conv function (this could call the vips V8 conv() function direct). I recommend that you become very familiar with using this search facility. In general, it is possible to find a way to do something by looking at these code examples. If you find that you want to grep for something, rather than using the definition browser, then all the standard toolkit definitions are located in the share\nip2\start subdirectory of your nip2 installation directory. Do not try changing anything in this subdirectory! You have been warned!

Let us try and understand how the built-in conv function works.The definition browser gives us the following definition:

conv mask image
    = oo_unary_function conv_op image, is_class image
    = im_conv image (to_matrix mask), is_image image
    = error (_ "bad arguments to " ++ "conv" ++ ": " ++
        "conv (" ++ print mask ++ ") (" ++ print image ++ ")")
{
    conv_op = Operator "conv"
        (conv mask) Operator_type.COMPOUND_REWRAP false;
}

conv takes two parameters: mask (our matrix) and image. The input order is important. We then have 3 guards, that check that the parameters are in order (See §6.3 of the nips2 guide). The last (error) will only be called if the first two are not satisfied, so that we get a (hopefully) meaningful error message. The code in between the '{' and '}' then calls the conv() function in the vips library. Unfortunately my understanding of nips2 does not go deep enough to explain what the "Operator_type.COMPOUND_REWRAP false" means.

7. Now lets try writing our own nips2 tool! I wanted to write a function which extracts edges from an image. It does this by applying a matrix filter to the image to sharpen the image to display the edges. I wanted to test what was the best edge filter (for further information about edge filters, search for edge detection with Google. A lot of the results are fairly mathematical. In principal however, an edge filter is implemented by generating a new image where each pixel is formed from applying a matrix to the adjacent pixels.) for my application. As I have already said, conv does exactly what I want - but that is not the point of the exercise!

7.1. We first need to create a new toolkit for out new tool. In the toolkit editor a new toolkit is created using:

 The "New Toolkit" dialog will be shown:

Here one enters the name of the new toolkit and a caption for the toolkit. It is now necessary to select the toolkit and create the new tool:


We will call the new tool myEdgeFilter. Calling the "New Tool" menu item does not seem to do anything, but we can now start typing in the Definition Window (to the right of the toolkits list).

Here I have typed the complete definition for the myEdgeFilter tool. You will ask: how did I know what to type? I looked at other tools and copied the best bits to get a working tool! Do not forget to save your work regularly!

7.2 We should now try the tool! Create a new column for the tool test using File/New/Column and complete the "New Column" dialog:

We now add the image to be edge-filtered with File/Open and select the file using the "File Open" dialog. The new column in our workspace should now look like:

We now need a matrix containing the filter. The way to create the matrix is detailed in (4) above. In this case, though we will use a 3 x 3 matrix. After adding the matrix (Toolkits/Matrix/New/Convolution) and setting the values in the cells of the matrix, our new column looks like:

We can now apply our new edge filter. Select the image first and then the matrix (using the Shift key). The order of selection is important! Then Toolkits/TestToolkit/myEdgeFilter. Lo and behold, We have now generated an edge-filtered image of the original image!

It is now possible to play with the values in the matrix, to see the effect of different types of edge filters. The sum of all the cells in the matrix should always add to zero. It is also possible to increase the size of the matrix (to for example 5 x 5), to see the effects of selecting a wider range of pixels.

David Singleton
dfksingleton@googlemail.com