Inkspot Software Tools

Defining the service

Wed, 14 Apr 2010 16:05:22 GMT

The proeprties of the service are defined in the file services.xml and an example is below

<WorkflowService>
    <!-- Name of the service, and also the caption that will appear     -->
    <!-- in the top line of the block on the workflow editor            -->
    <Name>CiteExplore</Name>

    <!-- Service description that appears at the bottom of the editor   -->
    <!-- window when the block is selected                              -->
    <Description>Searches CiteXplore Repository</Description>

    <!-- Category to place the service in on the editor palette         -->
    <Category>British Library</Category>

    <!-- Homepage for block documentation                               -->
    <Homepage>http://www.ebi.ac.uk/citexplore/newSimpleSearch.do</Homepage>

    <!-- Class name of the service. This needs to extend either the     -->
    <!-- DataProcessorService or the CloudDataProcessorService object   -->
    <ServiceRoutine>CiteExplore</ServiceRoutine>

    <!-- Auto deployed service. Do NOT change for dynamically deployed  -->
    <!-- services that are uploaded via this editor                     -->
    <ServiceType>AUTO</ServiceType>

    <!-- Data streaming mode for this service. This can be one of:      -->
    <!--                                                                -->
    <!-- nostream   - Data is passed in one block through service       -->
    <!-- sequential - Data is streamed one connection at a time         -->
    <!-- parallel   - Data is streamed from all connections in parallel -->
    <StreamMode>nostream</StreamMode>
    
    <!-- Editable service parameters. These properties define what is   -->
    <!-- displayed in the properties panel when a block is selected in  -->
    <!-- the workflow editor. The format of properties is:              -->
    <!--                                                                -->
    <!-- <Property name="" type="" description="" default=""/>          -->
    <!--                                                                -->
    <!-- Where:     name = property name without spaces                 -->
    <!--            type = Document - file reference                    -->
    <!--                   Folder - folder reference                    -->
    <!--                   Integer - integer paramater                  -->
    <!--                   Boolean - true / false value                 -->
    <!--                   String - text parameter                      -->
    <!--                   Double - floating point value                -->
    <!--                   Date - java date parameter                   -->
    <!--                   StringList - vector of text values           -->
    <!--                   TwoColumnList - two columns of text values   -->
    <Properties>
    <Property name="outputFormat" type="String" description="summary or verbose" default="core"/>
    <Property name="Query" type="String" description="search query string" default="TITLE:Blood" />
    </Properties>

    <!-- Definition of all of the inputs to a service. The format is:   -->
    <!--                                                                -->
    <!-- <Input name="" type="" streaming=""/>                          -->
    <!--                                                                -->
    <!-- Where:     name = name of input also displayed on connections  -->
    <!--            type = data-wrapper - mixed matrix of data          -->
    <!--                   file-wrapper - list of file names            -->
    <!--                   object-wrapper - Serialized Java object      -->
    <!--            streaming = true / false - is this a streaming link -->
    <Inputs>
    <!---    <Input name="Query" type="String"/> -->
    </Inputs>

    <!-- Definition of all of the outputs from service. The format is:  -->
    <!--                                                                -->
    <!-- <Output name="" type="" streaming=""/>                         -->
    <!--                                                                -->
    <!-- Where:     name = name of input also displayed on connections  -->
    <!--            type = data-wrapper - mixed matrix of data          -->
    <!--                   file-wrapper - list of file names            -->
    <!--                   object-wrapper - Serialized Java object      -->
    <!--            streaming = true / false - is this a streaming link -->
    <Outputs>
        <Output name="Results" type="data-wrapper"/>
    </Outputs>

</WorkflowService>

Inkspot Designer

Wed, 14 Apr 2010 15:47:35 GMT

Below is a screenshot for the current version of Designer showing the service panel. Designer make sit relatively easy to add new services and new workflows. The services and workflows are built and tested in Designer but synchronised with Inkspot so that once the service is built it is immediately available in the Inkspot cloud. The process of adding a new service is as follows.

Go to File, select new. The options at the moment are to create a java, Gnuplot, octave or binary service or a workflow. We will add other scripting libraries such as R, Python as we get round to it.

On selecting a new Java service the service editor comes up and we use this to do three things.

Edit services.xml to define the input and output port datatypes of the service and the properties. The input and ouytputs will appear as attachments and arrows in the user interface and the properties will appear in the proeprty tab of the workflow editor
create the service source code file(s) that reds the input, does stuff and writes the output
Add any libraries either to this service directory (in lib) or if you are going to use it for multiple services, create a new library from the file menu and add the library to there

Compile and if successful Designer will synchronise and instal your new service in nkspot

Piece of cake ...

Adding Services ith Inkspot Designer

Wed, 14 Apr 2010 14:19:35 GMT

I have looking been using the new "Inkspot Designer" desktop client for building services and workflows for upload to Inkspot and using the example of search and related functions of interest to Steven Andrews at the British Library. The are several objectives of this work

to create interesting workflows demonstrating technology integration that makes use of the Pubchem Central UK database offered as a web service by the EBI and the ZETOC table of contents search.
integrate these services with other technologies such as plotting and analysis
to test and document experiences with the new Designer client which is aimed at the developer of Inkspot workflow service blocks as well as workflows
to build a workflow that is personally of interest

The work is being done using the development version of Inkspot and therefore I can't link directly from here but I will add code snippets for review and screenshots where I can for review.

The general problem I am interested in the sort of initially vague query an early stage drug discovery project might make that leading to more detailed quantitiative understanding. So for example, I know from some reading I did a few years ago that arachidonic acid metabolism invovles two parallel pathways leading to inflammation nd other effects. One pathway has a drug target enzyme 5-Lipoxygenase that I once worked on and the other has the Cox-2 enzyme that was the target of drug design for alternative pain killers to NSAID's. There was an hypothesis around that a dual inhibitor of Cox-2 and 5-LPO would be an anti-cancer agent. So I want a workflow that automates roughly what I did over several weeks 5 years ago to refresh and update myself on this topic. Broadly speaking what I did was search the BL online catalogues, get the papers and fillet out compounds of interest and more specificaly the detailed Biochemistry (known known as Systems Bioogy) of the pathways involved. I was interested in the dynamics of these pathways and what the control points were as part of an ivestigation into whether the fundamental hypothesis was understandable (by me) and sound.

So that's what I will do. The automation aspect is that the services and the workflow should be reusabe. So some one else might want to use the workflow for something other than arachidonic acid.

The workflow will also demonstrate the integration of 3rd party services via clients with Inkspot hosted services such as Gnuplot, Octave, ... he says bravely.

Ok. Science over for now. Next post is programming

Workflow Hello World

Tue, 16 Feb 2010 02:22:16 GMT

How to create a simple workflow programme using the Inkspot workflow editor. The example uses a very simple case of reading a chemical structure property file (.sdf) and converting the chemical structures into SMILES format. FInally the list of structures is written out to a file for display.

The workflow tool integrates different technologies making it simpler for the user to build programs from components without having to learn lots of different languages. Inkspto already has a large collection of workflow components and this is expected to grow rapidly as new tools for adding software components are introduced and the user base continues to grow. Hello World for Inkspot Workflow

Adding a post with source highlighting

Mon, 15 Feb 2010 10:35:26 GMT

This is an example fo how to add a post, with a linked image and source code highlighting

Creating a Lab Notebook (Blog)

Mon, 15 Feb 2010 10:33:27 GMT

This is the third and shows how to create a new blog or lab notebook such as the one I am using now

Data and File management

Mon, 15 Feb 2010 10:20:37 GMT

In this second screencast, I go into more detail on file uploading and control of permissions

Feature Screencast

Mon, 15 Feb 2010 09:49:30 GMT

I have been making some screencasts showing how to use Inkspot and what can be done with it.

The first one is a general overview of the site how it can be used to give scientists collaborative access to cloud computing. It shows examples of file, communication and software hosting in a secure, collaborative environment

File Properties

Sat, 09 Jan 2010 13:49:33 GMT

I am just getting used to some of the collaboration tools we've built in to inkspot and rellay starting to like the properties page for any file or other object which manges multiple versions as well allowing you to add tags, linked files/objects, meta data, permissions and other stuff. These properties work for an y object including blog posts, workflows and code blocks of course. Here's a screen grab

ALOGP from the rcdk library

Sat, 09 Jan 2010 12:59:59 GMT

Here is one of the first workflow blocks I wrote using R. Very simple, the rcdk library does all the hard work of parsing the SMILES, creating the molecule and calculating LogP. I just have to read in and write out to the workflow ports correctly. SHould get even easier when Hugo delivers the new workflow block upload/definition tools

#open the rcdk library
library(rcdk);

#get the smiles from the input port of the workflow block
input <- input_text;

#parse the smiles to create a cdl molecule object
sp <- get.smiles.parser();
mols <- sapply(input, parse.smiles, parser = sp);

#add hydrogens
lapply(mols, convert.implicit.to.explicit);

#calculate ALogP value and pass to output port of workflow block
alogp <- data.matrix(as.double(sapply(mols, get.alogp)));

Model Assessment

Sat, 16 May 2009 07:01:40 GMT

Roc

http://en.wikipedia.org/wiki/Receiver_operating_characteristic

odds ratio

http://en.wikipedia.org/wiki/Odds_ratio#Worked_example

http://bio-geeks.com/?tag=roc

How to add an R service to Inkspot

Fri, 01 May 2009 10:15:17 GMT

The example workflow shown as HSA_HERG uses a block created from an R script. This particular R script uses the "rcdk" package by Rajarshi Guha for cheminformatics and the workflow has one standard block for accessing a file, in this case an .sdf format file downloaded from QSAR World and containing chemical structures and Herg toxicity data.

The block readSDF is written using an R script using the rcdk library and reads an sdf file to produce two outpots, a data matrix of text, the smiles of the chemical structures, and the values in the sdf. The block allows the user to define a property value and in this case it is the sdf tag that describes the property values in the file.

To write a script go to the Apps tab and open the Service Editor and click "Add new service" to launch the service editor.

Add in the name of the new service, a text description. YOu overwrite the "My Services" entry if you want to locate the service under a different heading. Addd in the invocation url which locates the RServe service that will run your block, and give the service a name in the final field entry.

Click on the external script button and the R service which is what we are using in this case. Note that Inkspot will be ading more scripting engines for Octave, Python and others in the near future.

Create a new script file by clicking on "New" and write the R script that you would like to use. In the case of readSDF the script is shown here. What it does is load the rcdk library and parses some molecules by reading the sdf from an input file produced by the previous block using the "load molecules" function from rcdk.

The script then converts the molecules to a matrix of smiels strings for output into the next block. It also gets the property values from the sdf and passes them as a second output.

# Load the R-CDK Library
library(rcdk);

# Parse some molecules
mol <- load.molecules(infile[1]);

# output as smiles
smi <- data.matrix(sapply(mol,get.smiles));

print(smi);

# output prop value
prop <- data.matrix(as.double(lapply(mol, get.property, key = label)));

Save the script file with a menaingful name. We then need to define the input and outputs for the block as well as any properties that the user can define.

Select "add new input" and give it the name you used in the script, in this case "infile". It is a file wrapper so select that radio button and save.

Select "add new output" and add the outputs, in this case there are two, both data wrappers and they are "smi" and "prop"

Finally, we click on "Add new property" and fill in the editor form. In this case it is a string "label" whihc is used in the script to define the proeprty label. It is a string and we give it a default value of "Label".

Save the new block and it is ready for testing.

If you know return to the workflow editor you should see the block in the services list, under the heading you used. You may need to click refresh. You can test it out on some real data files.

To help with testing, you can add some print statements. The output can be viewed by clicking the small magnifying glass on th ebottom right corner of the editor window.

Note that you are allowed to pass variables as matrices of doubles or of text. In the case your input is text simply append the input variable name as "input_text" in your script.

R Scripting

Fri, 01 May 2009 09:10:16 GMT

R is the leading open source software package for statistical analysis with a very wide range of sophisticated data handling, analysis and graphing components. Inskpot supports R scripting and the integration into the workflow and other tools which mean work in R can be integrated within a wide range of other tools and such R scripts can be shared as with all Inkspot objects.

A number of worked examples of R scripting partly taken from introductory books such as Everitt & Hothorn and Crawley will be developed here.