Building Ontologies with Tawny-OWL

Today, we will have some taught material, particularly at the beginning and the end, where I need to describe parts of the system, some practical time where we build a small ontology using tawny. Then I have a reasonably flexible tail where I try and address questions that people have. Also, there is a degree of guesswork into how much material I can get into the time available, which this helps to address.

I do not have time to describe all of the features of tawny, nor give a full tutorial.

Although the tutorial does "build" I have tried to make it as loosely coupled as possible, although the truth be told, I have probably not been entirely successful at this.

The latter half requires more programming knowledge, and there is probably little that I can do about this. I will skip over what some of it means, in the hope that those who care can go back later and look it up. I do not want this to become a Clojure tutorial.

We didn’t start out to develop a new ontology engineering tool. It happened along the way, as we tried at address a specific use case, which was modelling human karyotypes.

Technically, 1000 terms is not a problem for protege, it can scale to this size (or, indeed, considerably larger) with relative easy.

But the UI doesn’t scale in this way. It involves an awful lot of clicking — one report I have heard suggests that protege users spend upto 50% of their time expanding and closing the hierarchy. With the karyotype ontology this problem would be profound. Worse, with the karyotype ontology we have a specific computational use in mind, and we don’t know what the performance is going to be like — reasoners can change performance quite a lot with different axiomatisations.

In practice, we are more likely to end up like this; 1000 classes is an awful lot of clicking, particularly when many of the classes are very similar.

The main API out there is the OWL API. It’s nice, but is long winded, and difficult, both because of the complexity of a type system needed for OWL (from the javadoc it is hard to work out which methods can be invoked on which type), the change object system (so, you can use AddAxiom to add an annotation to an ontology, but only if you don’t care about it working), and the factory layer. All complex.

Another option, writtem by Samuel Croset is Brain. Much lighter weight than the OWL API. But EL only, and it is unclear how to marry what is essentially a script with Java’s OO design. And still we keep the essential characteristics of Java. Any changes require, recompile, restart: it is slow.

So, my paragon here is R, the statistical language. It is interactive, convienient to use. It can be used cleanly in batch. In general, very nice. Not to say that I want to copy all of its features though.

These are the limitations that we had to live within. Most importantly of all, I wanted it to be as simple as possible to build structurally simple ontologies. It should largely be possible to type and write ontologies without feeling that you are programming.

It was going to be written using the OWL API because there is too much code there to rewrite, and no one would trust me to do that in a standards compliant way.

This required the use of the JVM.

And I wanted access to pre-existing development tooling. I did not want to build a complete development environment, I needed something off-the-shelf, so that it was good.

Before I move onto tawny-owl, what did we achieve with the karyotype ontology. Well, I think quite a lot. We now have a large, consistent (in both the formal and informal sense of the word) ontology that describes most levels of the ISCN.

Next, we move onto a formal walk-through of tawny-owl features. In this section I do not intend to describe all of the features in detail, but to give an overview, so that you will know what is coming up.

This is a literately programmed document, so we start with a namespace, but I have hidden this from the slides because we do not need so much baggage so early on. It will be introduced in detail later.

For those of you from a functional programming background, tawny is not very functional. It is usuable as an API for manipulating ontologies, but was not really designed for that purpose. Still, it is no worse for this than the OWL API.

We have aimed as far as possible to make tawny simple to define simple ontologies. So this statement for instead defines a new ontology. Of course, the choice of programming language that we have chosen has implications and the parenthesis is the most obvious one to anyone from a lisp background.

I did not want to create my own syntax because there are really far too many of these already. One which was built for the specific use case of typing was Manchester syntax (also known as "OWL Manchester Notation" or OMN). It is a relatively clean syntax, and can be used to define new classes easily.

In my very early work on the karyotype ontology, I even considered writing an OMN environment, but OMN is not that easy to deal with programmatically, so I dropped the idea.

This is the equivalent in tawny-owl. Some things are easier, and some are harder.

Being a programming language rather than a format it is relatively easy to add new features with a clearly defined semantics. So, for example, I wanted to add a "sub" keyword so that I could build ontologeis bottom up. In practice, so far I have not really used this, but I do not feel that the syntax should dictate the ontology development style.

There are some other differences. Firstly, tawny has (two) comment syntaxes. OMN is commentable too but the parse doesn’t always work.

The other major one is the use of an "explicit definition" semantics. Classes must be defined before they are used by other classes. This is a semantics shared with Brain, and was chosen deliberately. My early experience with OMN showed that it was too easy to make typo’s without.

It is possible to avoid this if you wish.

General Concept Inclusions are fully supported which OMN doesn’t do.

We can also build patterns in the same syntax and files as the ontology. You will see many examples of this through the tutorial.

We can type, change and add new entities and reprogram things as we go. For the non-programmers this is likely to be so obvious that you cannot see why I am describing it, but for programmers from a Java background it is hard to under-estimate what a massive difference this makes to development styles.

There is no compile cycle — you can change things as you go, and you do not need to continually restart your application.

This is not entirely true, you do need to restart, but not that often.

Actually, there is a compile cycle. So those of you from a programming background, may be thinking "hmm, an interpreted language running on top of a JVM. Actually, clojure statements are compiled to bytecode on-the-fly them run directly on the JVM (which in turn will JIT compile them). So, it’s fast enough.

Broadcasting is a really very handy feature from R. You do not have to explictly deal with the lists and numbers (ontology entities in the case of tawny).

One statement in tawny unwinds to two in OMN. Two calls to the OWL API also. It is also one of those things that makes tawny less like an API and more like a textual UI. Although this is reasonably efficiently implemented, it does have a performance cost — more than made up for in saved typing for ontology developers.

This is the first example of a pattern that we see (although broadcasting is a pattern also, in a sense. This is the "some-only" pattern which is so common, it often not seen as a pattern.

This was also the motivation for broadcasting as some-only makes little sense without broadcasting, although it might not be immediately obvious why this is the case.

In theory, tawny does nothing that it is not possible to do already. But the single syntax and environment is important. I can easily add new syntax even for a specific ontology. Doing this where half the ontology is build in protege and half outside is just intractable. With a single syntax it becomes so easy that it happens often and all the time.

Most of this I am not going to show anything other than implicitly, but tawny is based on a commodity language. So it access to many APIs which can do useful things for you. We have used quite a few of these either in the context of programming tawny or in developing OWL ontologies using tawny (in fact all of those given here).

The key point to remember here is that programming tawny and developing ontologies are not disjoint. You have the same power in using tawny as we do in developing it.

Finally, we have full access to a rich tool chain, including a wide range of IDEs, power-editors or web editors, as well as some very novel environments (take a look at lighttable — implemented in Clojure and supporting it first).

We also make extensive use of version control — we’ve been using git, but you can use what ever you want. You can integrate your ontology development process and software development process.

Dependency — we’ll see later how to access ontologies using the maven dependency management system. Some one else can host your ontology without having to use their URIs!

Testing, continuous integration. Remote evaluation (actually, you will use this all the time even if it seems you are not).

Linters. Rewriters. We have only just got started with these.

While Tawny-OWL does come with many significant benefits as an ontology development, it also comes with one significant cost. It requires the use of a Clojure development environment. In this short section, we will get "hello world" up and running.

I could have opted to use a shrink-wrapped IDE. But while these provide a very rich environment, they also come with an enormous amount of baggage, and give a very programmatic feel to the experience. They also tend to use a lot of resource, which might not be ideal on laptops.

More over, the practical reality is that I do not use one myself and can only give partial help on them.

Instead I have opted for catnip. This is a small, light and simple clojure editor, which was primarily designed as a tutorial environment for Clojure. It is, itself, a Clojure application and works quite nicely.

It is not, of course, a fully featured environment. If you want to develop seriously, them you’d need to move to something else.

It does, however, reinforce the point that we are using a commodity language. There are many powerful tools around.

The slides and tutorial material that I am showing are all developed in files with names _s.clj, but I am providing cut-and-paste material for files without the _s.clj. This is the real namespace declaration. The false one comes next.

The Clojure namespace declaration is the only "programmatic" part of tawny that you have to see. Rather like Java, Clojure namespaces are consistent with the filename (although dashes are replaced with underscores for strange reasons). Most development environments will put this in for you.

We also add a statement to say that we wish to "use" tawny.owl. This is a file local import, and it will occur a lot!

Next we define a new ontology. It has a name that we can use to refer to it, which is a useful property as we shall see, although most of the time, we do not have to. For the defontology statement everything except for the name is optional, although there are quite a few frames more of which we will use as we move through the tutorial.

The name is also used as a prefix when saving the ontology (although this can be overriden), so I tend not to use hyphens, as it messes with syntax hightlight for my Manchester syntax viewer.

Now we define a new class. As we can see, by comparing against the defontology, statements, we have parenthical statements all the way through. This form of statement is also known in lisp speak as a "form", an "s-expression" (to distinguish is from m-expressions which don’t exist!) or rather more obscurely "sexp".

One key point from a programming point-of-view, clojure is a lisp-1. All variables are in the same namespace, and that includes all the ontology entities that we might define. It’s easy to clobber one with another so be careful!

Properties use defoproperty. There are also annotation and object properties, and I have opted for somewhat opaque function names to avoid a large amount of typing. Hard to know whether this was a great decision or not, but the alternative really seemed various unreadable to me. More over, there are only a few of these names, and OWL 3 is unlikely to come along any time soon.

Finally, we bring in a frame. Frames are introduced using "keywords" — that is something beginning with a ":" — this is a fundamental part of lisp syntax (it is one of two ways of defining self-evaluating forms if you are interested!), which by good fortune is also very similar to the way that Manchester syntax does it.

Hopefully, most people are familiar with the meaning of "some" in this context.

As we move through the material, you will notice the "owl" prefix popping up. Although, clojure includes full namespace support clashes with the clojure.core are still somewhat of a pain. Therefore, I have avoided them in tawny.owl. In this, I have followed the OWL API. However, I have not done so consistently; the OWL API does by putting "OWL" at the beginning of everything.

There are also options. We have symbol short-cuts which should be obvious to programmers. We have an owl-only function which I used to use because I used to forget which is the clojure.core function and which was not. And I have consistent namespace called tawny.english, but this requires using the namespace mechanism of Clojure in a way that is slightly more complex than I wish to describe.

To check my axiomatisation I often save my ontology in manchester syntax — I normally always use the same filename (makes using git, and .gitignore easier). And I have my editor auto-reload. We will be looking at how to build an auto-save function later.

Finally, let’s extend our definition somewhat. We could, of course, just change the defclass statement, but I wanted to introduce the refine function which allows addition of new frames to an existing definition. This is quite useful. You can different frames for different types of entity, but only the correct frames for each type of entity (although many frames overlap).

Tawny has a number of convenience frames that have no direct equivalent in OMN. :label adds labels in English, and :comment does the equivalent. Obviously you can specify any type of annotation you wish, and tawny can build internationalized ontologies perfectly well.

Defining the basic tree is not too hard — we just use the :super keyword. If you think this involves too much typing, yes, it does.

In theory, this should crash, but it may not if you have previously evaluated (or saved in catnip) the Asparagine form above. This reflects one problem with REPL based languages — sometimes your source can get out of sync with your REPL. If it does not crash now, it will crash when you restart and eval again.

Most REPL programmers restart defensively every hour or two to guard against this.

Explicit definition is a good thing, but does hold out the possibility for being a real pain. We think that it is not so in Tawny, because we have features for working around it. It is possible to avoid entirely anyway, but I personally do not, because it makes spelling errors all too likely. I will show this once later on.

Before we show the solution to the disjoint problem, we simplify our definitions. Typing :super AminoAcid a lot is a pain also, so lets avoid that. We introduce a new function, as-subclasses

The advantage of lexically grouping all of the subclasses in this way is also that it makes the intent of the developer obvious. If we need to add a new subclass later (unlikely in this case once finished, but during development yes), then adding it next in the list also makes it a subclass as it should be.

This ability to make the developer intent, and conformance to a pattern explicit is a good thing!

Now that we have achieved this, we can also solve the disjoint problem, by adding a keyword in. All the classes will now be made disjoint. If you want to be sure, evaluate the source, save it and look at the OMN.

We can add a covering axiom in the same way. Unlike disjoint this is not a formal part of OWL, but is a design pattern. The interesting thing about this is that biologically it is true — there are only 20 amino acids and we will name them all. But to a chemist, it is demonstrably false as there are many, many amino acids. How we scope the ontology and frame our competency questions can very much affect the model that we build. We will show later that this axiom has a very significant affect on the end model, much more significant than you might presuppose at the moment.

Putting in a covering axiom, then adding a new sibling and forgetting to modify the covering axiom is an easy mistake to make, and can be very difficult to pick up, either by eye or by testing. Tawny makes this harder.

I have not shown all the OMN because it is long and tedious, but these are the keypoints added by the subclasses function.

If you evaluated the :disjoint Alanine definitions early (and they did not crash!), then you will find that there are some Disjoint: frames on individual amino acids also. These make no semantic difference and are an artifact of the tutorial.

We start with our usual duplicated namespace.

It is, of course, possible to model continuous values as data type properties and just put the numbers in. This works to a certain extent and is an option for modelling.

For full details and a discussion of the advantages and disadvantages of this approach, the value partition is described in this recommendation edited by Alan Rector.

http://www.w3.org/TR/swbp-specified-values/

Putting "Size" under a ValuePartition superclass would also make sense.

In a "real" ontology it would be good to add annotation properties and comments describing exactly what these partitions mean.

See https://en.wikipedia.org/wiki/Faceted_classification for a description of a facetted classification.

Facets are a relatively new feature of tawny-owl and I may come to regret the name as "facet" also has a meaning wrt to the OWL specification (it’s a feature of a datatype property). The tawny-owl facet is unrelated to the OWL specification.

I had a number of options for the implementation of facets. It would be possible to do this without having them visible in the ontology, but this seemed to the best way forward. Happy to solict opinions on this.

We convert the facet into an existential restriction.

We have to add the as-facet call so, overall, the reduction in typing is not enormous, although it’s a fixed cost, while the saving is linear with the number of amino-acids, so the real saving would be a bit greater.

Now we can see the full saving advantage. Instead of four separate statements, we have a single one.

The advantage goes beyond typing, of course, this also is an advantage in the consistency of building our ontology. We cannot associate the wrong class with the wrong property. In this case, we have set the logical axioms up such that if we did use the wrong property, the reasoner is likely to pick the complaint up anyway. But this is immediate and at the point of use.

I missed half the axioms of Polarity in the last set of examples. Believe me or not, but I did not do this deliberately, it was a genuine mistake; as should be obvious from the existence of this note, I did leave it in deliberately once I discovered it!

As usual, we start with the real/false namespace

This is syntactically similar because it’s just a new function defined in the language. We are adding nothing clever here just using a language as it is intended to be used.

We have to inherit directly from the web, because all the software that we are building on depends on it and/or requires it.

IRIs have this unusual characteristic of being global.

Symbols are a core feature of Clojure and, indeed, any lisp. They are kind of equivalent to variable names in other programming languages, but not exactly the same since lisp gives you flexibility to handle them directly.

Using symbols also provides a set of other advantages — in addition to the define-before-use semantics, they are normally syntax highlighted specially by editors and, rather usefully, they will normally auto-complete ("code-complete" or "intellisense") in an IDE. Very useful for big ontologies.

If we do nothing else, tawny identifies an ontology using a random UUID. This is not entirely best practice and, indeed, is illegal for some serialisation formats. However, the OWL API supports it, and it’s useful where we are playing or giving a demo.

If we define a class, we use the ontology IRI to form the entity IRI, just adding the symbol name as the fragment. Note in this case I have used the :ontology frame as we have multiple ontologies in one file. This is the safe as otherwise the results depend on the order of evaluation.

To see the IRI, type "A" into the REPL after evaluating the two forms above. The UUID is random, so it will be different each time!

The "[OWLClassImpl]" stuff comes from the OWL API, and is just the "toString" method. It’s not great and at some point I will fix this.

This is a pretty pointless transformation!

With protege or another GUI, we can use the underlying identifiers and display something different to the user. With source code, we cannot. Clearly something like the example above is just not acceptible (although it is actually legal tawny code or lisp).

In this case, we have totally dissociated the IRI from the symbol. The IRI is not auto-generated here — it is stable, and will come out the same every time and on every machine. You should get the same IRIs exactly.

Interestingly, when we get to here, we get a strange ID with a UUID for fragment and an IRI that it unrelated to anything.

The mapping file has been designed to work with version control, because it needs to be shared between all developers. Although it is a generated file, it is source code, since it cannot be recreated from fresh (not in the same order anyway).

EDN format is a Clojure thing. It’s basically a Clojure read syntax.

While the preiris are automatically created, if we choose they can be made stable by simply saving them into the file with the form above. This can be safely done every time the file is evaluated, because the order is deterministic, so it will cause no false diffs in versioning.

This is potentially useful if you are collaborating with others and want to co-ordinate at pre-release time. It’s not essential if others are using tawny — there is no need, since classes can be refered to by symbol.

There is a potential disadvantage. This creates an IRI (and entry in the file) for every new entity created. Not a problem with protege, but tawny is fully programmatic. I can create 10^6 new classes in one line of code.

pre-iris all appear at the end of the EDN file!

At some point, you need to coin new IDs that will become permanent. This has to happen in a co-ordinated fashion. It could be done as part of release. Or by bot during continuous integration.

How well would this workflow work in practice? Not sure. It would work for a small number of developers. There are many tweaks that could be made for different scales — saving to multiple files, pre-iris in one place, perms in another. No pre-iris at all. Use URIGen. Manually coin permanent IRIs as part of quality control.

It’s programmatic and easy to change.

We can do without symbols and instead use just strings. Note that we also have to switch functions owl-class instead of defclass.

The use of strings means that we can define things without the tawny-name being in "primary position". It just happens. You need to be careful.

The main reason that I have left this in place is for use as an API. Clojure allows full manipulation of symbols like most lisps, but it’s a bit of a pain. It’s not possible, for example, to concatenate two symbols to make a longer one (or rather, they need to be converted to strings, then concat’d then converted back again). And the creation and interning of new symbols as variables requires the use of macros rather than normal functions.

Having said that, tawny does offer some facilities to help with this process

We will see a few of these later.

At this point I would like a rant. There has become a widely held believe, typified (although not solely caused by) the believe that we should always reuse terms from another ontology if they exist. And that this cross-linking is necessarily good.

But this is not true. It is as at least as wrong in the ontology world as it is in the software world. In software, reusing a library in your own is called adding a dependency. You become dependent on it. You can be affected by changes in it. Your release process can be affected by its release process. If it rots, your project rots. If it is insecure, yours is insecure.

Not only this, you become dependent on its dependencies also. And, in fact, the transitive closure of your dependencies. This project has over 70 dependencies. And these can change over time. While in "SNAPSHOT" mode, the dependency graph of your project can change overnight, without any change to your code. And you have multi-path problems, "OWL Hell", where one ontology can be imported multiple times. And no one knows have to deal sanely with versioning.

And this is just the software issue. With ontologies the problem is worse. You are making an ontological commitment. And it may not be one that want, it may not be one that you agree with, and it may be one that contradicts your actual use case.

Do not go contiually looking for terms that you can use and reuse. Consider searching for terms when you realise that you are getting too far away from the core requirements for your ontology, from your competency questions. If you really like someone elses definition, cut-and-paste it, give it your own identifiers, and comment that you have done so. Maybe, once you have five or six terms from the same ontology, then consider importing the other ontology. And don’t blame me if it goes wrong.

I did strongly consider combing the use or require step into the import; this would have been (and still is for anyone who wants to code it!) entirely possible. However, there are perfectly valid reasons not to.

Without the import statement, we gain access to the identifiers from the require'd ontology. And re-using the identifiers without using all of the axioms from that ontology allows us to do some useful things. For instance, MIREOT does exactly this. Or for a different take, consider my idea for Ontology connection points.

Here, we are reusing basic clojure functionality and it’s namespacing mechanism. That the symbols refer to ontology terms really makes no difference.

If you get bored of typing, then you can also alias lisbon.abc to a shorter form, as we saw earlier with clojure.string.

It is possible that you want access to an OWL ontology that was not developed using Tawny-OWL, but by some legacy editor like Protege?

Normally, you read an ontology into a namespace with nothing else in it, if for no other reason than to avoid name collisons.

Also, note that we are using the OWL file from local, which gives us a degree of flexibility — you do not want to download GO everytime you restart the REPL.

Although not covered here, defread is highly configurable. You can filter just the terms you want, change the names as you chose.

Having read our ontology this now gives us the ability to refer directly, with symbols. So, we can type A or B. This is safe, and has all the advantages of symbol based definition.

I’ve picked an autosave because it is a nice general function that we might want to use and demonstrates some of the possibilities. But ontology specific is perhaps most useful of all, because it allows ontology development groups to tailor Tawny to their own development practices without having to generate bespoke extensions for Protege or equivalent.

In general the use of :use clauses is falling out of favour in clojure circles; it may disappear in future versions (although it will be replaced with something else somewhat more verbose). The reason for this is that :use is a hostage to the future. If I use a namespace and it gains a new function, we could end up with a name collison. So, if clojure.core adds an only function, it would require changes to any namespace which use'd tawny.owl

For my own use, I think that with tawny this risk is worth it (and it is easy to fix if it happens). For ontology namespaces (i.e. those in which I define an ontology not much else), I tend use tawny.owl. For namespaces in which I want to extend tawny.owl, I tend to require tawny.owl normally aliased to o.

The name collison between import and owl-import is sort of an example of this problem.

Clojure was designed for concurrency and generally does not allow changing variables (although it does allow re-evaluation). Instead it has a set of objects which can store any value and which can be changed but which require the developer to make an explicit choice about how to deal with concurrent changes.

It’s very nice and very sensible, but largely we just ignore it here. The reality with tawny is that the it’s build on the OWL API which is mutable, is not thread-safe and is not build for concurrency.

This is our actual listener. Proxy objects are available in the Java core, and the Clojure ones do much the same thing — implement an interface on the fly. They are rather more common in Clojure because a) it makes more sense in Clojure and b) implementing a new class is a bit of a pain (although entirely possible).

Proxy objects work through reflection and have the performance characteristics that you would expect — but this is meant to be an end user function, so we really don’t care. Saving the ontology is likely to take far more effort than a reflective call.

Clojure follows the scheme convention of marking mutating functions with a !. Tawny doesn’t because we’d add ! to everything. Generally speaking reset! is considered rather bad form — Clojure programmers only change an atom by applying a function to the existing value.

Dropping the existing value is, of course, wrong and a memory leak, since the existing listener will be held by the manager, and may even result in strange behaviour.

It’s a demo!

I’ve used Clojure’s java interaction extensively during the development of the Tawny and I have to say I found it to be very nice. It fits very comfortably with normal Clojure development.

In general, I hide the existance of the OWLOntologyManager, and the OWLOntologyDataFactory. Generally, there is only one. There are more flexible approaches, of course, but I have had to make decisions in the development of tawny. Otherwise, pretty much every function would require a Manager, or Factory and an ontology as a argument and tawny, as it stands would have become unusable.

You might be wondering why owl-ontology-manager is a function call and not just a variable. Actually, it is because of the latter — I have so far found one occasion where I want multiple managers which is for protege integration. Turning this into a function allowed me to monkey-patch tawny for this purpose.

It would be possible to deal with this much more formally, and pass an environment, probably integrated with the "default-ontology" functionality of Tawny. But I have not found a strong use case for this yet.

And finish off. If you don’t know what @ and dereferencing mean, it’s really not interesting.

So, this has been a very rapid run through of how to integrate directly with the OWL API and add new functionality to Tawny that cannot be achieved through tawny itself.

This is very commonly done in Clojure (where the mantra is do not wrap if you do not need to) and so it is well supported.

This is going to be a highly advanced, showing an high programmatic use of Tawny. In this course of this we will generate some brand new syntax — this demonstrates one of the uses of tawny — while it is harder to generate this new syntax, than just use existing, once you have done it is easier to use.

There is also some interesting biology and an interesting ontological question that comes out of the end of it.

Probably this is an extreme example of use. If I were doing this normally, I would almost certainly require pattern, reasoner and util through an alias.

Building amino acids by defining a class for each is no good at all, as it’s too long. Also, it’s too risky. So, we are going to expand the syntax, so that it will work well.

The function for making a new amino acid is relatively simple, as these things go. It just passes off most of it’s work to owl-class. We could also add "AminoAcid" as a superclass here, but I chose to do this later for reasons that should become apparent.

This function does not intern — we can define a new amino-acid like this, but the entity would be a string and we cannot refer to it afterwards as anything other than a string.

We cannot just replace owl-class with defclass to achieve this. The explanation requires knowledge of lisp, but it is this: because amino-acid is a function it’s arguments are evaluated so we cannot pass a bare symbol — Clojure will crash. More defclass is a macro, so it will be called when the amino-acid is evaluated NOT called. We have to make a macro to do this.

In fact, defclass, defoproperty and the rest are all defined in this way. This is a common enough thing to want to do, that I made this macro public. It is an easy way, for instance, to add new frames or set default values for existing frames, or to define patterns.

defentity does one or two other things as well, chiefly adding metadata to the var created. If you don’t know what this means (unless you know clojure you probably wont) then it really isn’t important.

Better than creating one amino-acid, let’s create all of them at once. This function does two things — it calls the amino-acid function over several definitions. And secondly, it bundles the return value with the name of the entity (which is the first element of the definition). We use this next.

We want to transform a bunch of symbols into strings because we are going to use symbols we have not defined yet. We are missing the fact that the symbol name and the string are equivalent here, so we could do this better, but the name-tree function is too handy here. intern-owl-entities achieves the same thing as defentity.

Defined subclasses are the heart of reasoning in OWL. Effectively, they form queries and they tell us useful things.

So, having done one, surely we should do more. So here are the next two. Here we are combining two.

We can build a function to replicate this. We use some string manipulation for this to generate the name. I have not done the interning here — I leave this an exercise!

We now call actually run the cartesian product. We add "nil" so that we get single, double, and triple as well as full length products, and filter for nil to get rid of them again.

Tawny support a couple of reasoners out of the box, including Hermit and ELK. Here we are instantiating using a :keyword, but this is just a short-cut — any OWL API OWLReasonerFactory can be used directly.

The reasoner is invoked to check consistency automatically. Tawny uses a GUI (a progress bar) by default to show this process, but falls back to text if that is not possible (so you can check consistency in a CI environment without hassles.

Working out what has happened can be quite hard (this is something that we wish to fix in future versions of tawny), but counting subclasses work as well as anything. We now have a lot more inferred subclasses than asserted.

Protege is really nice for visualising ontologies. So, we use that here. I always save to the same file name, but better to save as OWL rather than OMN because it parses more correctly.

The reason that this happens is obscure, perhaps, but the base reason is because we have many more defined classes than we have primitive ones. So, we must have equivalences or unsatisifiables.

This only happens because of the magic :cover axiom. We know all of the amino acids that exist --- as there are no hydrophobic charged amino-acids (as the two facets are not independent for obvious reasons of chemistry!), that class becomes inconsistent. Without the :cover axiom, the reasoner would assume that this class could exist but we have just not mentioned it.

So, the reasoning is telling us something about the biology — and whether we want this form of conclusion depends on whether we are talking about biology or chemistry — after all if we were a chemist many amino acids could be created that separate out of equivalent classes, and many make some unsatisifiable classes satisfiable.

Life is complex but, in this case, simpler than chemistry.

One of the reasons for the complexity of the OWL API is that it allows annotations to be passed in lots of places, including on the axioms that assert the relationship between, for example, two classes. One simplification I made with Tawny is to hide this complexity. Moreover, Tawny is frame-centric so the axioms are not normally seen explicitly.

Unfortunately, it appears that in hiding the complexity, I had also hidden a capability that people actually use: GO uses axiom annotations for provenance for instance. So I have added this capability into tawny with a slightly expanded syntax.

The annotate function returns an object encapsulating and OWL API entity and an annotation that will be added to the axiom of the frame. We can see this in the example given. Note that we are annotating neither Person nor Man in this case, but the relationship between them.

We can deploy ontologies to bioportal or to anywhere else exactly as we do now. Save the OWL file, and do stuff to it manually.

Of course, in a programmable environment, it is also very easy to add additional deployment technology — so generate one or more files, copy them to another location, via ssh or http, check them in somewhere. This can be achieved either in the project source or as a leiningen plugin.

Or, finally, we can deploy our ontology like any other piece of Clojure code, as a maven artifact, to maven central, to clojars (like maven central but for Clojure), or to our own private repo. This also separates our the ontology identifier from the download location. It’s possible to argue whether this is a good thing or a bad thing.

Tawny-OWL uses a line-orientated syntax and what you edit is source code, not some XML that has been generated. So, it works very well with version control. The serialisation order is entirely predictable because there is no serialisation — it’s source code, it only changes if you edit it.

Working with source code also means that diff tools show you changes in the same form of the code that you edit, so it’s very easy to compare two versions, two branches what ever. The only exception to this are the EDN files used for numeric IDs of OBO identifiers. These have been designed to version well (they are generated but not regeneratable, so should be regarded a source code), but only time will tell. "They do not version well" will be considered to be a bug though, and will be something I would want to fix.

We’ve used git for all the various ontologies we have developed, and it works nicely. Actually "nicely" is an understatement. The move to modern version management, rather than anything bespoke build just for ontologies makes an enormous difference. For my money, it’s reason to move to tawny all by itself.

It is also possible to version in the sense of release tagged versions of an ontology and to use a dependency mechanism.

We test our ontologies explicitly and sometimes very heavily. Tawny-OWL provides some fixtures to make this easy and we use core.test but there other test frameworks and we might move at some point. This has been written up as a paper, and I would suggest reading these for further information.

Once you can test, then you can continuous integrate. We do this with Travis-CI which is nice and it supports Clojure and Leiningen out of the box (er, cloud). It also continuously integrates with the software environment (including tawny). You can reason on there as well (tawny works headless just fine).

If you import ontologies via URI you are totally dependent on them being stable or you will not get a repeatable build. It’s probably better to import ontologies via their version IRI anyway to ensure this, although it’s not widely done.

OWL raw does allow documentation but it’s poor. With annotation properties you have no structure at all unless you use microsyntax. Moreover most annotations are sets — so no order.

We have been experimenting heavily with literate programming tools. I started this quite a few years back, but they now work well, and we have build a specialised tool called "lentic".

Collaborative development is not a new requirement and is, in fact, the default for some environments. Just use existing tools. Git if you want asynchronous development, or floobits, or even a virtual machine, tmux and Emacs in the console. What ever.

The point is, it’s not a problem for tawny. It’s a problem for many software engineers world-wide and they have provided some very, very slick solutions.

It’s very easy. Tawny programmability means that you can also support a default language if you chose to, and have your ontologists use their own native language for all parts of the system. Our example of etichetta above is one of the complicated examples — tawny has to know the default language also. In most cases, we could just define aliases (see tawny.english for examples). tawny.polyglot uses property bundles which I believe integrate well with most machine-supported translation environments.

The problem here is that we have to do something to get readable names for OBO style ontologies. But we are now using a part of the OBO style ontology that is open to change with, perhaps, fewer guarantees than for identifiers.

Tawny has support for this. It’s solves the problem by saving the mapping that it creates between a label and a URI. If the URI remains, but the label disappears than tawny adds an alias and deprecation warning.

We do have a methodology for doing this. We can render most of the ontology automatically, which provides the basis for this kind of port. But actually making use of the advanced features of tawny (like patterns or the as-subclasses functionality) is manual. In general, at least some of this will be necessary although it can be done manually.

In short, it’s fast enough that you are probably never going to notice it. We cannot currently test how much difference the patternisation would make, but it might be substantial.

It is also worth noting that with an ontology the size of GO, iff it were developed in Tawny, it would be unlikely to be single file. Interactively (i.e. in the editor) you probably would not be loading the whole ontology most of the time anyway.

This should work nicely and it does, but the truth is that at the moment a REPL opened inside Protege hangs periodically and I do not know exactly why; I suspect it is that Protege is not entirely happy with having it’s data structures changed underneath it, but I have not had the leisure to debug this yet.

In our hands, the auto-reload function works well. I tend to render first to OMN and look at that. Tawny also has a documentation capability which shows you the "unwound" definition of terms. And then finally I use protege after that.

Clojure uses maven dependency management. As a tawny ontology is just a piece of clojure, we can use the same mechanism with tawny ontologies also. Which means that we can specify ontological dependencies also. This means we can specify version ranges (OWL doesn’t allow this to my knowledge). And we can can reuse tooling. We can use leiningin to show us a dependency graph, we can look for version conflicts, we can exclude duplicates from the transistive closure.

Interestingly, we can also publish our ontology independently from our IDs. So, we can get someone else to maintain all the infrastructure for deployment (including of multiple versions) without having to adopt their identifiers (like bioportal).

This rather breaks the Linked Open Data (LOD) principles, of course which says that IDs should resolve. Using maven dependencies we don’t need this at all. But it fulls the SLOD (significant load of dependencies) principle which says if your software has lots of dependencies and lots of different people maintaining the infrastructure for their availability it is going to break all the time.

Thanks to Helen Parkinson for inspiring (a slightly different version) of the SLOD acronym.

One of the great unexplored areas of Tawny at the moment is how much value we can get embedding an ontology into software. We did have a very short project integrating a semantic system (tawny) with a music generation system. This works and was fun. I think there is a lot of scope for research in this area yet.

I made this change very carefully and was very reticient about it: not least because it made my main user of Tawny at the time (Jennifer Warrender) change all of their existing code. But I had really confusing code inside Tawny where my add-subclass functions had backward semantics to all the others.

I changed from ":subclass" to a more plain ":super" at the same time. This opens up a slight risk because object-property has the same frame but for a different purpose. I do not worry about this too much because other tools will pick up, for example, the use of a class as a super-property.