WordNet was designed for use under program control.

– George Miller (CACM, Nov 95, 38:11, p39)

» The impatient can start browsing WordNet here

WordNet is one of the best known and widely used lexical database for English. Version 3.0 of the database regroups more than 155 287 entries (i.e. English words or expressions) into 117 659 sets of synonyms (called synsets) linked by semantic relations, such as synonymy, antonymy, hyponymy, entailment and others. This organization is akin to some perceived organization of the words within the brain in which words are associated with others so that one wanders (or swan, stray, roam ...) from one concept to another through word associations.

On top of its linguistic qualities and coverage, one reason for the popularity of WordNet is the fact that the authors made the data freely available in a well organized machine readable format. Along with the data, the designers of WordNet also provided many programs for Unix-like and Windows systems that can search the database to return results thus hiding the relatively low level representation of the underlying data. The fact that the source code was in portable C allowed the definition of APIs in many other programming languages.

WordNet information can also be accessed on the web or with specialized application programs. WordNet has been mapped to other formalisms such as Prolog, RDF and OWL and is also part of the Linked Data initiative.

Although the byte oriented format of the data files is useful for machines, it is hard for a human to make sense of it. In fact, it is produced by a program that processes lexicographer files whose format is also a bit cryptic.

XML version

As XML is now a widely used data format, we thought it would be interesting to produce XML versions of the WordNet database files with corresponding XML schemas for validating them. One important advantage of XML is the fact that it is self describing: each field is labelled with a descriptive name which is simpler, for a human, to understand than the current list of cryptic single character or hexadecimal codes used in the WordNet distribution. Given now that most modern programming language already provide well documented XML processing modules, we think this format will be easier to work with than the original file.

We developed an XSLT stylesheet to transform the data and index files available in the dict directory of the WordNet source distribution into a set of equivalent XML files. The stylesheet ensures that no information is lost between an original WordNet dict file and its XML counterpart.

In the original version of WordNet, an identification number (an eight digit decimal number corresponding to the number of bytes from the start of the file) is assigned to each synset. It is used for efficiently referencing the synsets. That number preceded by a letter indicating the part-of-speech (POS), is used as a unique identifier (of type xsd:ID) for synsets.

Each line of a data file corresponds to an XML synset element. There are four XML files data.{adj,adv,noun,verb}.xml, corresponding to each of the original data.{adj,adv,noun,verb} files in the dict directory. The four XML files are then included within a master XML file to form a single XML file for processing either by a stylesheet or by a program by means of an XML API. The master file can be validated with a Relax NG schema.

Each synset has the following structure formally described in the Relax NG schema.

• id : an attribute for the identification of the synset based on the original synset number (an 8 digit number) preceded by a lower-case letter corresponding to the part of speech (a for adjective, v for verb, n for noun and r for adverb);
• type: an attribute giving the type of the synset (noun, verb, adjective or adverb);
• word: one or more elements giving the English strings corresponding to the members of the synset;
• pointer: zero or more elements referencing other synsets, the content of the element indicates the type of reference such as hypernym, hyponym, similar to...;
• frame: zero or more elements describing the context of use;
• def: one or more elements giving definitions;
• example: one or more elements giving or examples of use.

In order to get an easier access to the synset starting from the English words, the WordNet distribution also provides index.{adj,adv,noun,verb} files giving the list of synsets in which a given English word or expression is defined. The stylesheet described above also produces corresponding index.{adj,adv,noun,verb}.xml files which are included in a master XML validated with a Relax NG schema.

The same type of processing has been done for the exceptions list used for morphological processing. For each {adj,adv,noun,verb}.exc file, a corresponding exc.{adj,adv,noun,verb}.xml is produced. An master exception file including these XML file is validated with a Relax NG schema.

Browsing synsets

From this XML format, another XSLT stylesheet has been written to create more than 117 000 HTML files (one per synset) for exploring the network of synsets with an internet browser. This network has 377 592 links between synsets.

An index page is also produced to give access to all words regrouped by their first letter (upper and lower case). From the index page, it is also possible to search for a given word to start browsing. No morphology analysis is performed on the input word (but see other mode of access), but the page dynamically offers some suggestions starting with the first letters already entered in the text field.

In each HTML page (Figure 1 shows a sample synset), the information for a synset is displayed as a table. The synset references are indicated by linked words that correspond to the words of the synset. A tooltip for each link (it appears when the mouse pointer stays on the link for a while) gives the definition of the synset. Each page gives access not only to outgoing references from this synset, but also to incoming links to see all synsets that reference this synset.

At the bottom of each page, it is possible to see the XML version of the synset by clicking on the last line.

Figure 1: Display of Synset 091917557 corresponding to the words: mania, passion, cacoethes.

Browsing senses of words

It is also possible to get all senses of words in WordNet starting from a word or expression, similarly to the WordNet online access form. Figure 2 shows the typical output of this type of searching for getting access to all synsets associated with this word.

Morphology is performed on the query through a Javascript version of Morphy, the reverse morphology program that returns base forms in a specified part of speech.

Figure 2: Display of all senses associated with the word passion.

Programming with the XML files

In order to show the use of the XML files, we have developed a Java version of wn program often used to get access to the original WordNet Program. Most of the features of the original wn are implemented using the same command line structure, but no morphological processing is performed on the search string. This directory gives the Java source files which should be seen more as an example of use than a production tool.

Data files of the XML version of WordNet

The first five types of files are described in the online WordNet documentation WNDB(5WN). The size of files of more than 1 Meg is given within square brackets after the link for downloading. An XML sample file is provided to give an idea of the organisation of the big XML files.

These files a freely downloadable with an unrestricted (see synset a01064167) license. A citation to this source would be appreciated.

^*located in the dict directory of the original Wordnet distribution.

For more information, contact Guy Lapalme
Université de Montréal
August 2013