Classification, RDF, and promiscuous vowels

"[He] provided (i) a classified schedule of things and concepts, (ii) a series of conjunctions or 'particles' whereby the elementary terms can be combined to express composite subjects, (iii) various kinds of notational devices ... as a means of displaying relationships between terms." [1]

"By reducing the complexity of natural language to manageable sets of nouns and verbs that are well-defined and unambiguous, sentence-like statements can be interpreted...."[2]

The "he" in the first quote is John Wilkins, and the date is 1668.[3] His goal was to create a scientifically correct language that would have one and only one term for each thing, and then would have a set of particles that would connect those things to make meaning. His one and only one term is essentially an identifier. His particles are linking elements.

The second quote is from a publication about OCLC's linked data experiements, and is about linked data, or RDF. The goals are so obviously similar that it can't be overlooked. Of course there are huge differences, not the least of which is the technology of the time.*

What I find particularly interesting about Wilkins is that he did not distinguish between classification of knowledge and language. In fact, he was creating a language, a vocabulary, that would be used to talk about the world as classified knowledge. Here we are at a distance of about 350 years, and the language basis of both his work and the abstract grammar of the semantic web share a lot of their DNA. They are probably proof of some Chomskian theory of our brain and language, but I'm really not up to reading Chomsky at this point.

The other interesting note is how similar Wilkins is to Melvil Dewey. He wanted to reform language and spelling. Here's the section where he decries alphabetization because the consonants and vowels are "promiscuously huddled together without distinction." This was a fault of language that I have not yet found noted in Dewey's work. Could he have missed some imperfection?!

*Also, Wilkins was a Bishop in the Anglican church, and so his description of the history of language is based literally on the Bible, which makes for some odd conclusions.



[1]Schulte-Albert, Hans G. Classificatory Thinking from Kinner to Wilkins: Classification and Thesaurus Construction, 1645-1668. Quoting from Vickery, B. C. "The Significance of John Wilkins in the History of Bibliographical Classification." Libri 2 (1953): 326-43.
[2]Godby, Carol J, Shenghui Wang, and Jeffrey Mixter. Library Linked Data in the Cloud: Oclc's Experiments with New Models of Resource Description. , 2015.
[3] Wilkins, John. Essay Towards a Real Character, and a Philosophical Language. S.l: Printed for Sa. Gellibrand, and for John Martyn, 1668.

By way of explanation

“Readers who are familiar with conventional logical semantics may find it useful to think of RDF as a version of existential binary relational logic in which relations are first-class entities in the universe of quantification. Such a logic can be obtained by encoding the relational atom R(a,b) into a conventional logical syntax, using a notional three-place relation Triple(a,R,b); the basic semantics described here can be reconstructed from this intuition by defining the extension of y as the set { : Triple(x,y,z)} and noting that this would be precisely the denotation of R in the conventional Tarskian model theory of the original form R(a,b) of the relational atom. This construction can also be traced in the semantics of the Lbase axiomatic description.”
        From the RDF Semantics document

"Doubts about the ability to know the order of the world catalyzed a crucial change, away from taxonomic forms of information storage based on natural language and toward new ones based on a symbolic language of analytical abstraction. Mathematics promised a new vision of order for both the natural and the moral worlds, where confusion was resolved by jettisoning whatever could not be known with certainty."
     Hobart, Michael E. Information Ages: Literacy, Numeracy, and the Computer Revolution. Baltimore: Johns Hopkins University Press, 1998. p. 90

“We could try to feed it algorithms for everything. There are only slightly more of them than there are particles in the universe. It would be like building a heart muscle molecule by molecule. And we’d still have a hell of an indexing and retrieval problem at the end. Even then, talking to such a decision tree would be like talking to a shopping list. It’d never get any smarter than a low-ranking government bureaucrat.”
     Richard Powers, Galatea 2.2, 1st Perenniel Ed., 1996. p. 78

Open to Creativity

The brilliance of Google's PageRank is not the computational methods behind it but the target of those methods: links created by people in the course of making something meaningful on the Web. Without that human input, Google (and Bing and Yahoo) would be simply counting up term frequencies and perhaps analyzing linguistic characteristics, but missing most of what makes Web searching work. The results would be at least as bad as the ranking on Google Books because it would be devoid of the human commitment of significant connections between the pages.

Although Google's mission statement is "... to organize the world's information and make it universally accessible," Google is not really organizing anything. It is reading the organization provided by the Web's population. Similarly, Facebook is reading the relationships between people that are made in the course of using its software, information that it would not have otherwise.

What has made the web the rich environment that it is is that anyone can link anything to anything else. That linking is an expression; and even though we might not be able to characterize it in a few words (what does it mean that page A links to page B?) Google has shown that we can make use of those patterns of linking to help people find stuff on the Web that meets their needs.

In this regard, the Semantic Web has a serious problem. Much of the focus of Semantic Web work is the creation of vocabularies of defined relationships between things, with the intention that these relationships can be traversed and manipulated by algorithms. That is fine in itself, but the Semantic Web enthusiasts are primarily creating pre-determined, fixed relationships between things, mainly based on defining each thing in a class/sub-class relationship with other things. The Semantic Web vocabularies also define requirements called "range" which limit what you can link to from a specific element of your vocabulary. [1]

The tendency to pre-define vocabularies with strict rules is a carry-over from the Artificial Intelligence (AI) environment from which the Semantic Web arose. If you expect to work with machines, and only machines, then you have to define for them exactly what they can and cannot do, and you must present all decisions as formulas that can be calculated.

For reasons that is unclear to me, the Semantic Web work seems to be unaware of the great success that Google has had in using human information sharing activity to creating a meaningful web of links. There is little attention to the fact that establishment of relationships through linking can be done by humans, much less that the best and most useful linking will be done by humans. Human information linking is not definable a priori -- in fact, an a priori definition of allowed links essentially limits the future to re-running past concepts in new variations. It is an absolute barrier to creativity if you can only act on what has been pre-defined.

It's the difference between a prefab house, which can only become what it was designed to become (with some minor modifications that don't change its essence) and a box of blocks that can be used to create anything within the realm of physics (although superglue could extend the possibilities).

In part this is why I tend to speak of "linked data" rather than "Semantic Web." Linked data, as it has evolved as the description of metadata activity, carries less of the AI baggage than Semantic Web does.

To me what will be exciting about linked data is what people will do with it; what they will create, what they will experiment with, and both successes and failures. However, for people to do something with linked data they need tools -- tools that are as easy to use as those used to create Web pages and the links between them.

They also need a box of blocks to work with. These blocks need to be as free of predetermined rules as possible. [2] The terms need to be defined just enough to make them usable. This is also true of the relationships or links. Using our box of blocks metaphor, we want to be able to put the blocks in relationships like "above" "below" "near" each other. If the square blocks are defined as always "below" the rectangular blocks, then that limits what we can create.

The Semantic Web as a machine environment guided with AI formalities appeals to some because it promises to be neat and unambiguous. It will, however, foster only a very constrained amount of creativity, and will not be able to satisfy the full range of human curiosity.

It is a shame that many Semantic Web enthusiasts have little faith (or little interest) in the human potential that linking and openness can unleash. I, for one, am looking for partners in the development of a messy, intelligent, quirky, technology that can produce surprising results, created by people using linked data as a tool of expression. I am particularly exciting by the fact that we don't know what forms that expression will take.

---

[1] For example, you can define a creator has having to be type "Person" and that Person must be expressed as a URI, like http://kcoyle.net/kcoyle.rdf. In that case, you can't have a creator that is "Karen Coyle" because "Karen Coyle" is just a string of characters, not an identified entity. This means that if you don't have identifiers for your creators, you can't create data about what they created.]

[2] This is referred to as "minimum ontological commitement," introduced in Toward Princiciples for the Design of Ontologies Used for Knowledge Sharing, by T Gruber [http://www2.iath.virginia.edu/time/readings/ontology-semantics-metaphor/designing-ontologies.pdf]

RDA, DBMS, RDF

I have written before about some issues relating to RDA and RDF. Today I want to actually consider some things we should consider that should cause us to question the concept of "RDA in RDF."

For many decades we have been using relational databases to store our bibliographic data, bibliographic data that we create and exchange using the MARC format. Doing so was not by any means natural or intuitive because there is nothing about the structure or content of the MARC record that lends itself to being stored and managed in a relational database. The results were often awkward, inefficient, and unsatisfying.

Part of the reason for this is the unitary and flat nature of MARC. In spite of the long history of creating separate authority files, each MARC record is a complete and closed document with no actual connections to data outside of itself. While some database implementations for MARC do create relational tables for headings, the degree to which a MARC record can be separated out into tables is minimal and gains us very little in terms of the functionality of an RDBMS.

The underlying problem, however, is not in the structure of the MARC record but in the content of our catalog records. Moving from the card to a database for our data requires more than adding mark-up coding around the catalog data; to do so successfully requires re-thinking the data in terms of relational database principles. There are two basic principles to relational database design: repetition and combination.

To design for relational databases you look at your data to see what elements will be repeated in many different records. Rather than carrying those data elements in multiple records, you create a separate database table for each repeating element, and you store that element once. For example, if you are creating a database of mailing addresses, you see quickly that elements like state and zip code will appear in multiple records. You therefore create a table of state names and one of zip codes, and perhaps even one that links zip codes to city names. In this way, your database carries the string "Mississippi" only once, and that string is replaced in the records with a database pointer that uses much less internal storage. Ditto the zip code. And if the zip code is associated in a table with a city name, you also only store city names once, and each address record needs only a pointer to the zip code, not a city name. In fact, with a zip code you can get the city and state, and your design might look like:

In this way you have saved a huge amount of storage space. You have also made selection of your records on zip code, city and state much more efficient than if they were stored in every address record, because a search on a zip code, for example, retrieves a single entry in the zip code table, and that entry has database-managed links to the relevant records.

In a database of customer orders that has your inventory information along with customer addresses, you use the tables in your database to search for things like "all customers in Mississippi who have ordered WidgetX in the last six months." Information about your inventory and information about purchases are all in appropriate sets of tables in your database and you can combine the data elements to develop different views of the data.

Where the goal in relational database design is to identify and isolate data elements that are the same, the goal in library cataloging data is exactly the opposite: headings are developed primarily to differentiate at the data creation point rather than allow combination within the database management system. The goal is to have each data point be as unique as possible and to be assigned to as few records as possible. Thus, library cataloging creates headings whose purpose is to distinguish between entries:

Shakespeare, William, 1564-1616. As you like it
Shakespeare, William, 1564-1616. As you like it. 1905
Shakespeare, William, 1564-1616. As you like it. 1911.
Shakespeare, William, 1564-1616. As you like it. 1919.
Shakespeare, William, 1564-1616. As you like it. Czech
Shakespeare, William, 1564-1616. As you like it. French

These headings are counter to the functioning of a database management system. If moved to a database table to facilitate retrieval, they will each point to only one or a very small number of records. This negates both the space-saving aspect of database management and it also does not facilitate combination of data elements for retrieval. In the case of headings, the combination of elements is pre-coordinated in the data, rather than post-coordinated in the database retrieval function.

A database approach might break this data into four tables:

In this way one could search for this data by title, by title + author, date + language, or by any other combination of these four data elements. To search the library headings as anything but a single keyworded string, that is to use these headings to perform searches on title or date or language, would be incredibly inefficient. The upshot is that library headings are not "relational" and do not contribute to the functionality that database management systems can provide. Instead, database management systems make use of the separate coded elements, such as date and language, for combinatorial retrieval. Names and titles, because they are text strings and do not have an identified presence in the stored records, must be searched separately rather than being available for relational combination. The results of this type of searching are less than optimal in speed and accuracy.

All of this may seem obvious to some of you, so you may be asking yourselves why I bring this up. I bring it up because even though RDA claims to have as its goal the creation of records in a relational design (see scenario one in this JSC document), it continues to instruct catalogers to create pre-coordinated headings like the ones above. Not only will these not be efficient or fruitful in a relational database, this brings into question whether RDA is truly modeled on the principles it claims to embrace. If it is not we have cause to worry: we cannot move forward with data that does not conform to a modern model.

Note that in this post I have been emphasizing the use of relational database design for the data. The current plans for a new bibliographic framework appear to plan to create a data model for RDA that is based on semantic web principles. Those principles are yet another significant evolution following on the database model, which is now considered waning technology. Other communities, ones that have been designing for database management requirements for their data for decades, are now looking at ways to transform that data to RDF. It is possible that we can skip the relational database phase of our data development and move directly into a semantic web model. However,  to think that data created following RDA instructions, which is not even suitable for a relational database, could be made usable on the semantic web without major modifications is simply wrong. If we create a bibliographic framework that takes RDA as it has been described and ports that, unchanged, to RDF we will create a data model that does not serve us, does not serve our users, and that cannot reasonably interact with other linked data on the web.

What we need is an analysis of our data, not a transformation of it "as is" to a new technology. If we aren't ready to admit that some traditional practices, like headings, are no longer useful or usable in today's technological environment, we cannot have any hope that our data will be relevant in the future.

(p.s. I anticipate that someone will state that headings are needed for alphabetical displays, to "collocate" the records. To that I reply: 1) you can do the same collocation using the data elements, and in fact you could devise multiple collocations by combining the elements in different ways and 2) a linear, alphabetic display is so anachronistic with today's technology, and so seldom used when available, that it is hard to justify the use of human catalogers to create these fields. If you still believe that library records must contain hand-crafted headings, all I can say is: you can believe what you want, but some of us will be exploring other solutions.)

Can libraries change?

The declaration by Library of Congress that the time has come to make the long overdue change to a new data format has rocked the library world. A common reaction is: "How can we do that, when we have 1) thousands of library systems that are designed primarily to work with MARC records 2) no money to pay for a major change, and 3) have no clear idea what we should be heading towards." This fear is increased by the fact that so far there is little public evidence of activity on this project.

A change of this nature is huge. It's not quite Europe converting to the Euro, but within the library world it is a change of the magnitude of converting the Internet from IPv4 to IPv6. We've made other big changes in the past, in particular the change from the card catalog to the OPAC. That effort required us to purchase new systems and to convert the whole of the printed card catalog to the MARC format. Amazingly, it took only about decade to complete that transition.

However, here is the key difference:  that change was entirely internal to the library community. This next one has an additional complication brought on by the fact that the target environment for the future of library data is the Web. This new framework will need to be integrated with that massively complex environment of networked information. This adds unknowns ("How will Web users interact with library data?"), but it also affords some possibilities that we didn't have with the change to MARC. Mainly, it allows us to make use of existing Web technology and the Web community for help in both designing and implementing the change.

What this means to me is that this is not a "library-only" activity that we are embarking on. Unknown numbers of users and systems will want or need to make use of library data on the Web. At least we hope so. Right now, Web services in need of bibliographic data often point to Amazon. Others rely on "crowd-sourced" solutions like Mendeley or Zotero. What will make library data most useful and usable to the larger community? This isn't a question we should be asking of libraries, but of potential users of the data.

There are other important questions we should be thinking about. How will we test whether the new framework is well designed for system functionality and efficiency? How will we convert from what we have to this new framework? What structures must we put in place to maintain and extend the framework over time?

It seems very unlikely that the Library of Congress can address all of this in the 18-month period that is allotted for this work (of which perhaps 12 months remain) because: a) their focus is understandably primarily on the needs of their organization and b) this effort, to be successful, must have input from organizations that are external to the library community. That’s a very tall order for short time span.

None of this should be taken to imply that LC doesn't have smart, skilled staff to work on this -- they do. But if you've ever taken on a large project in your institution you know that the staff working on the project is also doing much of the day-to-day work that occupied them before the project was begun. Few are able to dedicate 100% of their time to a new effort. The question therefore becomes: How can a larger community help LC with this project, taking on appropriate task areas?

I have in mind a set of tasks that could be worked on in parallel, by a number of different interested constituencies and with some good coordination. More details are needed, but the big picture is something like this:

The Web track is an obvious one, especially given that the W3C has already shown an interest in facilitating the entry of library data onto the Semantic Web. There is also a growing realization in the library community that we need to fairly quickly begin to build on the foundation standards developed by the W3C Semantic Web activity. There appears to be a similar awareness in the Semantic Web community that library data presents interesting challenges. For example, library data has revealed that an approach to authority data is needed that cannot currently be provided by SKOS (Simple Knowledge Organization System). Discussions on lists that focus on the Semantic Web make it clear that our early efforts in defining library data in RDF are helping to inform the thinking of the experts in linked data creation.

The bibliographic description track is of course the key one for libraries. This to me is the solid ground of LC, along with its community partners: to determine the semantics of the data that libraries will use to describe their resources and to provide access for users. RDA already does a great deal of this but the task ahead is to make sure that one can express those concepts in a new data format. There should also be an analysis of the cataloging workflow and even of the expected functionality of a cataloging interface. The requirements arising out of this track will inform the work of the Web and IT tracks as they help the Library convert these requirements to implementable structures and applications.

The IT track is absolutely essential: How do we assure that we have data structures that work well with the entire gamut of library systems functions, from acquisitions through circulation? One question I have in particular is about the efficiency of a large bibliographic database structured around the FRBR entities. Efficiency must consider more than just the creation of the bibliographic data, it also must be efficient for the retrieval and display of that data. The report on the Future of Bibliographic Control recommended testing of FRBR. RDA has served to test many of the FRBR concepts, but as yet there is no proof of concept of a data structure that uses the FRBR entities. This track, as I see it, would involve library systems vendors as well as some computer professionals who work with "big data" and semantic web technologies.

The management track is very important but in LC's plan it might be relegated to a later phase since it appears to deal mainly with future activities such as maintenance and modification of the standard. This, however, would be a mistake because the standards for maintenance and extension must be in place from the very beginning of the new framework. I would even say that the new framework should be developed from the beginning with a core and extensions. This eliminates the need to have on opening day a standard that is "everything for everybody," and could allow for a phased implementation of the framework. Note that there are some immediate issues in RDA that require a maintenance standard, such as how to handle open-ended controlled lists in a way that would be compatible with Semantic Web standards.

A critical part of this is the coordination between all of these activities. Such a role, however, is not unusual in a large IT project where work is spread across groups with intersecting milestones.

It seems to me that a division of this nature (and not necessarily exactly how I have described it here) would relieve LC of some of the work that it is undoubtedly currently considering taking on; it could increase the speed with which the full design could be completed; and in my opinion it has the possibility to produce a higher quality solution than could be achieved by a single organization. Logical participants include NISO (both in its role as the standards body that manages the MARC standard and in its role as a focus for the library technology community), W3C's Semantic Web community, Dublin Core Metadata Initiative (which is working on standards for application profiles in RDF), and IFLA (which now has a Semantic Web interest group). There is also some possible synergy with projects like the Internet Archive, the Digital Public Library of America, schema. org, and the Zotero community.  Clearly funding would be needed, and that's also not a simple task.

My concern is that if we don't organize ourselves in this way, that come January, 2013 we will not be anywhere near having the ability to create bibliographic data in a new framework. RDA will be implemented inadequately in MARC and, as that solution is the path of least resistance, work to create a new framework will slow to a crawl. If we don't step up to this task, for many years to come we will continue to see library data housed in frameworks and silos that are invisible to most information seekers.  That would indeed be very unfortunate.

---

Note: Planned session for ELAG2012 to be led by Lukas Koster with a very similar approach, and with the intention of delivering ideas to LC for the new framework.

Bibliographic Framework: RDF and Linked Data

With the newly developed enthusiasm for RDF as the basis for library bibliographic data we are seeing a number of efforts to transform library data into this modern, web-friendly format. This is a positive development in many ways, but we need to be careful to make this transition cleanly without bringing along baggage from our past.

Recent efforts have focused on translating library record formats into RDF with the result that we now have:
    ISBD in RDF
    FRBR in RDF
    RDA in RDF

and will soon have
    MODS in RDF

In addition there are various applications that convert MARC21 to RDF, although none is "official." That is, none has been endorsed by an appropriate standards body.

Each of these efforts takes a single library standard and, using RDF as its underlying technology, creates a full metadata schema that defines each element of the standard in RDF. The result is that we now have a series of RDF silos, each defining data elements as if they belong uniquely to that standard. We have, for example, at least four different declarations of "place of publication": in ISBD, RDA, FRBR and MODS, each with its own URI. There are some differences between them (e.g. RDA separates place of publication, manufacture, production while ISBD does not) but clearly they should descend from a common ancestor:

RDA: place of publication
RDA: place of distribution
RDA: place of manufacture
FRBRer: has place of publication or distribution
ISBD: has place of publication, production, distribution

This would be annoying, but not unworkable, if these different instances of "place of publication" could be treated as having some meaning in common such that one could link a FRBRer element to an ISBD element, but they cannot. The reason they cannot is that each of these constrains the elements in a particular way that defines its relationship to a single data context (what we generally think of as a "record structure"). The elements are not independent of that context, and this means that each can only be used within that particular context. This is the antithesis of the linked data concept, where data sets from diverse sources share metadata elements. It is this re-use of elements that creates the "link" in linked data. To achieve this, metadata elements need to be unconstrained by a particular context.

Linking can also be achieved through vertical relationships, similar to "broader" and "narrower" in thesauri. This is less direct, but makes it possible to mix data sets that have differing levels of granularity. In our case, the ISBD "place of publication, production, distribution" could be defined as broader to the three RDA elements that treat those separately. Unfortunately that is not possible because of the way that ISBD and RDA have been defined in RDF. (I'll post more detail about this later for those who want more.)

The result is that we now have a series of RDF silos, expressions of our data in RDF that lack the linking capabilities of linked data because they are bound to specific data structures. Clearly we gain little in terms of linked data by creating mutually incompatible bibliographic views. Not only are these RDF schemes not compatible with each other, none will be linkable to bibliographic data from communities outside of libraries who published their data on the Web. That means no linking to Amazon, to Wikipedia, to citations within documents.

Given where we are in the development of linked data for libraries, we now have two options:

1) Define 'super-elements' that float above the record formats and that are not bound by the constraints of the RDF-defined records. In this case there would be a general "place of publication" that is super- to all of the "place of publication" elements in the various records, and would be subordinate to a general concept of "place" that is widely used (possibly a property of GeoNames). To implement linking, each record element would be extrapolated to its super elements.

2) Define our data elements outside of any particular record format first, then use these in the record schemas. In this case there would be only one instance of "place of publication" and it would be used throughout the various bibliographic records whenever an element with that meaning is needed. Those records would be interchangeable as linked data using their component data elements, and would interact with other bibliographic data on the Web using the RDF-defined elements and their relationships.

My message here is that we need to be creating data, not records, and that we need to create the data first, then build records with it for those applications where records are needed. Those records will operate internally to library systems, while the data has the potential to make connections in linked data space. I would also suggest that we cease creating silo'd RDF record formats, as these will not move us forward. Instead, we should concentrate on discovering and defining the elements of our data, and begin looking outward at all of the data we want to link to in the vast information universe.


_____
* Note on RDA: RDA in RDF includes two "versions" of each data element: one bound to FRBR and one not. The latter has potential for re-use outside of a FRBR environment, and was designed for this purpose by the DCMI/RDA task force. Its relationship to "official" RDA is somewhat unclear at this time but hopefully will gain support as the linked data concept is absorbed into the bibliographic framework.

XML and Library Data Future

There is sometimes the assumption that the future data carrier for library data will be XML. I think this assumption may be misleading and I'm going to attempt to clarify how XML may fit into the library data future. Some of this explanation is necessarily over-simplified because a full exposition of the merits and de-merits of XML would be a tome, not a blog post.

What is XML?

The eXtensible Markup Language (XML) is a highly versatile markup language. A markup language is primarily a way to encode text or other expressions so that some machine-processing can be performed. That processing can manage display (e.g. presenting text in bold or italics) or it can be similar to metadata encoding of the meaning of a group of characters ("dateAndTime"). It makes the expression more machine-usable. It is not a data model in itself, but it can be used to mark up data based on a wide variety of models.*

XML is the flagship standard in a large family of markup languages, although not the first: it is an evolution of SGML which had (perhaps necessary) complexities that rendered it very difficult for most mortals to use. It's also the conceptual granddaddy of HTML, a much simplified markup language that many of us take for granted.

Defining Metadata in XML

There is a difference between using XML as a markup for documents or data and using XML to define your data. XML has some inherent structural qualities that may not be compatible with what you want your data to be. There is a reason why XML "records" are generally referred to as "documents": they tend to be quite linear in nature, with a beginning, a middle, and an end, just like a good story.

XML's main structural functionality is that of nesting, or the creation of containers that hold separate bits of data together.

<paragraph>
   <sentence></sentence>
   <sentence></sentence> ...
</paragraph>

<name>
   <familyname></familyname>
   <forenames></forenames>
</name>

This is useful for document markup and also handy when marking up data. It is not unusual for XML documents to have nesting of elements many layers deep. This nesting, however, can be deceptive. Just because you have things inside other things does not mean that the relationship is anything more than a convenience for the application for which it was designed.

<customer>
    <customerNumber></customerNumber>
    <phoneNumber></phoneNumber>
</customer>

Nested elements are most frequently in a whole/part relationship, with the container representing the whole and holding the elements (parts) together as a unit (in particular a unit that can be repeated).

<address>
    <street1></street1>
    <street2></street2>
    <city></city>
    <state></state>
    <zip></zip>
</address

While usually not hierarchical in the sense of genus/species or broader/narrower, this nesting has some of the same data processing issues that we find in other hierarchical arrangements:

• The difficulty of placing elements in a single hierarchy when many elements could be logically located in more than one place. That problem has to be weighed against the inconvenience and danger of carrying the same data more than once in a record or system and the chances that these redundant elements will not get updated together.
• The need to traverse the whole hierarchy to get to "buried" elements. This was the pain-in-the-neck that caused most data processing shops to drop hierarchical database management systems for relational ones. XML tools make this somewhat less painful, but not painless.
• Poor interoperability. The same data element can be in different containers in different XML documents, but the data elements may not be usable outside the context of the containing element (e.g. "street2").

Nesting, like hierarchy, is necessarily a separation of elements from each other, and XML does not provide a way to bring these together for a different view. Contrast the container structure of XML and a graph structure.

In the nested XML structure some of the same data is carried in separate containers and there isn't any inherent relationship between them. Were this data entered into a relational database it might be possible to create those relationships, somewhat like the graph view. But as a record the XML document has separate data elements for the same data because the element is not separate from the container. In other words, the XML document has two different data elements for the zip code:

  address:zip
  censusDistrict:zip

To use a library concept as an analogy, the nesting in XML is like pre-coordination in library subject headings. It binds elements together in a way that they cannot be readily used in any other context. Some coordination is definitely useful at the application level, but if all of your data is pre-coordinated it becomes difficult to create new uses for new contexts.

Avoid XML Pitfalls

XML does not make your data any better than it was, and it can be used to mark up data that is illogically organized and poorly defined. A misstep that I often see is data designers beginning to use XML before their data is fully described, and therefore letting the structure and limitations of XML influence what their data can express. Be very wary of any project that decides that the data format will be XML before the data itself has been fully defined.

XML and Library data

If XML had been available in 1965 when Henriette Avram was developing the MARC format it would have been a logical choice for that data. The task that Avram faced was to create a machine-readable version of the data on the catalog card that would allow cards to be printed that looked exactly like the cards that were created prior to MARC. It was a classic document mark-up situation. Had that been the case our records could very well have evolved in a way that is different to what we have today, because XML would not have had the need to separate fixed field data from variable field data, and expansion of some data areas might have been easier. But saying that XML would have been a good format in 1965 does not mean that it would be a good format in 2011.

For the future library data format, I can imagine that it will, at times, be conveyed over the internet in XML. If it can ONLY be conveyed in XML we will have created a problem for ourselves. Our data should be independent of any particular serialization and be designed so that it is not necessary to have any particular combination or nesting of elements in order to make use of the data. Applications that use the data can of course combine and structure the elements however they wish, but for our data to be usable in a variety of applications we need to keep the "pre-coordination" of elements to a minimum.

---

* For example, there is an XML serialization (essentially a record format) of RDF that is frequently used to exchange linked data, although other serializations are also often available. It is used primarily because there is a wide range of software tools available for making use of XML data in applications, and there are many fewer tools available for the more "native" RDF expressions such as triples or turtle. It encapsulates RDF data in a record format and I suspect that using XML for this data will turn out to be a transitional phase as we move from record-based data structures to graph-based ones.

New bibliographic framework: there is a way

Since my last post undoubtedly left readers with the idea that I have my head in the clouds about the future of bibliographic metadata, I wish to present here some of the reasons why I think this can work. Many of you were probably left thinking: Yea, right. Get together a committee of a gazillion different folks and decide on a new record format that works for everyone. That, of course, would not be possible. But that's not the task at hand. The task at hand is actually about the opposite of that. Here are a few parameters.


#1 What we need to develop is NOT a record format

The task ahead of us is to define an open set of data elements. Open, in this case, means usable and re-usable in a variety of metadata contexts. What wrapper (read: record format) you put around them does not change their meaning. Your chicken soup can be in a can, in a box, or in a bowl, but it is still chicken soup. That's the model we need for metadata. Content, not carrier. Meaning, not record format. Usable in many different situations.

#2 Everyone doesn't have to agree to use the exact same data elements

We only need to know the meaning of the data elements and what relationships exist between different data elements. For example, we need to know that my author and your composer are both persons and are both creators of the resource being described. That's enough for either of us to use the other's data under some circumstances. It isn't hard to find overlapping bits of meaning between different types of bibliographic metadata.

Not all metadata elements will overlap between communities. The cartographic community will have some elements that the music library community will never use, and vice versa. That's fine. That's even good. Each specialist community can expand its metadata to the level of detail that it needs in its area. If the music library finds a need to catalog a map, they can "borrow" what they need from the cartographic folks.

Where data elements are equivalent or are functionally similar, data definitions should include this information. Although defined differently, you can see that there are similarities among these data elements.

pbcoreTitle =  a name given to the media item you are cataloging
RDA:titleProper = A word, character, or group of words and/or characters that names a resource or a work contained in it.
MARC:245 $a = title of a work
dublincore:title = A name given to the resource

All of these are types of titles, and have a similar role in the descriptive cataloging of their respective communities: each names the target resource. These elements therefore can be considered members of a set that could be defined as: data elements that name the target resource. Having this relationship defined makes it possible to use this data in different contexts and even to bring these titles together into a unified display. This is no different to the way we create web pages with content from different sources like Flickr, YouTube, and a favorite music artist's web site, like the image here.

In this "My Favorites" case, the titles come from the Internet Movie Database, a library catalog display, the Billboard music site, and Facebook. It doesn't matter where they came from or what the data element was called at that site, what matters is that we know which part is the "name-of-the-thing" that we want to display here.

#3 You don't have to create all new data elements for your resources if appropriate ones already exist

When data elements are defined within the confines of a record, each community has to create an entire data element schema of their own, even if they would be coding some elements that are also used by other communities. Yet, there is no reason for different communities to each define a data element for an element like the ISBN because one will do. When data elements themselves are fully defined apart from any particular record format you can mix and match, borrowing from others as needed. This not only saves some time in the creation of metadata schemas but it also means that those data elements are 100% compatible across the metadata instances that use them.

In addition, if there are elements that you need only rarely for less common materials in your environment, it may be more economical to borrow data elements created by specialist communities when they are needed, saving your community the effort of defining additional elements under your metadata name space.

---

To do all of this, we need to agree on a few basic rules.

1) We need to define our data elements in a machine-readable and machine-actionable way, preferably using a widely accepted standard.

This requires a data format for data elements that contains the minimum needed to make use of a defined data element. Generally, this minimum information is:

• a name (for human readers)
• an identifier (for machines)
• a human-readable definition
• both human and machine-readable definitions of relationships to other elements (e.g. "equivalent to" "narrower than" "opposite of")

2) We must have the willingness and the right to make our decisions open and available online so others can re-use our metadata elements and/or create relationships to them.


3) We also must have a willingness to hold discussions about areas of mutual interest with other metadata creators and with metadata users. That includes the people we think of today as our "users": writers, scholars, researchers, and social network participants. Open communication is the key. Each of use can teach, and each of us can learn from others. We can cooperate on the building of metadata without getting in each others' way. I'm optimistic about this.

Metadata Seminar at ASIST

I'm going to be giving a half-day seminar on October 12 in New Orleans in association with ASIST. This is something I have been wanting to do for a while. I feel like I've spent the past two years presenting Semantic Web 101 in 45-minute segments, and I really want to start moving on to 102, 103, etc. I'm hoping this seminar will fill that gap.

The topics I will cover at that seminar are:

• Understanding data, data types, and data uses
• Identifiers, URIs and http URIs
• Statements and triples and their role in the 'web of data'
• Defining properties and vocabularies that can be used effectively on the web
• Brief introduction to semantic web standards

There will be hands-on exercises throughout the morning that give attendees a chance to learn by doing. I'm hoping that the exercises will also be fun. If you're going to ASIST and have any questions about the seminar, please contact me.

Visualizing linked data

Chris Oliver, Diane Hillmann and I will be reprising (and updating) our three-part webinar on RDA and the future of library metadata starting on May 11. As before, Chris will cover the principles behind RDA and why RDA is different from other cataloging codes; I will talk about the Semantic Web and why it is important for libraries to be part of the web of data (May 18); Diane will show how the Open Metadata Registry makes possible a Semantic Web-compatible version of RDA (May 25).
One of the questions I always get when talking about the Semantic Web is "What does it look like?" This is kind of like asking what electricity looks like: it doesn't so much look like anything, as it makes certain things possible. But I fully understand that people need to see something for this all to make sense, so when the webinar technology allows it I have started showing some web pages. When it doesn't, I send people to links they can explore on their own. Since some of you may have this same question, here are a few illustrations using two sites that can present authors in a Semantic Web form.

When you do a search for an author on the Open Library you retrieve a page for the author. This is a page for the author Barbara Cartland. The page has not been hand-coded by a human but is derived "on the fly" from the information in the Open Library database.

That same information is available in a semantic web format, RDF in XML. (Note: it is common to code Semantic Web data in XML, but that's not the only possible data format. There is nothing inherent in the Semantic Web that would make it XML-like, it's just a convenience.) This is not intended to be human friendly -- it is code to be used by programs. You should notice that it makes use of identifiers that look like URLs:

<foaf:person about="http://openlibrary.org/authors/OL22022A"></foaf:person>

The above establishes the primary identifier for all of the information that follows in the XML.

You will also see that, like other applications using XML, it allows you to mix data elements from different "namespaces." The Open Library RDF uses a mix of elements from Dublin Core, Friend-of-a-Friend (FOAF), the Bibliographic Ontology, and RDA Vocabularies.

Another database that provides its data in RDF is the Virtual International Authority File, VIAF. VIAF combines the name authority data from about twenty national authority files, making it possible to translate from different name display forms when exchanging data. Here is part of the VIAF display for Barbara Cartland:


You can retrieve or export the metadata for this author in various formats including MARC and RDF/XML. Once again you will see that the RDF form of the data makes use of FOAF, a standard called "Simple Knowledge Organization System" or SKOS, and also RDA vocabularies for the FRBR Group2 entities from the Open Metadata Registry.

You can look at more examples on my links page, but I hope that this takes some of the mystery out of Semantic Web data, or at least makes the mystery a known rather than unknown puzzler.

Bib data and the Semantic Web

I know that I've gone on and on about transforming bibliographic data into a semantic web format. And whenever folks have asked me: "What will it look like?" I haven't had a good response. Now there is something to show you: Freebase.

Freebase is a database of interlinked semantic web "statements": essentially what are called by the SemWeb types as "triples." The statements come from a variety of open data sources such as Wikipedia, TVDB.com, a science fiction fan database, and Open Library. By placing a user interface over these data they now have a searchable, navigable site that can link books to movies to (theoretically) music to science to... well, anything where linked data is available.

Their book data isn't as strong as it should be, given that they claim to have imported the Open Library file (I suspect it was only partially imported). When you look at the Freebase entry for Emily Dickinson you only see two works listed. Open library has 137 Works for Dickinson, and WorldCat Identities lists 3, 388. Also, their approach is more "popular" than rigorous. However, there is no reason why this same technique could not be used with "pure" library data, and library catalogs could make use of any of the data in such a database because it is all available through linking and APIs. A database like Freebase essentially serves as a huge pot of available, re-usable information.

In its current form, Freebase would not be sufficient for library data sharing, although it could provide an interesting testing ground. What we need to work out for libraries is a way to version and source content so that you know who provided each statement and when, and to make it easy to contribute new information or improvements to the information in a sensible and automated way. There is no reason why we could not create a "LibBase" that exists solely of what libraries would consider to be authoritative information; a kind of linked data WorldCat. That data would have to be able to interact with other data on the Web, and by doing so libraries would become discoverable on the Web. It would be logical for projects like Freebase to link to the library data. Library users would have a rich, navigable information base that could help them follow (or even make) connections between library resources -- connections that are much less evident in today's catalogs. Some technical magic would need to occur to allow users to move seamlessly from the whole world to their local library, but I don't think that's going to take rocket science to solve.

There is a group of interested souls planning to get together on the Friday morning of ALA DC to begin some exploration of how we might make semantic web technology work for libraries. There will be announcements on various lists (I'm guessing NGC4LIB, CODE4LIB, LITA-L and RDA-L, a the very least). If you can get to ALA a little early, please mark that slot on your calendar. It'll be a free-floating, working, barcamp-style meeting, as I understand it.

Shameless Self-Promotion

The American Library Association has published two reports that I prepared on metadata and the semantic web.

Report 1 is called: Understanding the Semantic Web: Bibliographic Data and Metadata. This is a broad overview of new concepts, aimed especially at those who are new to the semantic web and to web-based metadata. (Note: To understand the diagrams, you will need a copy of the Errata page, since a key set of the diagrams was borked.)

Report 2 has the catchy title of: RDA Vocabularies for a Twenty-First-Century Data Environment. This builds on the first report, and gives more information about building semantic web vocabularies. This report is for all of you who are wondering what on earth it is that Diane Hillmann and I keep going on about when we talk about registering RDA for semantic web use. It is not overly technical so anyone who reads through Report 1 should be able to understand the general direction that we are advocating.

Feel free to ask questions, make comments, argue with me, or tell me why I'm wrong. I don't claim to have the final answers, and want very much to have a dialog about these concepts that will lead us to a new interesting place to be in library technology.

LC forces take-down of lcsh.info

I am beside myself with fury. I hardly know where to begin. Not long ago, Ed Summers took the LCSH authority file and created an online site with the LC Subject Heading authority file re-formatted as a SKOS vocabulary. For the first time, Web services could link directly to LC subjects as represented in the authority file. And some did.

But the Library of Congress, our Federal, if not National, library, has required Ed to take down the site. A site that contained nothing more than LCSH in a usable form. Data that SHOULD be in the public domain, for anyone to use as they wish. This is an assault against libraries everywhere, an act of censorship.

You can read Ed's statement on lcsh.info.

I would very much like to hear LoC's statement about this. They should not be allowed to control the use of this data, data that belongs to all of us.

Ed couldn't refuse the Library's demand, but anyone who isn't an employee of LoC should have greater freedom. Let's gather around a find a new home for LCSH, one that can't be removed from the public.

The Semantics of Semantic

I've always had a hard time with the Semantic Web because it didn't appear to me to be semantic at all. Thus I started calling it the Syntactic Web since it seemed to be mainly about structure, more like diagramming sentences than having a conversation. I now think I understand why that is.

If you are like me, you assume the term "semantic" is about meaning, and in particular the meaning in words and language. That's what you'll find in the dictionary. It is only recently that I learned that there is another use of the term "semantic" and that is in an area of mathematics called "formal semantics". Basically, formal semantics are about formal languages, such as mathematics, programming languages, and such. You can perform operations, often called "inferences," based on the rules of formal languages. This is an example:

A ☠ B
B ☠ C
therefore, A ☠ C

Using some set of rules, this statement is true even though A, B and C are black boxes, as is the relationship "☠". The statement after "therefore" can be calculated without ever considering that A, B or C have any meaning beyond being the symbols A, B and C.

This is a whole different meaning of "meaning" compared to the meaning of words in the human language sense. This is the kind of meaning that works with machines and algorithms and is therefore quite suited to automation. And it is this meaning of semantic that is meant for the Semantic Web.

Well, no wonder those of us who aren't mathematicians (or more specifically, involved in the use of formal languages) have been confused by the Semantic Web! It isn't semantic at all in the human language sense, it is mainly about structure and syntax. It's a shame that its developers used the confusing term "semantic" in its name. Although not incorrect, it is definitely a minority view of the meaning (that is, the semantics) of the term, and leads to confusion.

With this new knowledge, I would characterize the semantic web as a basic structure within which one could insert human-meaningful data, and a set of rules that make that structure operational in a computing environment. It greatly resembles the structure of simple human utterances, like "Moby Dick is the title of this book," although the semantic web would express this something like:

URI:abcd (has relation) URI:1234 (with) URI:xxzz

While Semantic Web devotees may be enthusiastic about that statement, most of us are going to get more out of: "Moby Dick is the title of this book." In other words, we only connect with the statement when it has human-understandable meaning; the formal language alone just doesn't do it. It's rather like the difference between the architectural drawings and the actual building: architects can understand what the drawings represent, but most of us will have to wait until the building is completed and we can walk though it in order to experience what the drawings mean.

Now I have to ask myself what to do with this knowledge. I don't think it makes sense to require everyone to be an architect in order to walk through a building, and I know for sure that all of us can speak in sentences even if we aren't experts in linguistics. So it should be possible to interact with, nay even be an active creator of, the semantic web without being conversant with the field of formal semantics. At the moment, though, I don't know how that's going to work, but I do know that if it doesn't work that way the Semantic Web isn't going very far. It just has to become wysiwim - what you see is what I mean. I suggest something Pipes-like might do the trick.

Bibliographic ER

No, I'm not sending libraries to the emergency room, although there are days when I feel like we're at that point. The ER in the title refers to Entity-Relationship, a way to look at data that emphasizes the general viewpoint that there are things, and those things exist in relation to each other.

In one sense, this is what we have done for over a century with our library data. The bibliographic records that we create have in them many relationships: Person authored Book; Publishing House published Book; Book is in Series; Book has Topics. Those relationships are implicit in our records, but the data isn't formatted in an entity-relationship model. Our records, instead, talk about the relationships but don't make it easy to give the various entities their own existence. So we create a record that contains:

Author
Book title.
Place, publisher, date
Series
Subject A
Subject B

The record represents all of the information about the book, but there is no record that represents all of the information about the author, or all of the information about the publisher, etc. Instead, those "entities" are buried in bibliographic records scattered throughout the file.

An E-R model would give each of these entities an identity on which you could hang information about the entity.



OK, I can't draw worth beans. But basically the idea is that authors, subjects, publishers, topics, all become entries in their own right. This means that you can add information to the author record or the series record, because they have their own place in the design. It also makes it easy to look at your data from many different points of view, while still retaining all of the richness of the relationships. So from the point of view of the person who is the illustrator in the book above, the bibliographic world may look like this:

This type of model is expressed in FRBR, but the E-R aspect of FRBR does not seem to be incorporated into RDA as it stands today. Instead, RDA appears to be aimed at creating the same flat structure that we have in library data today.

If you take a look at the OpenLibrary you will see that books get a page that is about the book, and authors get a separate page that is about the author. This is very simple, but it is also very important. It means that the catalog is no longer just a list of books with authors but can become a rich source of information about authors. You can add bios for authors, link to web sites about the author, launch a discussion group about a favorite author. Because the author is an entity, not just a data element in a record about the book, it becomes a potentially active part of your information system.

In the future, I hope that we can give life to many more entities in the OpenLibrary, and also that we can give them meaningful relationships between each other. This would mean taking a semantic web approach to library data. I don't have a clear picture of where we'll end up, but I'm glad that folks there are interested in experimenting. If you've already thought this through or have ideas in this direction, please step forward. I'd love to hear from you.