My report after the BOF on distributed bugtrackers at DebConf

Here are my notes about the very insteresting BOF on distributed bugtrackers held at DebConf

This is just some elements of report (some taken from the gobby edits, and written down by me after watching the video (unfortunately missing 2 mins of sound in the beginning of the video stream/recording)). This is not a full report of all the discussions, as some discussed elements weren’t of interest IMHO.

You’ll find here many ideas of mine, links and pointers (resulting of our work in the context of the Helios project on similar topics), so this is in no way a report of what was said exactly during the BOF.

Aspects of interest:

• for Debian bugs
  • Sharing state between upstream, downstream and sidestream
    • Forwarding upstream
  • Sharing comments between the same bugs
  • merging / conflict resolution (technically challenging, as well as in order to match people’s habits/process)
• Same problem in general
  • distributed bug tracking
    • working offline (modify locally, push, etc.)
  • but also distributed version/commit tracking : tracking bug presence or fix along the branches of commits in distributed VCSes
• Interop with other bugtrackers:
  • method to pass data back and forth between different bugtrackers
  • standard method / format / API : lack of standard widely used
  • visibility of issues known upstream (or elsewhere) to Debian users while using “reportbug”
  • scraping other trackers

Note that these topics may in principle be discussed on the Distributed bug trackers mailing list: http://kitenet.net/cgi-bin/mailman/listinfo/dist-bugs

Tools / implementation ideas :

• First, there’s a wiki resource about lots of software tools : http://dist-bugs.kitenet.net/software/
• SD (presented later by Christine Spang – see my other blog post about her great presentation)
• LaunchPad’s tracker already knows about lots of other bugs in other distros including Debian bugs (through “bug tasks”)
• bts-link watches links between bugs and status changes (mainly through scraping)
• Generic scraping tools could be used :
  • Scraperwiki (http://scraperwiki.com/)
  • or WebReformatter (http://code.google.com/p/webreformatter/
• Virtuoso spongers to perform on the fly conversion of bugs XML feeds to RDF to inject them in a RDF store (ex: search “bugzilla” inside http://www.openlinksw.com/dataspace/dav/wiki/Main/RDFMappersUseCases)

Storage of repositories / databases of remote bugs :

• Mail … lots of suggestions, but to be discarded IMHO
• in users’ Virtuoso RDF stores, eventually in an integrated way with the Semantic Desktop (KDE 4.4 / Tracker / Zeitgeist) which may be an option (integrating bugs in other desktop tools aware of the semantic desktop).
• through desktopcouch (as used by the bughugger client for LP tracker – see the bughugger presentation at RMLL/LSM 2010)

About such storage, and the RDF store option, here’s a prototype of ours that exposes UDD’s bug facts through a SPARQL interface, allowing to publish bug facts as RDF (setup in the frame of the Helios project). It includes both Debian and Ubuntu bugs, and allows interlinking them in a “standard” way (with Semantically described links in RDF). See http://testforge.int-evry.fr/d2r-server/resource/spackage/evolution for an example.

The problem is not how to distribute (distributed database systems exist), but the semantics of bug properties, and the different implementations of interfaces of different tools : which standard to implement, or how many converters are necessary if no standard ? We believe that navigation/referencing/tracking of external bugs would be much easier if bugs were LinkedData citizens, i.e. they would be exposed with (standard) formats like Semantic Web ontologies served with Semantic Web standards (like RDF).

The worst case is currently to have to perform web scraping on bugtracker interfaces. How to avoiding scraping ? A solution could be to publish RDFa / microformats using common standard semantics, directly from the bugtracker pages at bugs URLs (see my wishlist for debbugs on that subject : 590931: Would be great to integrate RDFa metadata into debbugs pages).

Now some pointers to historical details on the same subject and Debian (old posts by Lars Wirzenius) :

• “Distributed bug tracking”
• “Bug exchange format”

Overall, the discussion was quite interesting, but didn’t reach any sensitive progress regarding the state of the art. Much needs to be worked on, mainly on standardizing APIs, semantic descriptions of bugs, and implementation in tools like SD for instance. Stay tuned for more progress.