Tagged a first version of the TWiki to FusionForge’s MediaWiki converter

As announced previously, I’ve been hacking on a migration tool allowing to import into the MediaWiki of a FusionForge project, a conversion of the contents of a TWiki wiki.

I’ve succesfully imported a first project (from PicoForge to FusionForge) using the tool, so I’ve decided to tag a first release and make the Git repo accessible.

More details at : https://fusionforge.int-evry.fr/projects/pytwiki2mediawi/

Feel free to ask here in the comments or by email, in case of need.

And, yes, my Python is most likely awful, but at least, this works, and much more featureful than existing tools I could test.

Working on a TWiki to MediaWiki converter (targetting FusionForge wikis)

I’m currently working on a wiki converter allowing me to transfer old TWiki wikis (hosted on picoforge) to MediaWikis hosted on FusionForge.

Unlike existing tools that I’ve found that more or less target the same needs, mine will address two peculiarities :

  • using MediaWiki’s API to perform the import, where many tools seemed to use SQL requests: this should allow non-administrator users to do the job,
  • importing to wikis of projects hosted on FusionForge instances, even when the project is not public, which means that the API calls need to authenticate to FusionForge first.

The tool is written in Python, and will include my own crappy wiki syntax converter in Python, instead of spawning existing Perl scripts, as others did.

It may happen to work for FosWiki too, but I don’t intend to use it beyond our old TWiki installations, for a start.

Stay tuned for more progress updates.

Edit: I’ve now released a first version.

Experimenting with Linked Open Data about FLOSS projects : matching Debian upstream projects

I’ve been experimenting with Linked Open Data about FLOSS projects harvested from different sources of DOAP or ADMS.SW descriptions. I’ve tried and match upstream projects of Debian packages with upstream projects hosted at Apache, Gnome, or Alioth.debian.org, or catalogued on Pypi.

I’m matching them on identical values of the Homepage field (comparing the Homepage Control field set by Debian packagers with the doap:homepage meta-data in the RDF documents harvested from the upstream project catalogues).

Here are initial results of my little experiment, for number of matched projects, and results on project name’s similarity :

Upstream catalogue Total matching projs Exact same project name Same project name (case independant)
apache 31 0 (0 %) 0 (0 %)
alioth 16 13 (81 %) 13 (81 %)
pypi 439 217 (49 %) 273 (62 %)
gnome 21 0 (0 %) 7 (33 %)
Total 507 230 (45%) 293 (58 %)

The data set contains tens of thousands of projects, with probably many duplicates, but from all of these, only 507 have common homepages.

As you can see, in some cases, the Debian source package names match the upstream project name (sometimes with lower/upper case variants), but in general, the project names aren’t identical, so it is interesting to try and match them by homepage.

For the curious ones, the Apache, Gnome and Pypi project catalogues use to provide RDF meta-data for quite some time. More recently have we introduced ADMS.SW meta-data for Debian source packages, and even more recently for the Alioth projects (through the ADMS.SW exporter plugin for FusionForge).

There are still some ways for improvements, for instance to normalize homepage URLs which tend to vary (trailing slashes, or different HTTP/HTTPS schemes).

Stay tuned for more details.

Compact preview of resources in FusionForge

I’ve been working on the ‘compactpreview’ plugin (in FusionForge’s SVN trunk), in order to support some javascript popups that can be used to display some “compact preview” of users and projects.

As can be see in the screencast below (also here) there are 2 types of compact preview popups :

  • those for local resources of the forge, that are displayed directly, as queried by the JS code on the /users/ or /projects/ pages with a specific “application/x-fusionforge-compact+html“content-type (required in the Accept HTTP header).
  • those compatible with the OSLC compact preview specifications, that can be displayed, should any other application want to display a compact preview. Again, these are served with content-negociation for “application/x-oslc-compact+xml“, which returns a short RDF document, which points to a script of the forge in charge of rendering the HTML compact preview.

The latter is demonstrated in the screencast, for display of popups for remote projects linked to a fusionforge project with the ‘extsubproj‘ plugin I already blogged about.

This code is still new, but will hopefully extend to other forge resources.

In the meantime, I’d be glad to see other forges implement similar mechanisms.