Aug 23, 2008
Summer of Code
This has now finished, and I am really happy with how it went. I was able to complete a Python binding to libsyncml. This was done with the help of Pybindgen, which aside from a few quirks, performed admirably. Expect this to become the premier tool to automatically create python bindings to C/C++ libraries. The binding still contains too many bugs to be considered usable in Conduit trunk (read: crasher bugs) but I can see the light at the end of the tunnel.
Alexandre Rosenfeld was also successful in his Conduit Summer of Code project. He contributed audio and video support for the iPod, and a comprehensive audio/video converter/trans-coder using python-gstreamer. The iPod support seems quite comprehensive, and the converter component is a necessary component of the Conduit architecture for our future plans.
Conduit
While it may look like I have been dormant at Conduit hacking, work has been ongoing in a number of branches. Unfortunately I still cannot push these branches to bzr-playground.gnome.org because the SSH keys from GNOME have not been synced across in over two weeks, leaving me locked out.. The GIO port is now working (with the exception of gvfs bug #547133, which I would dearly like someone to commit the fix for).
One of the major tasks necessary for the GIO port was the isolation of the platform specific parts, such as GConf, and GnomeVFS. One thing that fell out of this work is that Conduit now works on Windows. With no (~10 lines) code changes. Amazing really. It should be noted that this is not actually using GIO on windows, it is using a pure Python File class implementation.
Conduit can haz Windows!
I am not really serious about maintaining this port, but it shows what is possible. If someone wants to hack on this I can point them to the necessary places. But basically you will need
I have also moved over to using PyWebkitGtk for the Conduit Web browser. They just made a 1.0 release, and I would really like it if those responsible for packaging Conduit, could please also package PyWebkitGtk, and ensure that it gets the necessary exceptions so that it is accepted into the appropriate distributions. Words cannot express how happy I am to be rid of gtkmozembed. It is a shame that webkitgtk was not accepted as an external dependency for GNOME 2.24, as this now makes getting things like pywebkitgtk into distributions a royal PITA.
Openstreetmap GPS Widget
Some time ago I mentioned the osm-gps-map widget that I have been working on, semi-related to my PhD. I just made the inaugural 0.1 release. This widget basically lets one easily write moving-map display widgets very easily, showing points of interest, and multiple gps paths atop of tiles fetched from openstreetmap, or other mapping providers. It started as a port of Tango GPS, and can now basically do everything that application did, but behind a simple GObject API. Python bindings are also provided
Miscellany
Oct 27, 2006
Words cannot describe how much I am looking forward to the Borat premiere. Before you see the premiere make sure you watch these deleted scenes.
[youtube]Om7SkkN2T7c[/youtube]
Nov 13, 2006
This wont be a particuarly coherant post, More a collection of links that I dont want to lose, and that others might find interesting. Also some updates on the foo that I am working on.
One of the great things I like about free software is that there is always something exciting around the corner. Sometimes it just takes way to much procrastinating to find out about it. Here are a few things that I am looking forward to in the next 6 months;
-
Xorg 7.2 with RandR - Should make managing multiple monitors painless.
-
i810 Modesetting branch. Thanks intel for your rocking OSS drivers.
-
Simple way to theme colours in GNOME.
-
Some new GNOME infrastructure;
-
Glade-3.1 (single window mode) - Rocks!
-
Nemiver is awesome, and i'm using it daily.
-
I want to play with pigment, I particuarly like the idea of having a built in animation framework, making it easy to create fluid user interfaces; like
-
Photosynth (made by microsoft - credit where credit is due).
-
Canola - WOW thats running on a 770!
-
Aerith - Eeeww thats java (but hot)
-
I have been running Tracker for a long time, and can recommend it to everyone. I would love to see this go into GNOME along with a bung of helper Gtk widgets to make tagging a first class citizen on the GNOME desktop.
-
After the terribly shortsighted Beryl fork its good to see Compiz get a community website. I wish I had a dual screen graphics card to test all the multihead support stuff that DavidR has just added to compiz.
I have also been super busy of late. I have been working hard on Albatross, interfacing with flightgear ad JSBSim, and learning about Eigenstate assignement for controller design. It looks like my next task is to clean up the AVL stuff and start running some code through Matlab to test these theories.
Finally I am back working on Conduit, and it looks like its the right time for stuff to come together.
-
Rewrote the TreeModel stuff to support adding things at run time which is important because....
-
OpenSync just released 0.20 with better python bindings, so I can start to play with cellphones!
-
I added service discovery of other conduit instances using avahi. This will eventually allow things to be synchronized with remote computers directly and transparently over the local network.
-
Started to add ipod support. When an ipod is plugged in new datasinks should become available which will allow you to synchronize your ipod notes and contacts (and possibly photos thanks to GPixPod)
Anyway, thats all for now. Talk to you later or see you at linux.conf.au.
Feb 22, 2008
-
Multiple site photo upload
-
PIM sync to your desktop (including over the air sync using our network sync support)
-
File/Folder sync to your desktop and to online services such as box.net
-
The same codebase as Conduit for the desktop
-
The awesome work of Thomas Van Machelen
-
Did I say shiny?
Aug 9, 2006
After a solid month of work I just release v0.1 of Conduit.
Conduit is a synchronization solution for GNOME which allows the user to take their emails, files, bookmarks, and any other type of personal information and synchronize that data with another computer, an online service, or even another electronic device.
This version is the first version which is actually usable so I thought I would release it to gather feedback, particuarly about the GUI, and how people interact with the application.
The next version will bring more bling including
-
Treeview for datasources and sinks. They will be categorised into groups to more easily distinguish between them
-
Threading cleanups so that sync's may be cancelled while in process
-
Flickr datasource and sink
-
Two way sync for backpackit.com and files
-
Gmail dataprovider tidy ups
I am looking for help and interested developers. Check it out!
http://www.conduit-project.org/
Oct 3, 2007
Wahoo!, after mad busy weeks of development, I just released Conduit 0.3.4. Check out the screencast (download here). This release is fresh with new features, and heaps of polish and integration work. These new features include;
-
Google calendar support (Paul Novotny).
-
Youtube dataprovider added (Renato Araujo).
-
Photo sites are now two way, you data is no longer locked with one provider. Also see #480754 for improvements to Fspots DBus interface allowing two way photo sync with FSpot.
-
Nautilus extension. (disabled by default).
-
EOG plugin added. (disabled by default).
-
Conduit now supports resizing and converting photos before uploading them to photo sites. This infrastructure means improved support for other types of conversions, such as transcoding music and video, will be added in the future.
-
Progress indicator shows percent complete during synchronization.
-
Add support for Facebook photo upload.
-
Add included web browser for authenticating against web sites.
Website Authentication
The last point deserves some explanation. Currently, to authenticate against some sites the user must log in either once (Flickr), or once per session (box.net). This is difficult when using the system default web browser, if the browser is currently running on a different virtual desktop to conduit, the user often doesnt notice the web browser go to the login website. This is compounded by the different APIs of some sites. Flickr is good, I can just call am_i_authenticated(token) in a loop, and wait for success, or time out. Facebook is not good. Any call involving the facebook token before it has been validated (i.e. user logs in) invalidates the pending login. Because there is no guarenteed way to block in all cases (i.e. until the user logs in and closes the window), things like gnome.open_url() or python.webbrowser() are unsuitable. This leaves me with two options. I can wait a random amount of time, and hope its long enough for the user to log in, or I can write my own web browser that will block until the user closes it. Suggestions for option 3 are appreciated....
I know this is not ideal. I hope something like web-login-browser can solve my needs. Also, I know my gtkmozembed based browser has some (crasher..) issues (maybe related to this). I hope to fix them and/or move to a pywebkitgtk based browser, depending on what GNOME favors.
Here is a screencast showing the latest version and its features. You may download the original here. Apologies for the google video quality.
[googlevideo]http://video.google.com/videoplay?docid=-262844619875208739&hl=en[/googlevideo]
Boston Summit
Unfortunately, owing to my proximity to the opposite side of the world as Boston, I will not be attending the summit. I understand that the GNOME online desktop is on the agenda, and will no doubt be discussed at some length. I hope those involved in the discussion try out conduit before the summit, as it would be a shame if the future of the GNOME online desktop was set without due consideration. This is not a criticism, sour grapes, or anything like that. Its just a reflection of what appeared to happen after GUADEC - which I also did not attend (New Zealand = isolated and distant).
Have fun with Conduit. Please report bugs, join the mailing list, and abuse me at will. We are currently moving to GNOME servers, so if my server goes down from all the traffic I promise it will be the last time.
Jan 17, 2008
Quote courtesy of John Carr, who I had cook me up a patch at 1:30AM UK time. Ahhh the wonder of a 12h time difference.
I just released Conduit 0.3.6, a quick follow up to the last release. It fixes a number of bugs which were preventing some common sync scenarios.
-
Download
-
Release Notes
-
Report Bugs
-
Mailing List
-
Test Results
-
Bugs Fixed
-
Fix two-way Tomboy <--> folder sync
-
Fix two-way GConf <--> folder sync
-
Dont recreate the backpack API everytime we refresh
-
Fix Shutterfly to support image uploads and delete again (Jeremy Slater)
-
Fix attribution of Banshee dataprovider. It was contributed by Don Smith
-
Many fixes to image dataproviders
OK I lied, there is one new feature..
Feb 14, 2008
I just released Conduit 0.3.7. This was another regular release which contains the usual array of bug fixes and a few small new features and tidy ups. This was also the first release made automatically using maintainer. Aside from a few quirks, the tool worked amazingly well.
The next release will feature rewritten user documentation, more conflict UI improvements, and fix whatever bugs crop up in testing its compatibility with new GNOME. Highlights of this release are;
-
Update to use the newer Tomboy Set/GetNoteComplete API. This finally allows lossless sync of Tomboy notes including metadata such as tags and notebooks.
-
Update to the latest versions of our shipped static libraries
-
flickrapi (fixes many Flickr bugs)
-
pyfacebook
-
python-gdata (and use this for Picasa support)
-
libgmail (make emails work again)
-
The Eye of gnome plugin for uploading photos to multiple sites now shows status while synchronizing
-
Add one-way support for syncing Gmail contacts (with evolution, ipod, etc)
-
Add a 'developer' sub-menu to the help menu if the user is running a development version. This sub-menu contains items which link to the newly improved developer documentation. The linked pages are displayed in conduit's built in web browser.
-
Make the Conflict preferences per Conduit add descriptive icons to the preferences menu
Update: I completely forgot. Thomas Van Machelen recently added C# bindings for our DBus API to SVN. If any of you enterprising Novell Hack Week hackers wants to play with them, I have some ideas (use Conduit to export photos from FSpot, use Conduit as the sync engine for Tomboy, for both synchronization of notes between two computers and/or export to ipod, box.net, etc). Please email me if you are interested. I know Conduit is ready, there are many examples of the Conduit DBus API in SVN.
Sep 25, 2006
So I finally got around to releasing Conduit 0.2.0, which, while having many limitations, is relatively useful and safe to use on a daily basis. I have been swaped by university work at the moment, as this is the last week, but I hope to be back developing again soon.
Other Things on the hack-agenda over the next few weeks (months)
Mar 10, 2007
A lot has been happening on the Conduit front of late, and I probably should have blogged about it earlier. Unfortunately, it has seemed as though I have made 5 steps forward and 4 steps back with every hour of work on the project, so why, and whats up?
Background
I will be dropping a 0.3 release to coincide with the release of GNOME 2.18. This will be a developer release designed to get things working well before a stable 0.4 release some time after.
All of Conduits synchronization capability is determined at runtime. Conduit scans the users' and the system wide dataproviders directories in the same way that Deskbar loads handlers when it starts. The core synchronization logic is decoupled from the dataproviders, and only those relevant dataproviders are shown to the user (i.e. only show iPod Notes if an iPod is connected, etc).
Remember that I want Conduit to be a GNOME wide synchronization service that other applications can hook into, so I need to not only decouple the core sync logic from the application and usage specific dataprovider back ends, but also allow conduit to be extended by third parties.
Extension by 3rd Party Applications
I mentioned that all synchronization capability is determined at runtime. The other half of this story is that a third party may wish to synchronize something that Conduit knows nothing about (I'll use a Jokosher project as an example, although in my next post I will address these issues in the Tomboy case). Lets assume that a Jokosher project cannot be described in terms of the basic types of data that Conduit knows about (email, contact, file, etc). In order to allow Jokosher projects to be synchronized between different computers the Jokosher developers would need to do the following
-
Define a Jokosher datatype that describes what it is to be synchronized.
-
(optionally) Define some conversions between the Jokosher datatype and the basic Conduit datatypes (File, Text, etc)
-
Define a dataprovider that encapsulates Jokosher invocation specific information such as the Jokosher user, and how does one go about extracting/modifying Jokosher specific data when Jokosher may be running.
The Conduit framework will do a lot of heavy lifting here, such as caching modification times, providing a way for the user to configure a Jokosher sync partnership, providing a way for a user to resolve conflicts, and even allowing the whole sync process to be controlled from outside of the application over DBus. Most interestingly however, depending on the fidelity and number of conversions specified in #2, Conduit will facilitate a number of other useful sync scenarios including,
-
Sync Jokosher projects via any gnomevfs compatible location (ftp, etc)
-
Sync via Amazon S3 (coming soon)
-
Sync via USB key
-
Sync directly over a local network using Avahi. (next release)
The point here is that this decoupling of the dataproviders, from the core and the datatypes (sans the conversion functions) lets the Jokosher dataprovider take advantage of any/all additional $FUTURE dataproviders (e.g. Amazon S3) and all improvements to the Conduit core (such as direct sync using pickle'd python over network via Avahi).
Testing
As I get closer to release I get more nervous about releasing a tool which will could eat peoples data. To mitigate this risk I spent a week or two implementing an extensive testing framework for Conduit, a task I should have tackled much earlier in the project.
Unfortunately the testing is made more complicated when the applications capabilities can change between invocations (dynamic loading of dataproviders and conversions explained earlier). Aside from the usual unit level tests (does this throw the correct exceptions, etc) I am also required to test the fidelity and accuracy of the conversions between datatypes (round trip comparisons), and the fidelity of synchronization from dataprovider $FOO to dataprovider $BAR via any of these datatypes.
The testing framework uses a mixture of python and shell script, and is even web 2.0 compliant ;-). It currently tests available conversions, saving and getting data from various dataprovider back ends (Flickr, gnomevfs, Tomboy, etc). It even does code coverage and pushes the results on line.
What's Next
I apologize for this post being mostly academic in nature, lacking in screen shots. I will post again in a few days with some more relevant information (from a users perspective). I just wanted to highlight how and why I believe a desktop sync service is useful, and how I am trying to satisfy this in a sustainable way for the GNOME platform as a whole.
I anticipate finishing up the two way Tomboy support (via iPod notes, gnomevfs location, and backpackit.com) in the next few days. There are some bugs preventing running Conduit on Feisty (goocanvas 0.6.0 is ABI incompatible with 0.4.0). There are also some other lose ends (like some GUI blocking calls in conflict resolution, and some difficulties saving program configuration). Unfortunately, like a lot of FOSS, the best is always just around the corner....
Conduit can haz Windows!