Johns Blog

Conduit Is Online Desktop

After a short hacking hiatus, soul searching, and general uncertainty regarding Conduits place in the Gnome Online Desktop plan, we the Conduit developers have decided to keep doing what we do best; adding awesome features, and making Conduit into the best desktop synchronization service for the free desktop. I invite you all to spot the new features in the image below;

Going through the new features one by one;

• The Network datasink - thats how you tell conduit to share data to other people on the network

• The chad.local dataprovider - ... and thats how you'll choose to consume other peoples shared data

• The evo2-sync dataprovider - before long you'll be able to use opensync plugins in conduit

• The f-spot dataprovider - before long that will be two-way and networkable, work is ongoing with upstream

• Always-Up-To-Date is on the horizon (i.e. auto update and sync changed data)

• iPod photos support - our patches are accepted upstream, so soon we will support those too

• The facebook dataprovider - just photos so far, but more facebook features will be exposed soon

Other than those Massive New Features, there has been continued bug fixing and some improvements to the DBus interface to make it more Havoc Compliant. Check out the sample below for how to use Conduits DBus interface to synchronize things...

bus = dbus.SessionBus()

#Create an Interface wrapper for the remote object
remote_object = bus.get_object(APPLICATION_DBUS_IFACE,"/")
app = dbus.Interface(remote_object, APPLICATION_DBUS_IFACE)

#get a dataprovider
path = app.GetDataProvider("TestSource")
source = dbus.Interface(
bus.get_object(DATAPROVIDER_DBUS_IFACE,path),
DATAPROVIDER_DBUS_IFACE
)

#get another
path = app.GetDataProvider("TestSink")
sink = dbus.Interface(
bus.get_object(DATAPROVIDER_DBUS_IFACE,path),
DATAPROVIDER_DBUS_IFACE
)

#get a conduit
path = app.BuildConduit(source, sink)
cond = dbus.Interface(
bus.get_object(CONDUIT_DBUS_IFACE,path),
CONDUIT_DBUS_IFACE
)

print "Synchronize the conduit"
cond.Sync()

I have written an example which shows how to use Conduit to upload photos to any of our supported photo sites with even fewer lines of code. The DBus API documentation lives at the top of this file. Dont forget that by using a desktop wide sync framework you get all the Conduit bling for free, including

• Don't upload duplicates

• Newer photos replace older ones

• Asynchronous and all authentication is handled by Conduit

Conduit Network Sync

Excuse the shameless quoting of fellow conduit contributor John Carr's blog post but this video is pretty cool;

[googlevideo]http://video.google.co.uk/videoplay?docid=-2623798477165899506&pr=goog-sl[/googlevideo]

Video showing how you can use Conduit to synchronise Evolution Contacts and Tomboy Notes between two machines running the GNOME desktop environment.

To start with, laptop 1 has 2 contacts and 1 tomboy note. Laptop 2 just has 1 tomboy note. After syncing, both will have 2 contacts and 2 notes. Not too exciting.. but then... (on laptop 2) 1 note and 1 contact are deleted. The remaining note and contact are modified as well. When the camera flips back to laptop 1 and the sync happens you will see the windows update in real time - including the an open tomboy note getting updated.

Apologies for the poor quality

Conduit UI Experiments

The discussions regarding my proposal for inclusion of Conduit into GNOME 2.24 seem to be going OK. The inevitable issue of Conduit's user interface has been raised, and it is good to see some constructive comments being posted.

It is a bit of a simplification, but I see basically three schools of thought with regard to the user interface for synchronizing things. What follows is a brief discussion of the pros and cons of each approach.

1. Everything should be automatically configured, and the current Conduit user interface should cease to exist - synchronization functionality/UI should be put into the respective application, calling into Conduit over DBus.

   1. While I agree in theory, there is a enormous divide between theory and practice. For the foreseeable future it is not going to be possible to auto-configure synchronization parameters for all mobile devices. This is in part due to quirks in the devices, and in part because its sometimes not possible to enumerate all the things that a device supports without first configuring/pairing/connecting/etc with it.

   2. Many web services require a authorization step, either once per application, or once per session. To make this easier on the user, Conduit hosts its own web browser for showing the login dialog in a consistent manner. I believe that this is the least surprising result to the user, if they sync something in conduit, the authentication should be done in conduit, not in the users default web browser, where they may not immediately recognize that it is Conduit that requires the authentication. Until we get some more online-desktopy stuff into GNOME (sharing auth cookies, authenticating services, etc) I think our approach is  a good one.

   3. I think it is important to have a single, consistent interface for resolving conflicts, this is currently in the Conduit GUI, but perhaps we could export it over DBus, like we do for the configuration dialogs.

   4. Doing everything in other applications, reduces Conduits power to the lowest common denominator of these applications. Some of the cool power of Conduit is in the synchronization partnerships you can create that are not even tied to an application. For example, its possible to configure a 'sync' from Youtube, onto your iPod, that downloads and transcodes videos from you favorite user/channel. In what app would something like that go?

2. The conduit GUI should be simpler once you have configured everything.

   1. This is the approach I agree with the most. I think. This could potentially involve some sort of minimize/maximize type button that when pushed, would collaps the main canvas down into something like the iSync user interface.

   2. While I quite like that approach, I would almost rather implement an entire second GUI using only the Conduit DBus interface. This is the approach being persued by Alexandre (SOC student working on iPod support), and by John Carr (for a simplified windows mobile experience)

   3. It was also suggested to make configuration easier, we should try gtk.Assistant.

3. Redesign the entire UI

Redesigning the Conduit GUI Today I decided to work on improving the current UI based upon some feedback on desktop-devel-list. First I spoke with my user interaction designer friend, and asked her what she thought. After 15 minutes of talking to an imaginary person that did not exist (any volunteers to be my interaction designer friend?) I decided to instead make some incremental improvements to the current UI. Some people may see this as re-arranging deck chairs on the Titanic, but hey, what have I got to lose?

The first step was to use the current theme colors. After a bit of looking around, I couldn't actually find a simple way to see what these were, so I wrote a quick application that draws color swatches for all the theme colors, in each of the widget states. If anyone wants so play with it, the code is available from my git repository.

Rounded rectangles make anything pretty.

With an idea of what colors are actually present, I changed the Conduit and DataProvider colors to use sensible colors from the theme. I also implemented some subtle gradients to make things a bit more GNOME.

Just kidding.

The result turned out pretty nice I think. This is the first time I have actually tried to make a custom widget match the user theme, and it was not too painful. One annoying limitation is that I only seem to get access to the matching style, but I would quite like to use colors from the tooltip style. It also gave me the opportunity to remove some UI clutter; I no longer show sources and sinks in different colors. Anyway the final result was;

Beautiful in brown, blue and black.

When I said subtle I really meant it. I would really appreciate it if a talented artistic hacker had a look at the code, and experimented with some more creative gradient settings. The function in question is get_style_properties which lives in conduit/gtkui/Canvas.py, and is implemented in all CanvasItem derived classes.

I then moved onto experimenting with waking the user through the sync and configuration process. I did some experiments using the Gedit message area to suggest steps the user could take. Thanks to Colin and HotSSH for the widget. Im not sure about this one.

Too much like Clippy?

Mathias suggested a visible drop target. I think this will be my next challenge. I also did an example implementation showing what the Conduit UI would look like if the outer, grouping, rounded rectangle was removed. Now a simple line differentiates different between conduits. What do you think?

Now with 29% less rounded rectangles.

Summary

• I'm pretty happy with the themed colors. The work is available in my bzr branch, and will make it to trunk soon

  • bzr branch http://bzr-playground.gnome.org/~jstowers/conduit/devel

• There is continued discussion regarding UI ideas happening on live.gnome.org

• My Gtk theme swatch too is available from

  • git clone http://gist.github.com/49799

• There will be a Conduit UI, of some sort, for the forseeable future. Lets spend some time making it better!

Updates

• Fixed BZR Urls.

• Added an example showing what Conduit look likes using a dividing line to differentiate dataproviders.

• The theme swatch utility has been moved to github

Conduit UI Experiments - Part Two

Thank you everyone for their constructive comments on my previous post. A number of posters suggested the rounded boxes that conduit uses to group dataproviders were too bold, unnecessary, or both. I would like your opinion on which of the following four choices is the most suitable as a replacement.

Which is the best; 1,2,3 or 4?

Number four is the current implementation. I am leaning towards the first option as a replacement. From testing it seems to look the most consistent on different themes, and I believe it is the most suitable background to place (future) drag and drop hints on.

Conduit UI Experiments - Some Conclusions

Thanks once again to the many people who commented on my last two blog posts. All the feedback has been gratefully received. I just released Conduit 0.3.13 which, amongst other things, features some UI improvements based upon this feedback.

Improvements to the Conduit user interface.

Technically speaking, the release adds a knowledge framework to the application. This makes it easy to assist the user as they work through the interface. New features, utilizing this framework are;

• Conduit now ships with a few example synchronization groups. These groups are shown to the user based upon the devices connected, and the data providers loaded. When the user selects an example, it is added to the canvas automatically.

• I have added a message area to the bottom of the window. This provides a limited number of hints to the user depending on their interactions with the application.

Coming Up This release also lay a lot of the foundations for the few remaining big ticket items on the 0.4.0 TODO list. In the very near future we will see

• Merging GIO port

• Merging of Alexandre's SOC work, which will improve sound and video support in Conduit

• The first release supporting Windows Mobile synchronization

• Miscellaneous PIM improvements (Improved Google support, ability to merge attributes during conflict)

John Carr is currently working on the ATK accessibility implementation. As far as we can tell, there are no other PyGTK applications that have implemented accessibility in manner we wish to (manual implementation of the Atk interfaces from a objects not derived from GtkWidget). Lucly he works with Mark!

As far as using and distributing Conduit goes, it may get painful before it gets better. The next release will likely depend on PyGObject > 2.17 (for GIO support), and maybe PyWebkitGtk (SVN because of lack of recent, stable, compatible releases of this and WebkitGtk). My experience with (py)GIO so far has been exceptionally positive, and those out there who have used the train wreck that is gtkmozembed before will appreciate my desire to move to PyWebkitGtk.

Conduit Updates

Its been a while since I blogged about Conduit, and a lot has been going on. My main task has been working on the conflict resolution UI and on tweaking the conversions between datatypes so as to maintain maximum fidelity. John Carr, the other main Conduit contributor has also been working on some awesome stuff.

Conflict Resolution

During the synchronization process a conflict can occur in any of the following scenarios

1. In a one way synchronization, if the destination data is newer that the source data (the exported data shoud not clobber user updated data case)

2. In a two way sync when the destination data has been modified since the last synchronization (the two computers sync to the same location but both have had source changes case)

3. In any situation when (for backend specific reasons) a comparison between data is unable to determine which is newer than the other.

4. A piece of source data has been deleted and the user has selected a synchronization policy in effect which states they want to be certain before deleting the last remaining copy of that data at the destination. (I handle delete as a special case of a conflict)

Dataprovider backends communicate conflicts to the UI via signals. The UI has access to the conflicting data so can offer the user the ability to compare which is newer. Conflicts use the same arrow metaphor that the main UI uses and can either be resolved in the main window, or be shown with more detail in their own window.

If the data has set a (gnome-open'able) URI then the user can view the data at this location and decide which is newer. For example, a user is synchronizing his Tomboy notes via an iPod. If there is a conflict in this scenario then clicking Compare will cause Tomboy to show the local note and will launch a text editor to inspect the conflicting note off the iPod. (Lazyweb request: Any idea on how I might arrange the windows side-by-side via libwnck for example).

Fidelity

The main difficulty with synchronizing different datatypes, or the same datatypes but via a middleman is maintaining fidelity (that is not losing any information) during the whole process. In the case of contact or calendar data then the default would be to support just standard datatypes (vcard, ical, etc).

• Opensync 0.30 is looking to define their own XML schema for basic datatypes, and I would support this idea and likely adopt the same schema.

• Apple .mac uses strict datatypes and pushes everything to a central server. One alternative to maintaining fidelity is to keep data in all formats but maintain a single canonical source of modification times, and perform conversions (even if they lose fidelity) only when needed from this central store.

This leads onto my next suggestion Having an optional central repository (like .mac) not only addresses some of the difficulties of maintaining fidelity through conversion but also opens up some other use-cases which the open source world has not addressed.

• Online editing of tomboy notes while also being able to sync them on multiple computers (I can currently do this through backpackit.com but the conversions lose fidelity/information)

• Synchronize desktop settings (wallpaper, theme, nautilus prefs, etc between two computers)

• One click file sync with no configuration.

So lazywab hackers. Who here is a Django hacker that wants to write me a web service to do this. I already have a documented API that you could write against. I have the spec, the idea, but not the time. I had considered writing this myself, and setting it up as a paid service (I would pay a few dollars a month to be able to do this) but dont have the time, nor the web development experience, nor an inclination of artistic ability to make it look nice! Any takers, please email me!

Miscellany

• John has finished up all the remaining pieces to allow iPods to be hotplugged/removed in conduit, so that synchronization settings and state are preserved.

• He is also hacking on direct computer-computer sync over local network. Using avahi for discovery and pickling data over the wire. Should be awesome when this lands!

• Unfortunately I was not eligible for summer of code this year, I am on holiday from university for six months and wont be starting PhD till after the summer. I was pleased to see mention of Conduit in the Ubuntu SOC ideas page, so I am glad to see distros recognizing the importance of making synchronization easy!

• Releasing 0.3.0 is blocking on my having a working Ubuntu Fesity install to test against. (Lazyweb request: Does suspend,resume and hotkeys for panasonic sub notebooks such as the CF-R4 work on Feisty out of the box - it has not done so in previous releases)

Update: Google just released python GData bindings. This will make excellent support for Google Calendar, Google Notebook and a few other ideas I have up my sleeve a reality in the near future!. My blog is also back up, sorry for the trouble.

Conduit v0.2.0 Coming Soon

I have been working really hard for the last month on getting the Conduit core to a v0.2.0 ready state.... However after talking to a few people I have decided to delay v0.2.0 by a week to give more time for testing, and to increase the usefullness of some of the included dataproviders (Flickr and Fspot in particular).

The rationale behind this is, however cool an Idea Conduit is, it is only as useful as the things it can be used to sync with. I want this release to make a big splash, so I want the ability to sync with included GNOME apps to work really well. This means focusing a lot of testing on the Tomboy and F-spot datasources, and the Flickr datasink.

I encourage users to check out the latest version from svn to help in testing. In the meantime, here is a screenshot of Conduit running happily on Edgy.

Edgy Beta 1 Impressions

So I just upgraded my laptop to Ubuntu Edgy Beta 1. This is just a small list of my impressions so far;

• USplash doesnt work :-(

• My laptop hotkeys do not work anymore (so I cant change screen brightness)

• The new theme and sound theme are very attractive

• Compiz-freedesktop 0.2.0 works great, thanks to this guide

• Boot is way faster, 10 seconds by my stopwatch

• Suspend and resume is improved

  • NetworkManager reconnects to networks on resume

  • Compiz works after resume

• Gnome Power Manager battery status graphs rock

More news to come...

End of an Era: PyGTK

I just released PyGTK 2.24, which will almost certainly be the last major PyGTK release. The future of Python on the GNOME platform is PyGObject + GObject Introspection. From my experience over the last few months porting a number of my projects, the future is bright.

In a cruel twist of irony, the state of PyGTK on Windows and Mac has never been better. The credit for the windows work (and some great documentation improvements this cycle) must go to Dieter Verfaillie

• Windows installer (homepage) (download)

• Mac improvements (all in one installer work) (bundling advice)

I hope that the new stack will reach the same level of capability on other platforms as GTK+ 2.24, but in the large scheme of things the renewed development excitement surrounding GTK+ 3.0 and GNOME 3 is excellent consolation.

As a user I would like to thank those developers before me for creating PyGTK. It was the first 'pythonic' UI toolkit for linux, and a pleasure to use. As the recent maintainer of PyGTK I would especially like to thank those recent developers who helped me, in particular Dieter Verfaillie who really pushed PyGTK over the line regarding Windows support, into the great state it is now.

I'll leave you with some graphical statistics (generated using pepper) for the 12 year history of PyGTK. If planet strips the wordpress gallery then please click here.

update: To clarify a point raised in the comments, PyGTK will be maintained in the exact same was as the GTK+-2.0 series will be maintained. Bug fix releases will be made if necessary, but no new features will be added. If you want the new GTK+-3.0 features then you should use PyGObject + GObject Introspection.

The PyGTK code will not disappear from any servers, it will continue to be shipped in all distributions for the forseeable future, it will continue to work very well on windows, and many applications will continue to use it.

Epiphany is AWESOME!

After reading this post I have now switched from firefox to epiphany. I would have done so earlier if this was hyped during the (or Davyd's) 2.14 release notes!

My favorite features (so far) are

• Extensions (especially integration with liferea)

• When a page cannot epiphany offers the GCace or Archive.org version

• Starts faster

• Taggable style bookmarks!

• Deskbar integration

• Download manager (with notification area icon)

I would recommend any gnomie out there to switch to epiphany today!