Conduit recently reached 1000 commits, and with it came together some functionality that has been brewing for a while.
Far far far improved nokia n800 support
You can now sync files, photos, music, and videos to your nokia n800 in an intelligent way (i.e. its not just a dumb file/folder sync).
The ability to transcode said data to the most appropriate format for the device. FFmpeg/Mencoder is used to transcode audio/video, and gtk.Pixbuf is used to convert photos. The converters use conduits built in dynamic modular plugin system, so if someone wants to write a audio video converter using GStreamer then please contact me.
Also, those that are familiar with FFmpeg/Mencoder/Transcoding please feel free to contribute preset encoding profiles (video codec, size, etc). Presets are stored here (video), here (audio) and here (photo).
Another one of those conduit screenshots that always look the same
Looser dependence on gtk. This means that conduit can be run from a console (-c command line switch) without gtk/goocanvas installed. This reduces the memory usage by about 10Mb!
Rhythmbox support. A simple dataprovider has been added that is able to sync rhythmbox static playlists with things. Initially this is used to test sync music with n800 (with the ability to transcode audio formats), but in my ideal world, sync with rhythmbox would involve
Improving rhythmboxes DBus interface to support adding/removing songs to the library and querying playlists (including smart playlists).
Remember because we are gnomevfs aware some cool things are possible. For instance, one can sync + transcode videos/music to/from their phone which is mounted using gnomevfs-obex.
Work is currently proceeding furiously on
Integrating with opensync
Removing the twisted dependence, and using pythons built in xml-rpc server for network synchronization
Replacing our current in memory database with one based on sqlite
Support for iPod photos
Test coverage andbug fixing.
Anyway, svn HEAD is pretty rocky at the moment, but please experiment. Once the database changeover has taken place things should stabilise for a 0.3.5 release. This will be a release candidate and then we will focus on bug fixing only.
Ideally we will release a stable 0.4.0 release at or before gnome.conf.au. As an aside, I would be interested in talking to any of the RH online desktop folks at linux.conf.au as I am unable to make it to any of the northern hemisphere conferences on account of me being poor and in New Zealand.
[1] Excluding a dependence on gnomevfs (yuck). Hopefully gio/gvfs will make all my dreams come true.
Some of you out there might be familiar with Gloobus. Over the last few nights I spent some time integrating Gloobus inside nautilus.
This is a proof of concept. I have done very little so far - it shows the first 8 files in the directory, and allows you to navigate between them with animation. It is mostly just a port of Gloobus from C++ into a ClutterGroup derived Actor in C, most of the thanks should go to the Gloobus author.
It features the same bugs as Gloobus, like poor support for resizing the window, positioning bugs, and it does not scale to very many files. It also leaks like a sieve (that one's on me). Here is my inadequacy represented in video form.
Those of you who regularly work with computers and other technology will be aware of how important it is to have a good quality internet connection. If you are looking into getting a new broadband connection then I recommend you look at O2 uk broadband reviews.
However, if you would still like to take a look and perhaps fix all the bugs, the steps for testing it are
The PyGTK team is pleased to announce the return of the highly popular all-in-one installer for Windows.
The PyGTK All-in-one installer provides an alternative installation method for PyGTK users on Windows. It bundles PyGTK, PyGObject, PyCairo, PyGtkSourceView2, PyGooCanvas, PyRsvg, the gtk+-bundle and Glade in one handy installer.
Currently 32 bit Python 2.6 and 2.7 versions are supported on Windows XP and above.
Dieter Verfaillie deserves enormous thanks for this work. Firstly, he performed the tedious job of ensuring that all the component MSI installers were exactly correct, and secondly, the really difficult task of deconstructing these individual installers and reassembling their contents into a single cohesive executable.
This is a true all-in-one installer, it does not simply call out to launch the individual MSI files.
We are looking to collaborate with others who want to create gtk+ (and friends) all-in-one installers for Windows. We anticipate the tools to generate these installers will move to GNOME in future - perhaps in a common repository. Suggestions and feedback welcome.
In order to add evolution support (contacts, task items and memos) to Conduit I needed to be able to access the relevant parts of evolution-data-server from python. evolution-python is an incomplete wrapper around libebook and libecal. It has now got to the stage of being useful, so here is a somewhat nervous announcement of evolution-python v0.0.0
This should be considered a work in progress and will be changing a lot over the next few days as I implement methods needed for the two way synchronization of evolution data using Conduit. However the basic skeleton is there, and I invite people to help with the bindings. An easy way to access evolution-data-server from python has been missing for a while now so I hope evolution-python will be useful for others.
There are some known issues and items to work on listed on the project website. Here is a small screenshot of the included test application.
The main limitation of the current implementation is that is only supports
the PointXYZ point type. PCL is a heavily optimized and templated API,
and the best method for creating specializations correspoinding to the correct point type
in a dynamic language like Python is not clear.
Nevertheless, the binding is already capable of smoothing, filtering and
the fitting of geometries in arbitary 3D point cloud data.
The binding is written using Cython, and is one of the more complex C++
bindings I could find.
The current release has been tested with
pcl 1.5.1
Cython 0.16
although it should work with more recent releases.
I would be interested in adressing the specialization issues using the recently
added and improved
fused types support in Cython.
This work has been supported by, and is currently in production use at,
Strawlab.
In June 2013 I was lucky to speak at the fantastic SciPy2013 conference
(scientific computing with python). I spoke about a work flow and tools we have
developed at strawlab. The title of my talk
was Managing Complex Experiments, Automation, and
Analysis using Robot Operating System.
The video of that
talk is included below;
This post describes a tool I developed for
distributing ROS packages to scientific collaborators. That software is
called ros-freeze.
For those of you not aware, ROS is a great framework
traditionally targeted for robotics but usable in other fields too. In particular
it has a relatively good performance and simple, strongly typed, inter-process-communication
framework and serialization format. This is simultaneously useful for creating
distributed realtime-ish systems with comprehensive logging of the system
state. Best of all, the python interface to ROS is very clean.
Unfortunately, being a framework, ROS is rather all-or-nothing (going as
far as to describe itself as a meta-operating system). The basic ROS
install is several gigabytes, and building it yourself can be rather
difficult. Furthermore, as I mentioned in my presentation, it is attractive
to use the built in ROS tool rosbag for recording timestamped data
to disk. Unfortunately, reading these files again needs ROS,
thus necessarily coupling experimental data to the software used
to collect it.
To remedy this I wrote ros-freeze,
a python tool to convert any ROS package into a pure-python package including
all of the dependencies. Collaborators can then install the python
package and immediately have access to all the same ROS packages and libraries
without having to build the whole ROS stack.
this is currently working on ROS Electric (an old release, at work
we have chosen to stick with Ubuntu 12.04LTS)
changes for other ROS distributions might be necessary, so please get in touch
this is successfully tested on ROS packages containing tens of thousands of lines of
code and dozens of ROS dependencies.
although recent ROS releases have improved the package management situation (by embracing
deb packaging, yay!) this tool provides an unprecedentedly easy way to distribute your
pure python ROS applications
A Pure Python ROS Distribution?
One side effect of this was the packaging of the pure python ROS core as an easily installable
python egg. This means that you can write, debug and test python ROS nodes without having to
install the whole ROS distribution.
One can even go as far as running rosmaster and the command line tools
(rosnode, rosparam, etc)!
I just committed the first version of a working GObject binding to the Conduit DBus API. You can check out an example test program here.
Contrived Example-like Code
/* The running conduit app/daemon */
ConduitGApplication *application = conduit_g_application_new ();
/* Get two dataproviders, a datasource and datasink */
ConduitGDataprovider *source = conduit_g_application_get_dataprovider(application, "TestSource");
ConduitGDataprovider *sink = conduit_g_application_get_dataprovider(application, "TestSink");
/* Put them in a conduit so we can sync them */
ConduitGConduit *conduit = conduit_g_application_build_conduit(application,source,sink);
g_signal_connect (conduit, "sync-progress", (GCallback) _sync_progress_cb, NULL);
g_signal_connect (conduit, "sync-completed", (GCallback) _sync_completed_cb, NULL);
g_signal_connect (conduit, "sync-conflict", (GCallback) _sync_conflict_cb, NULL);
/* Add it to the GUI so the user can see it (optional) */
ConduitGSyncset *gui_syncset = conduit_g_syncset_new(application, "/syncset/gui");
conduit_g_syncset_add_conduit(gui_syncset, conduit);
/* Perform the sync (boom!) */
conduit_g_conduit_sync(conduit);
g_main_loop_run (mainloop);
Conclusion
A GObject binding to the Conduit DBus API means you can synchronize your application data, from within your application, in very few lines of code;
Your application is not going to crash
Your application is not going to hang or block the UI
The difficult sync logic for interfacing with various other places can be written in Python (faster, more productive, most web-services get python bindings first), live in the Conduit source tree, and immediately be available to all other applications which use Conduit to sync stuff.
In SVN NOW!
Hat-tip to Mikkel Kamstrup Erlandsen whose Xesam Glib bindings provided a useful starting point, and saved me much autotools pain.
Yes, the names are a bit long.Who has a better idea for the prefix, gcond_ perhaps?
Well the first week of summer of code has finished. This week I spend my time evaluating and testing the various options available to (semi)automatically wrap C code (libsyncml) so that it is accessible from Python. My priorities when evaluating the options go something like,
Capability - the tool should be able to (semi) automatically wrap a large majority of the libsyncml api. Any customizations required in order to make the wrapping more complete should be readable and maintainable by people other than myself.
Documentation availablitly. Follows from #1, can I actually learn and use the tool within the SOC duration.
The wrapping tool is actively developed.
Does not introduce additional runtime dependencies other than the library being wrapped.
Minimal compile time dependencies when creating the bindings.
Community service value (i.e does the selection and use of the tool bring a positive benefit to the FOSS ecosystem greater than the actual library being wrapped).
The following is a list of available options I looked at (see cython for more explanation)
Pyrex
Produce very nice and clean C file, which you just compile to .so and that's it. Allows to wrap almost any C and C++ code. IDL is python-ish.
Cython
The same as Pyrex, but some new nice features added
SWIG
The defacto standard I guess. SWIG is one of the oldest and most mature methods of wrapping C or C++ code into Python (SWIG works for other target languages as well). SWIG produces a C file from an IDL, which gets compiled to a .so, but then it also produces a Python wrapper on top of this. Because Python wrappers are written for you, if their design is not exactly what you want, you end up doing more work to create your final Python API.
SIP
Similar to SWIG, but only aimed at wrapping C and C++ to Python. Unlike SWIG there is no Python wrapper. Used by PyQT and PyKDE.
Boost.Python
Writes C++. Not evaluated due to the additional dependencies required.
Ctypes
Ctypes is included standard in Python 2.5. The IDL is typically a python class hiding the ctypes calls, making the API more pythonic. It allows one to call library functions defined in shared object libraries directory from interpreted Python code.
Py++
It generates Boost.Python wrappers. Not evaluated.
f2py
It's mostly for wrapping fortran files, but it can also wrap C files, even though it's not a very well-known feature. Not evaluated
PyD
This works like boost.python, but for the D language. Not evaluated.
Interrogate
This works similar to SWIG. It created dynamic link libraries that can be used both from python and c++ via the Python C API. No other files are needed. Its not very well documented but is used in several commercial mmorpg's and is native to the Panda3d engine. Not evaluated.
Robin
Insufficient documentation to evaluate.Similar approact to swig, sans the intermediate IDL.
PyBindgen
The IDL is itself python, and it generates clean readable dependency free C code. Designed for wrapping C++, but has some support for wrapping C libs.
pygobject (codegen.py and h2defs.py)
The Gobject way, and the way I am most familiar. Unfortunately, in order to wrap the libsyncml library I would first need to wrape it in GObject.
Conclusions
The libsyncml library uses the Gobject mainloop, and custom error types. In order to integrate this with pygtk applications It would need to link to Pygobject/C, and propogate the error types to exceptions.
Somewhat unsurprisingly, the weak point in almost all of these approaches is there documentation. While I like the look of PyBindgen, it is a nightmare to build, and docs are sparse. The SWIG IDL is hairy, and one must also maintain pythonic wrappers to make a nice library. Pyrex and friends do not seem suited to the integration of libsyncml and pygobject without additional C glue
At this stage I am leaning towards SWIG, for community service value (others can come along after and make C# wrappers for instance), its availability of documentation, and even if the IDL is quirky, others are familiar with it.
Distributed Version Control Systems and visibility of development
My opinion on the 'best' DVCS is not relevant. What I am concerned about is that if GNOME does not pick one, and/or provide some sort of hosting or method to track other peoples development branches then the visible activity level, and subsequently health of the whole project will suffer.
The premise here is that centralized version control systems make it easy to follow what developers are working one, and the activity level of development, via the svn-commits mailing list for example.
I can only offer anecdotal evidence here, but I think that the visibility a projects development is just as important as the actual rate of development being done.
If developers cannot see what other people are hacking on, then there is the potential for duplication of work, or conflicting implementations.
If users do not see people actually doing work, then there is a tendency to assume the project is 'abandoned' or dead. The only thing worst than a 'dead' project is being proclaimed as such when one is not.
I consider the plethora of ways one can follow what developers are doing part of the problem, not part of the solution.
Who has time to follow planet, IRC, github, repo.or.cz, freedesktop git, launchpad.net bzr, mailing lists, twitter, $COMPANY gitweb, $PERSONAL gitweb, $DISTRO viewvc and gnome.org/$USER_HOME_DIR to see what people are working on.
This post is not meant to be Reductio ad absurdum, its just a slight generalization of why I read planet.gnome.org/svn-commits, etc, etc.
Part of the reason is to see what other hackers are up to.
Part is academic, to learn techniques and design from some of the great hackers on here.
Part is flagrant procrastination.
The small remaining part is the keep the voice in my head thats says "you should be using KDE, it appears to be more actively developed" at bay.
Conclusions, if any;
planet.ubuntu.com seems to have excellent visibility of active development, even if it doesn't have as many developers as other distributions.
freedesktop (via planet.freedesktop.org and http://gitweb.freedesktop.org/) seems to have excellent visibility of development (many people put git branches in their home directories, which are subsequently picked up by gitweb).
I am not advocating activity over productivity (obviously we are all free to use the tools which allow us to be the most productive, not just appear the most active). I just think that public FOSS development is an interesting space, in many ways the developers of the products are the marketers of it.
GNOME used to have the balance of visibility about right, but I think we are losing that with all this dilution.
As was the theme of yesterdays post, here comes some more software that I have hacked on recently and can now be found on Github.
osm-gps-map
osm-gps-map is a Gtk+ widget (and Python bindings) that when given GPS co-ordinates, draws a GPS track, and points of interest on a moving map display. It Currently supports a number of different mapping sources;
openstreetmap (default)
openaerialmap
maps-for-free
satellite maps from a number of proprietary providers
It also has the following features
Intelligent caching of maps, including the ability to request a specific area of the map to be cached ahead of time
Recording of points of interest on the map (and the ability to add arbitary pixmaps at those points
Automatically draws a GPS track (a line showing the history of past added points)
Automatic centering on new GPS points
Support for multiple other tracks of co-ordinate points
I have been running it on a number of embedded boards (beagleboard, Overo) for some commercial work I have been doing (hence the minimal dependencies). For those Planet GNOME readers, It is kind of like libchamplain but without the Clutter part.
facebook-notify
Reaction to the posting of my Facebook notifier for GNOME was positive, and I made a few quick fixes to remove some crasher bugs.
The biggest source of frustration at this point is that libwebkitgtk segfaults, upon destroy, almost every time it encounters a page that requires the initialization of an NPAPI plugin. Disabling plugins using the built in BrowserSettings object does not seem to have any effect.
Is there a RoadMap or any plans for an updated WebKitGtk release before the next round of distro updates?
Conduit
I have been hacking on Conduit a bit over the last few days, and thanks to Julien Lavergne we now have updated builds for Intrepid and Jaunty. Julien also packaged a trunk snapshot (and set up the infrastructure to allow easier updating of such snapshots). I recommend all users of Conduit add our PPA.
