Canto | News

First, the bad news. I didn't make as much progress over the weekend as I'd hoped. In fact, I really achieved only one thing from that whole post and it was a side-effect of some other developments.

I threaded the Canto interface not because I really wanted to take advantage of multiple cores or get a speedup in the actually task of filtering and sorting item. I threaded it to get the interface to be more responsive. To an extent, it worked. The interface became more responsive, but only because in implementing the thread model, I had to implement partial updates. It effectively removed the problem with long lists by allowing you to use the interface when only part of the total work was done.

I never really cared to ponder why, as a whole, performance didn't really work out until I read David Beazley's slides on the GIL (PDF). It really clicked with me why I was still running into trouble, even with the threads.

Now, the good news. I converted the entirety of the threading model to use processes, which avoids the GIL all together by communicating through traditional pipes. No GIL locking required. Worst case scenario is a moderate speed up (because the locking overhead is completely gone). Best case scenario is a hefty speed up brought on by today's multi-core processors. I've noticed a big difference on my Core2 even when one core is being dominated by another process.

There are still some issues in the implementation (interested parties can glance at the git log. But as a whole, the process based code is not that much different than the thread based code was. Everything is still protocolized but the protocol has to be entirely comprised of pickle-able objects.

I don't anticipate too many problems directly related to the new process based code, but it's possible. Also, you're going to take a hit in memory usage because the same set of objects has to be synced between the two processes, but the dumb process doesn't keep too much state by default so it's been minimized. I also don't think it's enough to outweigh the memory gains made by keeping more content on disk so overally the memory performance is worse than when threading, but should still be better than 0.6.x.

New dependency for 2.5

To ease the use of processes, Python 2.5 users will have to install python-processing. Fortunately, this module has been integrated into the Python standard library in 2.6+ as the "multiprocessing" module.

Testing

As always, the code is in the experimental branch. If you 'git' it, feel free to report on the mailing lists or #canto on irc.freenode.net.

Have fun, submit bugs!

Since the last update, the merging of the validating config code, there have been a number of improvements to the experimental source. Mostly bug fixes. A lot of things that have been broken for awhile have been updated to use the new data structures.

• Sorts are fixed
• Tag filters are fixed
• tags should now be totally implicit
• Sluggish reader scrolling fixed
• More canto.extra sugar for adding content to the interface

The most work has been done on code documentation though. At this point, basically every file has a large header explaining its purpose and highlighting important blocks of code, in addition to the typical as-you-go interpretation comments. I know I'm the only real developer on Canto at the moment, but I've added hundreds of comments to explain design decisions and give a good overview about the internal functioning of the project.

I'm excited to say that I think that the majority of the goals for 0.7.0 have been reached and are working in the code. I hope to put a major dent in the remaining areas this weekend, particularly in getting the new site / configuration / styling documentation up to speed. It's going to be a full rewrite to fix a lot of documentation deficiencies.

What remains

• Minor amounts of code documentation (there are a few files left)
• Website documentation rewrite
• Style opt-out flag / option
• Feed error reporting (will also keep Canto from trying to update 404'd or otherwise unfetchable feeds every time it starts)
• More complete config validation (i.e. validate filters/sorts that aren't put into typical places)
• Various canto.extra improvements for ease of use

Some of these are reflected in the bug tracker.

I can also say that there are still some performance problems with atypical usage. For example, if you open canto and immediately switch to another global filter, it lags because it's trying to update from disk and refilter all of the previously updated items. This can easily be resolved by setting your default global filter correctly in the config so you don't have to immediately switch and if you have to switch for some other reason (i.e. to find a read item when your default filter is show_unread), the penalty is only a couple of seconds. I am actively looking into making this work better ( have no fear =) ), but none of the poor performance cases are significant enough to keep me from releasing.

Anyway, keep watching for a release some time in the nearish future, I'll probably update here with a link to the experimental page with a beta before I actually put out 0.7.0 for the distros to pick up.

In the meantime, have fun and submit bugs!

I finally merged the validating configuration code into the experimental branch. Experimental users might want to read the last few git-log entries after they pull the changes but before they install the new code. In particular git log a9c86. The content is mirrored in gitweb here

The long and the short of it is that there's been some heavy changes to some particulars of the config and if you're running your own filters and sorts or are using sorts heavily then your config might need a little tweaking and the official documentation hasn't been written up yet.

As I mention in the log, if you have trouble, feel free to mail the list or drop by #canto and get some help straightening it out.

I missed the last week's edition, so this has a little more scope, but I've been pretty slammed at work so not much has been going on.

These past two weeks have brought a number of bugfixes in the experimental branch, but the main improvements were the addition of the state change hook (which will eventually be used for integration with other readers, like Google Reader), and the start of the validating modular configuration code (in the exp-cfg branch). I'm very pleased with how the new configuration code is going and it should offer much more informative configuration error messages and is much stricter so that strange crashes don't come from strange corner cases in the config.

Unfortunately, I've gone down some developer blind alleys in the past week. The first of which is that I finally got around to determining just how difficult integrating with GReader is going to be. As a result, I'm going to have to delay that until sometime after 0.7.0 because the update is already going to add enough features with their corresponding bugs that I'd like to get it out there. I am, however, going to add some changes that make GReader integration less difficult before the release.

The second is getting Canto to work on FreeBSD. I'm a Linux user/developer by choice and by trade, but I'm never averse to expanding Canto's range to similar operating systems. With this, I struggled for a little while, and ended up submitting a bug in the Python tracker about it because the WINDOW taken from the Python object is somehow not valid. I'm not sure if the Python tracker is really the best place for it, but I'm hoping that someone who knows a lot more about Python internals will take a look and help me out.

Any experienced FreeBSD programmers that want to take a look at it are certainly welcome to take a look as well, my experience with FreeBSD is very limited. I still hope that any changes won't be have to be very invasive so I can add FreeBSD support without adding much complexity.

Have fun! Submit bugs!

You can grab the 0.6.13 release in tar or OSX portfile format from the download page.

• Fix drawing regression.

Apparently I introduced a pretty bad bug in the last version. Thankfully evg_krsk from IRC saw it and reported it.

Have fun! Submit bugs!