Accelerated Compositing in webkit-clutter

For a while now my fellow Collaboran Joone Hur has been working on implementing the Accelerated Compositing infrastructure available in WebKit in webkit-clutter, so that we can use Clutter’s powers for compositing separate layers and perform animations. This work is being done by Collabora and is sponsored by BOSCH, whom I’d like to thank! What does all this mean, you ask? Let me tell me a bit about it.

The way animations usually work in WebKit is by repainting parts of the page every few milliseconds. What that means in technical terms is that an area of the page gets invalidated, and since the whole page is one big image, all of the pieces that are in that part of the page have to be repainted: the background, any divs, images, text that are at that part of the page.

What the accelerated compositing code paths allow is the creation of separate pieces to represent some of the layers, allowing the composition to happen on the GPU, removing the need to perform lots of cairo paint operations per second in many cases. So if we have a semi-transparent video moving around the page, we can have that video be a separate texture that is layered on top of the page, made transparent and animated by the GPU. In webkit-clutter’s case this is done by having separate actors for each of the layers.

I have been looking at this code on and off, and recently joined Joone in the implementation of some of the pieces. The accelerated compositing infrastructure was originally built by Apple and is, for that reason, works in a way that is very similar to Core Animation. The code is still a bit over the place as we work on figuring out how to best translate the concepts into clutter concepts and there are several bugs, but some cool demos are already possible! Bellow you have one of the CSS3 demos that were made by Apple to demo this new functionality running on our MxLauncher test browser.

You can also see that the non-Accelerated version is unable to represent the 3D space correctly. Also, can you guess which of the two MxLauncher instances is spending less CPU? ;) In this second video I show the debug borders being painted around the actors that were created to represent layers.

The code, should you like to peek or test is available in the ac2 branch of our webkit-clutter repository:

We still have plenty of work to do, so expect to hear more about it. During our annual hackfest in A Coruña we plan to discuss how this work could be integrated also in the WebKitGTK+ port, perhaps by taking advantage of clutter-gtk, which would benefit both ports, by sharing code and maintenance, and providing this great functionality to Epiphany users. Stay tuned!

My experience with GNOME 3 so far

You know, GNOME Shell and I are not really strangers to each other for a long time now. I have been using it almost daily as my main desktop since late 2009, when I started shipping it to Debian experimental. That means that I had ample opportunity to both get used to it and witness the huge improvements it had with every new release.

My general feeling towards GNOME 3 is this: ♥. Yes, I love it! I love the new themes, I love the window borders, I love the top panel, the overview, the dynamic workspaces, I love the Me menu, I love the clock and the calendar, the system indicators with the beautiful symbolic icons, being able to search for applications in such a nice way, the window animations, the multi-screen support, the new nautilus, the dash, looking glass. It’s hard for me to even express how thankful I am and how much admiration I have for the awesome folks who helped bring this to life. Thanks so much!

My GNOME3 desktop
My GNOME 3 desktop

After all this time, there are only two things I can say I dislike about GNOME3, apart from some minor wishlists: the alt-tab behaviour and the message tray. Let me expand on those.

Message Tray

Of the very few things I dislike, there is only 1 I hate and cannot see myself living with: the accordion animation in the message tray. No, really, it’s such a terrible, terrible idea. Every single time I try to use that thing I overshoot while moving the mouse to the left, then overshoot again moving the mouse to the right because the frigging icon has moved. It’s no good knowing that I can click in the text or in the empty area to its right, it feels wrong. It’s terrible that my actual target is moving at all. Every single time I use it is a small frustration for me – it’s as if the message tray was playing games with me, laughing at me for not having good enough mouse pointer driving skills – even more than the infamous sub-menus used to. And I had to go through that penance whenever I wanted to find the person I was chating with to resume the conversation.

I wrote an extension that disables the accordion animation by simply not showing the title at all when you hover the icon, and I patched GNOME Shell’s CSS to make the icons a bit bigger, so that it’s easier for me to hit them with the mouse. It’s clearly not ideal, and you still have to click the various “people” icons to figure out which of your friends who were lazy enough to not add a picture to their IM profiles is that one, but it’s still much better than chasing the (smaller) ones around to figure that out. Perhaps we should have the icons be bigger and always have the title bellow them? I don’t know, I trust the awesome designers who designed the awesomess that’s everywhere else will come up with a great idea.

My Message Tray
My message tray with bigger icons and no accordion animation.

Alt Tab

The number 1 feature of workspaces for me has always been locality – being able to not see all of the other applications and windows that are open elsewhere. This lowers the amount of noise when I’m trying to find something. The overview is very nice in this matter – only windows in the current workspace are shown, and even when you have an extra screen, the windows in there appear in that screen.

The alt-tab behaviour, on the other hand, of showing all windows and apps, even with the separator, bothers me. It’s really useful to have when you want to go to a specific window no matter where it is (I usually use the dash for that, though), but it adds noise when you want to go to a specific window _in this_ workspace, which is the most common use case for my usage. So I copied the alt-tab code over to an extension and modified it so that only windows in the current workspace would be considered.

In addition to that, with the windows in the extra screen always being there no matter what workspace you’re in (which I think is an awesome idea), they are effectively in all workspaces, so they add constant noise even with my extension. It’s also weird to have to look at the main screen to switch to a window in the extra screen. There’s a huge discontinuity. What I would prefer is having alt-tab to follow the mouse regarding screens – only show windows in the extra screen if you hit alt-tab in there, making sure the selector thing appears in there as well.

A clutter port of WebKit

In case you missed the news on webkit-dev, Collabora has been working on developing a clutter port of WebKit. It shares the build system with EFL, and most of the backend code comes from the GTK+ port. That means networking is handled by soup, drawing by cairo, multimedia by GStreamer, and so on.

If you’d like to give it a try, you can clone the repository from gitorious:

$ git clone git://

Then to build it you use cmake. From inside the source code directory do this:

$ mkdir build
$ cd build
$ cmake .. -DPORT=Clutter -DSHARED_CORE=1 -DBUILD_MX_LIB=1
$ make

The BUILD_MX_LIB option is optional – it will build what we call the “Mx toolkit library” in addition to the vanilla one. Then you can test that the library is built and working by running the programs inside the “Programs” directory. Enjoy!

WebKitGTK+ hackfest 2010!

Last week I attended the WebKitGTK+ 2010 hackfest. It was a great opportunity to meet up with the other developers, discuss some plans for the future, hack away at WebKitGTK+. But, most importantly, play Street Figher 2 =). Thanks to Collabora and Igalia for sponsoring the hackfest, Igalia for hosting and organizing it (well done!), and the GNOME foundation for having sponsored my trip to Coruña!

Unlike last year we didn’t find any big design issues hurting our work (page cache, I’m looking at you!) on new futures. I also didn’t have any huge plans for new API, although we did manage to get some new stuff in there, like the plugins management API Xan created, and the further work done by Dan in soup. This meant, from my point of view the hackfest has been a great oportunity to look at refactorings that we could do to further simplify understanding the code, changing it, and even sharing it =). Besides pushing the debian packaging of the 1.3.x series a bit further.

Coruña was great as always, and I enjoyed going around, eating and hacking there, although I got a cold on the last days which kinda hindered my ability to stare at the screen for too long, some times. Now that I’m at the hot brazilian summer again I’ll hopefully get better soon =)

Cheers \o/

Sponsored by the GNOME Foundation

Google’s User-Agent sniffing makes one more victim

Remember when I said Epiphany worked out of the box with Youtube’s WebM? Well, Google has recently decided to deny us WebM, like it did before with Wave, the Pacman doodle, and who knows what else? \o/

Wouldn’t it be nice if Google practiced what they preach?

Update: so it looks like my message went through to the people who needed to see it, and they found a filtering error in the User Agent sniffing code that made it think Epiphany was a too old Safari – I’m told the change will land in Youtube soon, thanks for those paying attention, and working on this! User Agent sniffing keeps being a problem, of course, and there are other stuff to fix, so I will probably still push my patch to spoof the user agent to google services which are still mishandling Epiphany, but it’s good to see some progress being made!

Update2: I started shipping a patch to send the Chrome user agent string to google domains in the Debian package for WebKitGTK+, when the “enable-site-specific-quirks” setting is enabled (which is the case for Epiphany); I already found something we were missing out on =D Google Images seems to have been greatly improved, and now faking being Chrome we are also able to enjoy it:

Google Images improved

WebKitGTK+ and the Web Inspector

When I started working on WebKitGTK+ I was a web developer, writing IT applications using Python and Django, and building features for content portals running Plone (argh). Even though I was an Epiphany user ever since it was forked off Galeon, I still had to use Firefox for my work, because I couldn’t really live without Firebug.

It should come as no surprise, then, that one of my first patches to WebKitGTK+ was actually making the awesome Web Inspector work in our port. After the initial support, though, not a lot has been done to further improve it, partly because it was already good enough for many uses, partly because I somehow started doing non-web development again ;).

These last weeks, through my R&D efforts in Collabora, I have been able to push Web Inspector features and integration a bit further. A simple change that boosts the Inspector’s usability quite a bit is having the nodes that are being hovered highlighted. Along with that, the ability to attach the inspector to Epiphany’s window should make it easier to use for poking the DOM.

The Web Inspector has a number of settings that control its behaviour. Since, for instance, enabling javascript debugging may slow down javascript performance, the inspector usually has it disabled by default, and provides a button to enable it. It also provides an option for always enabling that feature, but that does not work right now, because we are not saving/restoring the relevant settings. A solution to that is in the works using the GSettings infrastructure that was recently merged into glib.

Here’s a simple screencast, showing these improvements in action (click the video to check it out in full size):

WebKitGTK+ 1.2.2 and 1.2.3 released!

Some of you may have noticed WebKitGTK+ 1.2.2 and 1.2.3 have been uploaded recently. Here’s their announcement =). A quick summary: if you’re running the 1.2.x series upgrade to 1.2.3.

Here’s some information regarding 1.2.2:

1.2.2 is an update to the 1.2.x stable series; along with a lot of crash, and misc fixes the biggest changes are: 1) the inclusion of a new API from the development branch (webkit_back_forward_list_clear()),
because it’s simple and will help with fixing a problem in Epiphany stable, and 2) lots of drag and drop, and clipboard related work by Martin Robinson.

Despite not being strictly fixes, we believe the stable series has a lot to gain from this work; a couple examples should illustrate this better: the changes included fix both a crash when dragging links from WebKit into other browsers, and the annoying bug that made the cursor get stuck in a grab when dragging, sometimes.
MD5: 40338001324a38b977c163291e8816d3

Here’s some information regarding 1.2.3:

To some such a quick succession in releases may look like a brown paper bag was in order. Not strictly, but indeed 1.2.3 aims to fix some oversights with easy fixes. First of all, WebKit was not buildable with ICU 4.4.1, but thankfully a fix had already been checked in to trunk, so 1.2.3 includes that fix. Secondly, Debian’s Michael Gilbert has done a great job going through all CVEs released about WebKit, and including patches in the Debian package. 1.2.3 includes all of the commits from trunk to fix those, too.
MD5: 0ab5c478a6f5b74a1ae96bf13a456662

You can read some more details, including the list of CVEs that were addressed, in the NEWS file:


Google’s pacman doodle in Epiphany/Midori?

Google has had a very nice idea today, to celebrate Pacman’s aniversary: they made their logo become a playable HTML5 pacman. If you’re wondering why your WebKitGTK+ browser is not being able to play the game here’s why: Google is doing User-Agent sniffing and denying you the fun, sending a static image that you can click to perform a search instead of the game.

If you make Epiphany or Midori identify themselves as Chrome or Firefox, the game will work. I really don’t get this User Agent sniffing bullshit coming from Google. If you go to gconf-editor, and under epiphany->general set the user_agent key to “Mozilla/5.0 (X11; U; Linux; en-gb; rv: Gecko/2008092313 Firefox/3.8″ it works. I’m starting to seriously consider User Agent spoofing for * as a quirk on WebKitGTK+. Lame.

Update: as a protest, I’m making and say Chrome/Chromium are not supported, by doing User Agent sniffing.
Update2: it’s been pointed out to me that the game is not HTML5 – it’s actually smart usage of divs, and flash *urgh* for the audio
Update3: thanks to a friend who works at Google getting in touch with pacman’s designer, it looks like it now works without faking U-A – I’m happy for this, thank you! Despite this good step forward, google is still denying us the nice fade in effect, and still sees us as ‘unsupported’ in Wave and similar products, so I’ll keep my protest for now.

WebKitGTK+ and WebM

So you probably heard about WebM, right? It’s the awesome new media format being pushed by Google and a large number of partners, including Collabora, following the release of the VP8 video codec free of royalties and patents, along with a Free Software implementation.

It turns out that if you are a user or developer of applications that use the GStreamer framework, you can start taking advantage of all that freedom right away! Collabora Multimedia has developed, along with Entropy Wave GStreamer support for the new format, and the code has already landed in the public repositories, and is already being packaged for some distributions.

I just couldn’t wait the few days it will take for the support to be properly landed in Debian unstable, so I went ahead and downloaded all of the current packages from the pkg-gstreamer svn repository, built everything after having the libvpx-dev package installed, and went straight to a rather unknown, small video site called Youtube with my GStreamer-powered WebKitGTK+-based browser, Epiphany!:

Youtube showing a webm video in Epiphany

If you’re running Debian unstable, or any of the other distributions which will be lucky to get the new codecs, and support packages soon, you should be able to get this working out of the box real soon now. Check the tips on WebM’s web site on how to find WebM videos on youtube.

WebKit2 and WebKitGTK+

So you’ve seen people talking about WebKit2, perhaps have seen someone claiming it “drops support for Linux“, and you’ve been wondering what the hell that means for WebKitGTK+. Well, welcome to the preemptive Q&A section with WebKitGTK+ maintainers =D. Let’s first explore some history so we can better understand what exactly is going on.

What exactly is WebKit2?

Currently, when we say “WebKit” we really mean one of the ports that are built on top of WebCore using the WebKit layer. WebCore is the part that does all of the hard Web-related work, WebKit an API layer that exposes WebCore functionality in a coherent way, so that the platform-specific ports can expose a public API layer for their applications to use, which is usually also called “WebKit”. This WebKit layer was designed by Apple to build the Mac, and Windows ports it maintains, and was later released as Free Software so that other ports, such as the GTK+, Qt, EFL ports could be built on top of it, instead of having to do all the heavy lifting from WebCore directly.

Current WebKit model

WebKit2 is nothing more than the second version of that interface, with a whole lot of changes on what you can expect from it, and on how it interacts with WebCore, and the platform-specific API and UI. First of all, the first WebKit was not API stable, and that interface was usually not made public by the various ports – they only exposed their platform-specific APIs. WebKit2 is being designed to provide a stable, cross-platform, C-based, non-blocking public API. This is huge. It will allow cross-platform code to be written without having to consider language, and port differences for basic functionality.

The second big change is the API will be made fully non-blocking. Currently most things you do are asynchronous already, but some of them may be completed in a synchronous ways (like, loading a string into WebKit instead of an URI). This is important for responsiveness, and is also a very important need for what comes next: process splitting.

WebKit2 will bring into WebKit proper the concept of splitting the UI process from the Web process, similar to what Chromium has. It also much more awesome than what Chrome has for a large number of reasons, including, but not limited to:

  1. It’s being contributed directly to the WebKit project, in a cross-platform way that lets ports such as WebKitGTK+ take advantage of it, instead of being shipped directly into Safari, like Google does with Chrome;
  2. The process separation goes bellow the API layer, meaning that all complexity involved in managing the process separation is handled by the library, and hopefully none of it leaks to the application using it; that means that applications like Devhelp and Yelp will be able to take advantage of this without having to make their lives more complicated;
There’s a much better diagram in WebKit2’s wiki page, but here goes a simplified version that demonstrates what I’m talking about:
WebKit2 model

What WebKit2 is not?

WebKit2 is NOT a rewrite of the whole WebKit stack. Webcore will continue mostly unchanged, and all ports currently building on top of it will keep working. It is also not a fork – the code lives in the same tree as the current version of WebKit, which will allow us to progressively move towards using this new, improved layer. WebKit2 is not Apple-only, and it is not dropping Linux support. Initial builds of the code that is being landed will likely show up building on Linux in the near future (specially because us porters are already eager to play with it).

What happens to WebKitGTK+?

In the near future, nothing special. We will continue working towards making it feature-complete, more stable, faster, and rocking on it as always. We will, though, start working out how we can best take advantage of WebKit2 in order to provide an even more awesome library for the G world. What this means is you can expect us to have a library that will provide a nice GTK+ widget, just like we have today, with a GObject-based API, like we have today, but that is built on top of this new WebKit2 infrastructure, taking advantage of the process-splitting, and the bigger focus on not blocking the UI thread. This should give us a platform that is more stable, and faster and more responsive than what we already have today.

The API is bound to change, of course, but the WebKit2 version of WebKitGTK+ will be a separate, parallel-installable library, and we will keep supporting the WebKit1 version while we work on making the new one at least as good as the current one. This is long term we’re talking here. We’ll likely see WebKitGTK+ 1.4, and 1.6 come to life before we are satisfied enough with WebKitGTK+2.

We hope this clears some of the doubts up, and lightens your hearts!

The WebKitGTK+ maintainers.