This was one of those cases in which the web shows all of its potential to behave weirdly. The HTTP/1.1 RFC says that if an Accept-Encoding header is not present, the server MAY assume the client accepts any encoding, so we were having many sites send us gzip content even though we did not support it. We then started sending a header saying “we support identity, and nothing else!”.

It turns out the web sucks, so many servers were not happy with a full header, and started giving us angry looks (slashdot, for instance, would not render correctly because it started sending encoded CSS files!). We then simplified the header we were sending, which made those servers happy again. Some sites, though, completely ignored our saying we didn’t support anything except identity, and sent us gzipped content anyway. Most of these were misbehaving caches (this was the case for Wikipedia), so would work after you asked for a forced reload, which would ignore the cache, but some servers, such as the Internet Archive’s didn’t really want to talk about encodings - they only wanted to send gzip-encoded content.

So, in the end, our only way out was implementing the damn encoding support, which finally happened during the hackfest. Take that, web!

Xan blogged about day 0, and also a summary of all that was done, so I’ll focus on the stuff he forgot to mention ;D. The hackfest, for me, started on day -1 with me not allowing Xan to go sleep before he had reviewed a couple patches of mine to fix DOM context menu handling. It always bothered me that Epiphany failed to open right-click menus in some pages, or let pages handle the right click. Well, this is fixed now, and Zimbra users can now have their right click menus, and WoW players can remove talent points from their calculators =P.

It turns out that many of the attendees don’t like pages messing with their context menus, though, and they had some good points to back up their positions (like pages making it hard to save images, for instance), so I implemented a way to force openning the custom menu: Ctrl-rightclick.

We wanted to use a GtkInfoBar to display questions regarding the form saving - our initial implementation always saved all credentials, but that didn’t sound good enough. Xan and I thought it would be very complicated to make this work, because there were assumptions in the code regarding which widget contains which, but it turned out to be quite trivial - making EphyEmbed a descendant of GtkVBox instead of GtkScrolledWindow, fixing a small number of assumptions, and that was it.

The passwords are saved in the GNOME Keyring. It’s interesting to point out that GNOME Keyring seems to be unhappy with the number of passwords a browser stores - Xan’s daemon was hanging, crashing, and spawning a large number of threads. My daemon decided to take up some 300MB of RAM at one point. It’s somewhat funny to see how much a browser pushes the limits of our platform. We are hoping this will improve with the new keyring APIs, and the rewrite that is ongoing. It’s nice to see my browser form passwords in seahorse, though, and be able to manage them like any other.

One more thing worth of notice, although this post is already a bit too big: one of the main concerns people had during the Hackfest was on making build time smaller. Touching a single file in WebCore causes a debug build of 10 minutes on my laptop. Evan Martin and Benjamin Otte made a push at removing unnecessary includes from WebCore, and WebKitGTK+ files, which brough the build time down a bit. They end up inspiring Aroben, from Apple, to go even further into this, and remove many includes from files all over WebKit.

Evan was also able to bring linking time down by making it possible to link libwebkit without having to build all the intermediate libraries, which brought build time down to 1 minute, when touching a single file in WebCore. Behdad and I also started looking into breaking WebCore up into lots of shared libraries for Debug builds, since we don’t care too much about speed penalties in those. None of these experiments got committed yet, but I am hoping we will be having a better time hacking on WebKitGTK+ in the near future.

It was awesome meeting everyone, by the way! See you around =).

As part of my work on WebKitGTK+, I always keep an eye on regressions, both from previous WebKitGTK+ releases, and those imposed on embedding applications on their migration away from Gecko, and try to focus some of my efforts into lowering their numbers, whenever I can.

In recent times I have worked on removing a few very user-visible regressions in Epiphany, which I see as the most demanding WebKitGTK+ user in GNOME, such as save page not working, missing
favicon support, failing to
perform server-pushed downloads (such as GMail attachments), and not being able to view source. An example of a regression from a previous version of WebKit also exists: in 1.1.17 we started advertising more than we should as supported by the HTML5 media player, causing download to be almost completely broken.

All of these are working if you are using WebKit and Epiphany from trunk/master, so should be on the next development versions of WebKitGTK+ and Epiphany. Other people have also fixed many other regressions; a few examples: Xan has reimplemented the Epiphany customization of the context menu, Frederic Peters provided a work-around for mailto: links while we don’t have SoupURILoader yet, and Joanmarie Diggs keeps rocking on the accessibility front!

If you find regressions, keep them coming! If you have a patch, even better! =)

Next week WebKitGTK+ team gets together to work furiously on improving WebKitGTK+ in a hackfest sponsored by Collabora, and Igalia, and hosted/organized by Igalia. While there I should also get my hands on one of these. Can’t wait! =)

What I did was create a very simple GTK+ widget that derives from GtkClutterEmbed, and works as a somewhat replacement for GtkNotebook, called, proving how bad I am at naming things, GkOverview. Like I said, it’s not really stable or well-done, it doesn’t even free its resources (in fact, it doesn’t even have implementations for finalize and dispose!), it’s really just an experiment.

What GkOverview does is provide a simple API for you to append widgets into it, and it is able to show you one of those widgets, or an overview of all of them. This is quite simple, and yet very powerful. With the help of my significant other, I have got a layout of the widgets in the overview that I really like.

Of course I used WebKitGTK+ to try it out, what else? And since I had effectivelly, at least in my head, created a fairly convincible replacement for GtkNotebook functionality, I decided a second challenge I could take on myself was to make my preferred browser, Epiphany, use that instead of its EphyNotebook widget. Epiphany being quite well-designed, replacing EphyNotebook was quite a breeze, and here’s the result!

Before I go on, let me repeat it: this is all crappy, experimental, curiosity-induced work. It may be that in the future we can use stuff like this to make, say, improving the back/forward mechanism, history navigation, as other browsers do, replacing tabs with a better UI mechanism, and whatnot.

There’s a somewhat big “video”, for your pleasure (I was finally able to create a nice video, using Byzanz =)).

We have an initial set of APIs to deal with the data that was loaded to render the page, represented by the new DataSource object. This allows you to peek at the source of the HTML that was rendered, for instance. In the near future we will have ways of getting the data and other information for each resource that is downloaded to compose the page. Jan Alonzo has spent quite some time studying, implementing and perfecting this API, with some help from me and Xan.

One initial piece of the larger API to control resource loading is already there, as well. The WebView will use the new resource-request-starting signal to let you know that a request is about to be sent, and allow you to modify headers, or even the URI that is going to be fetched. For some, the greatest news here is that this allows a basic implementation of AD blocking, which is now supported in Epiphany with the WebKit backend, as of version 2.27.92, released today! This is something I have been looking at in my Collabora time for some time now, and working along with Xan and Jan, we finally seem to have figure out the API interactions correctly. There’s more to come regarding resource loading tracking, in the near future.

That would already be a lot, from my point of view, but there’s a whole lot more. A new contributor, Martin Robinson, has arrived proposing new API pieces for an important HTML5 tool: web databases, which brings with it the concept of SecurityOrigin. Jan has exposed API to put the WebView in “view source” mode, Xan has given us undo/redo support for editable content, and Zan Dobersek has done some overall improvement of our HTML5 media player.

That’s a lot of change for a release, which also goes to show we still have quite a gap to fill, but we are being able to move forward and fill them quite quickly. If you want to help out, pick something in your area of interest, and send us patches!

My last blog was about 1.1.7, and we are now at 1.1.11, so let me try to get a quick summary of important changes since 1.1.17:

• Lots of fixes have been made on the scrolling code, which now behaves more like you would expect from a GTK+ widget, and notifications to web applications are also working now
• Thanks to the awesome zecke, you can embed arbitrary GTK+ widgets using the <object> and <embed> tags, using the create-plugin-widget signal
• WebKitNetworkRequest received a lot of the love it was asking for, and is now very useful for tracking the HTTP conversation
• Xan has done awesome work on a11y, and we may have enough done before 2.28 to make WebKitGTK+ be accepted, but that remains to be seen; if you care about that, now is a good time to go to https://bugs.webkit.org/show_bug.cgi?id=25531 and help out!
• XSS auditor, a new feature built to improve WebKit’s security is now enabled
• Copying and pasting has seen many improvements
• WebKitGTK+ no longer has hackish content sniffing, since we now have that implemented into libsoup!

In other news, WebKitGTK+ seems to have gained another application for its ecosystem: Uzbl, which promises to be a good browser for people who like to work with the UNIX way. It doesn’t really appeal to my GNOMEr ‘I like things that just work’ heart, but it looks like a tool that may appeal to people who prefer building their work environments from various different pieces, so that it works exactly like they want to. I am very happy to see WebKitGTK+ is making it possible for people to write such tools!

The patch which finally got committed this week is, how can I put it, VERY different from what I had originally written. You can take a look at the long discussions about it in the bug report I used to track progress. I think I should point out that Marco Barisione and Christian Dywan were crucial in helping me get going with my contribution to WebKit at that time.

What this change gives us is basically the fact that a WebKitNetworkRequest now carries more than just the URI for the request (it actually carries with it a reference to the SoupMessage that will be used later in the request processing, which we are planning to expose in the near future), meaning that when WebKit API gives you a request, and you use it to cause a new load (for, say, opening in a new tab), you still get all the headers that were supposed to go with the request, so you don’t lose things such as, for instance, Referer. So, now, after more than 5 years, the bug that complained that Epiphany did not set Referer (and Galeon before that) for new tabs is finally closed.

By the way, this problem has been fixed for Mozilla’s browser back in 2002, but the embedding API is still buggy up to now. There is still hope, since there’s an attached patch that fixes the issue to be reviewed, and landed. If anyone is reading, it might be a good oportunity to get this fixed in there as well, so that users of applications that use Gecko’s embedding API can also benefit!

The good thing about those packages is I have patched them heavily to allow for easy parallel installation with Epiphany/Gecko, so you don’t need to give up your current browser to experiment and test Epiphany with the WebKitGTK+ backend. The gconf tree used for this package is /apps/epiphany-webkit, separate from the normal /apps/epiphany, for extra safety, but notice that your ~/.gnome2/epiphany will be shared between the Gecko and WebKit versions, even though the files used by each of them are different most of the time.

Go ahead and install the epiphany-webkit package, and have fun. Notice that if you have an already running session of Epiphany/Gecko, running epiphany-webkit will not be enough to launch Epiphany/Webkit, since Epiphany will just request that a new window be opened through D-Bus. The easiest way to test, if you just want a quick peek, is to run epiphany-webkit -p; this will run a ‘private’ instance of Epiphany/WebKit, which doesn’t touch your history, bookmarks, and passwords.

If you are feeling adventurous and want to make Epiphany/WebKit your default Epiphany you can do so using the following command, and selecting epiphany-webkit:

# update-alternatives --config epiphany-browser

Thanks to everyone who contributed to this release. I am sure there are many rough edges to sharpen on the new features, so come test and report bugs, (and provide patches if possible =)) please!