Stuff Michael Meeks is doing |
Older items: 2021: ( J F M A M J J A S O N D ), 2019, 2019, 2018, 2017, 2016, 2015, 2014, 2013, 2012, 2011, 2010, 2009, 2009, 2008, 2007, 2006, 2005, 2004, 2003, 2002, 2001, 2000, 1999, legacy html
Today we release CODE 2.0 which includes Collaborative Editing. We've done a huge amount of work since CODE 1.0 - and many of these improvements have been back-ported for our customers & community, but it is perhaps well to credit the authors in one place and survey progress over the last six months.
There have been 1200 commits to 'online' and around 500 to the LibreOffice core since we branched for 1.0. While we celebrate and credit the fantastic work of the LibreOffice community without which there could be no LibreOffice Online - it is clear that Collabora is the architect and driving force behind putting LibreOffice in the Cloud - we have 14 online commits from non-collaborans. On the flip side we're eager to change that - that there is lots of low hanging fruit here, for web developers, and we love contributions. Anyhow - to the details:
benchmarking
:
$ ./loolstress --bench --iter=100 /tmp/looltrace.gz Iterations: 100 Latency best: 67 microsecs, 95th percentile: 4740 microsecs. Tile best: 52 microsecs, rendering 95th percentile: 1446 microsecs. Cached best: 34 microsecs, tile 95th percentile: 67 microsecs. Rendering power: 55.4497 MPixels/sec. Cache power: 826.327 MPixels/sec.
While each LibreOffice Kit process is fundamentally single-threaded and isolated into its own jail, we serve many hundreds of clients per WSD daemon involving a certain amount of concurrency excitement. In 2.0 Ashod Nakashian removed a per-connected-client thread count from the slave kit processes significantly simplifying things, he also did a lot of lock auditing, session cleanup and admin console state synchronisation improvements.
Meanwhile Pranav Kant ensured that users' WOPI credentials are tracked such that the right user is tagged as having hit the explicit 'save' (of course we also auto-save regularly).
Other cleanups and fixes here include switching to un-named pipes thanks to Tor Lillqvist, catching and fixing races around tile rendering (Ash) and avoiding a race when loading the same document multiple times concurrently thanks to Kendy.
Performance and interactivity is significantly improved in 2.0 with a large number of improvements. Tile requests are now prioritized such that tiles near active cursors are rendered and returned before others thanks to Ash and Kendy who also re-wrote the tile request, combining, de-combining, and re-aggregating in the core to ensure that we render the largest area possible to avoid the costs of repeated in-document image scaling. Ash also added a tile versioning scheme to ensure that we always get the latest, correct tile under heavy load and avoid duplicate painting. Kendy and Tor added a prioritisation scheme for presentation slide thumbnailing to improve interactivity.
Other improvements are to use LD_BIND_NOW
to have
all the linking performed by the forkit process once and not again,
Ash also improved the forkit child spawning and lifecycle
management.
Pranav profiled and optimized the WOPI GetFileInfo usage to reduce it to a single call from three, some implementations have unexpectedly expensive implementations of this. Marco Cecchetti - made SSL support run-time configurable to make it easier to configure SSL off-load / acceleration in large setups and Ash added compiled-in limits to avoid misleading enterprise users that CODE is suitable for deployment at scale without support. CODE remains firmly targetted at home users, Henry - added a warning for too many users/connections.
The most obvious improvement in 2.0 is collaborative editing,
which has a lot of UX elements to enable users to see other users' cursors,
selections, with name popups when they are active, or mouse-overed, and more
thanks to Henry Castro. There are also a number of nice new elements
such as the user-list pop-up in the bottom toolbar containing the
colors of users' cursors thanks to Pranav you can also click to
jump to that user's position in the document which is neat.
Pranav also dunged out lots of the shared editing pieces from 1.0.
Meanwhile Miklos Vajna did lots of heavy-lifting around Undo & Redo
which gets significantly more complex with multiple users. Our eventual solution
is of building an infrastructure for simple de-confliction, while also
allowing a Repair Document power mode to unwind the rare
<ctrl>-a, <space>
issues. This seems to work
really well under our normal collaborative editing conditions.
Various general cleanups have been implemented such as re-locating, and anti-aliasing the progess-bar and spinner thanks to Pranav, who also added git hashes to help->about dialog - with Andras adding cgit web links to make QA versioning trivial. Andras also did lots of l10n enablement work to the Admin Console, slide layouts, status bar, colour pickers, menus and context menus, while Pranav added context sensitive enablement to menubar items. Meanwhile Kendy switched us to use larger icons, and improved the toolbar look, while Andras made it easy to escape menus with that key. Meanwhile Henry added status bar items to all of the component - with much requested features like calc selection quick summing, and powerful word-count in writer.
One beautiful win for calc users is Henry's tweak to highlight row and column headers to show cursor and selection positions. He also took the time to implement lots of other nice wins around row & column headers from drag & drop re-sizing, double-click to optimial size, as well as adding context menus to show & hide rows & columns.
Meanwhile Marco fixed a pernicious range issue with page-up/down to make editing smoother while Pranav fixed some formula-bar ergonomics and Henry tackled the interactive auto-sum functionality. Henry also extended the toolbars and menus to include sorting, simple number formats, merge & center, wrap text and more. Meanwhile, for this release, Andras disabled zooming in calc - interestingly our competition don't do this either - and the non-linear co-ordinate space caused by row height rounding was causing real issues.
Meanwhile in writer Andras significantly extended the format menu allowing simple page size, line-spacing, alignment and other simple formatting. Pranav improved the context menus exposed on in-document comments, to allow replying and deleting, while Andras added foot & end-notes, page & column break insertion, as well as Table of Contents and image wrap & anchor context menu items for one of our customers.
Lots of other miscellaneous but useful wins went in such as Henry fixing <ctrl>-f to focus the 'find' toolbar and Pranav mapping <ctrl>-s to .uno:Save. A volunteer Feyza Yavuz added the insert comment toolbar button. Faruk Uzun implemented a draggable insert table toolbar button grid, and Pranav customized a nice read-only mode for users without edit permission.
Collaboration was the largest, and the most substatial part of the 2.0 work. It is hard to list everything that was done there though - it is infrastructure work, many parts of that are implementation details, it was done incrementally, and also with some trial & error on the way; really more of a research task than just plain development.
Conceptually, we wanted to reuse as much of existing code as possible - as we always do - which was luckily possible for the collaboration too. LibreOffice has a feature that allows users to open multiple views of the same document: Try Window -> New Window in your desktop LibreOffice. You will see that this already gives you multiple cursors, and multiple selections. We "just" mapped the multiple document views to the multiple users. That means, on the server, there is only one document open, but with many views - one view for one user.
Of course, the hard part was to make it all fit together: Each user (view) needs to see the cursors and selections of the other users (views) which is something that LibreOffice did not provide previously. There were many bugs when update in one view did not trigger update of the other view (which is probably because the feature itself has not been much used until now). Lots of code had to be updated and fixed. And also, lots of decision had to be made about the updates - what change has to be broadcasted to all the views ? vs. which changes apply only to the current view, and what to other views than the current one. Not to mention all the corner cases when one of the views is closed in the middle of the broadcast to all the views, etc. You get the picture.
To mention the most important parts of this work:
Of course, on top of the new features, various people have contributed a tremendous amount of bugfixes. To list the most important ones:
While 99% of the code comes from Collabora, many thanks again to everyone who contributed including the LibreOffice translation community. As announced at the LibreOffice conference - we're working hard to get a first up-stream release of LibreOffice Online included into LibreOffice 5.3 - and to make daily image builds available for testing as we move towards that.
We've done a huge amount of work here; this is just a short summary of it, and it is an amazing privilege to be able to work with such a talented set of developers at Collabora and across the wider community around LibreOffice. Why not head to and get involved with LibreOffice, or deploy the stable 1.0 version of Collabora Online in your organization - while we harden Collabora Online 2.0 for your delectation.
My content in this blog and associated images / data under
images/
and data/
directories are (usually)
created by me and (unless obviously labelled otherwise) are licensed under
the public domain, and/or if that doesn't float your boat a CC0
license. I encourage linking back (of course) to help people decide for
themselves, in context, in the battle for ideas, and I love fixes /
improvements / corrections by private mail.
In case it's not painfully obvious: the reflections reflected here are my own; mine, all mine ! and don't reflect the views of Collabora, SUSE, Novell, The Document Foundation, Spaghetti Hurlers (International), or anyone else. It's also important to realise that I'm not in on the Swedish Conspiracy. Occasionally people ask for formal photos for conferences or fun.
Michael Meeks (michael.meeks@collabora.com)