Stuff Michael Meeks is doing
This is my (in)activity log. You might like to visit Collabora Productivity a subsidiary of Collabora focusing on LibreOffice support and services for whom I work. Also if you have the time to read this sort of stuff you could enlighten yourself by going to Unraveling Wittgenstein's net or if you are feeling objectionable perhaps here. Failing that, there are all manner of interesting things to read on the LibreOffice Planet news feed.

Older items: 2023: ( J F M A M J ), 2022: ( J F M A M J J A S O N D ), 2021, 2019, 2018, 2017, 2016, 2015, 2014, 2013, 2012, 2011, 2010, 2009, 2009, 2008, 2007, 2006, 2005, 2004, 2003, 2002, 2001, 2000, 1999, legacy html

LibreOffice under the hood: six months of progress to 5.3

Today we release LibreOffice 5.3.0, the next step in our journey: rich in features indeed - but (understandably) the media like to focus on the things you can see. What about the things you cannot ? the increasingly awesome underpinnings on which we're building the next round of improvements. Again - to see the pretty things people made and (more importantly) who did the heavy lifting checkout the user visible features from many great hackers, translators, UX designers etc. Here I am going to focus on the under-sung heros of making everything else better. There is an official 5.3 wiki page, but I expand on this and dive in more deeply here.

First apologies if I missed your good work; (and this is a very incomplete view based on a few hours of analysis) - please do add yourself to the wiki page and potentially mail me a unified diff to this; thanks ! It is only really possible to skim 9750 commits and nearly one hundred thousand lines of git log message across core and online - with:
26,080 files changed, 1,698,565 insertions(+), 361,298 deletions(-)

The last stubborn German comments

One thing that perplexes me about LibreOffice 5.3 is how we can have managed to translate 50,000 lines of German comments to English, but the last 3700 lines continue to defy us. Is it that people don't like finishing things ? Is it that the interest in comment translation is proportional to the number of remaining comments ? Seems unlikely - many of the remaning comments are in writer and calc. Either way - there is some lingering low-hanging glory for the person who can rid us of the last comments here ! Many thanks to Johannes Berg, Maarten Bosmans, Tor Lillqvist (Collabora), Julian Mehne and Albert Thuswaldner for taking on a few comments and to Phillip Szelat for updating the bin/find-german-comments tool.

A short list of strings to whet your appetite; please do help finally dispatch this gasping dragon if you're a native German speaker (or competent context reading programmer).

Graph showing perplexing exponential drop-off of comments

Crash Reporting

Markus Mohrhard's great work integrating a Breakpad based crash reporter has been extraordinarily useful. You can browse the crash reports here by version, and we've had over thirty crashreporter related fixes in 5.3 targetting particular issues. It is great to see commits like this that can quantify the win (20% of daily crashes in 5.2.0.4). We managed to nail lots of the most frequently reported issues soon after deploying the reporter.

Unfortunately, as Tomaz fixed another very long standing issue that was deadlocking the Windows clipboard code we managed to create a big spike in crashes (when exiting with a live clipboard) - this immediately jumped out of the stats:

Graph showing big jump in crashes

Which allowed us to somewhat accelerate the next release containing a fix (hence 5.2.5.1 vs. 5.2.5.2). Markus also nailed a whole clutch of evil badness around processing of events during very late shutdown of LibreOffice that we hope will improve things still further.

Another curious issue is that Windows, even Windows 64bit has a fairly hard (for an ISV) limit of 10,000 gdi objects (should be enough for anybody) handles per process. This was leading to lots of extremely diverse symptoms - with inexplicable failure cases above the window abstraction. Thanks to Markus Morhard for adding the ability to report the number of open GDI handles to crash reports, and Kohei Yoshida (Collabora) for fixing a gratuitous consumer of these handles for 5.3.1.

Ongoing Code quality work

As always lots of work from many people

Unit testing

As always we continue to grow our unit test count, and number of assertions (and assert calls). The tests are run by the Jenkins CI infrastructure on commits before they are merged. Thanks to so many for adding tests and stopping things from regressing particuarly those who had more than 20 commits to the unit tests: Zdeněk Crhonek, Miklos Vajna (Collabora), Stephan Bergmann (RedHat), Caolán McNamara (RedHat), Justin Luth (SIL), Eike Rathke (RedHat), Noel Grandin (Collabora), Armin Le Grand (CIB), Michael Stahl (RedHat)

Graph showing cppunit tests & assertions
And also to those with more than 10 commits to the unit tests: Tamás Zolnai (Collabora), Laurent Balland-Poirier, Tor Lillqvist (Collabora), Katarina Behrens (CIB), Markus Mohrhard, Mike Kaganski (Collabora), Takeshi Abe, Mark Hung, Giuseppe Castagno. It always takes more time to write a test, there is always a temptation not to - thank goodness they helped.

Thanks too to Michael Stahl (RedHat) for tackling a number of horrible threading races and mis-designs causing intermittent failure in our unit tests.

One notable improvement in 5.3 has been the work from Zdeněk Crhonek adding 280 or so nicely organized unit test sheets for each Calc function and some of its corner-cases to the tree, and even more encouraging to see these extended when bugs are fixed.

Another area that got a number of nice unit tests was the old-style Parser and new XFastParser implementations thanks to Mohammed Abdul Azeem laying the ground for our incremental adoption of the XFastParser for LibreOffice 5.4 for ODF formats. Great to write tests before the code.

UI Testing

Thanks to Markus Mohrhard and our generous donors we have a shiny new UI testing framework in LibreOffice 5.3. We even have tutorials ( part 1 and part 2) to help people add new tests. Checkout the code in uitest/.

It is always good to see a whole new class of testing enabled, and provides a nice place for people comfortable with python to make a really important contribution.

Online

There were really a lot of changes in the LibreOfficeKit APIs and in the online code to interface with them; these two modules are tightly coupled. For details and credits I wrote most of this up for the CODE 2.0 release, around the time of the feature freeze.

Hardware Acceleration

A number of important fixes and performance improvements made their way into 5.3 (many of which are back-ported to 5.2 as well). We have used guards - which are created and destroyed in pairs around any OpenGL block - to catch buggy & crashing drivers inside those execution blocks, in order to disable OpenGL. In 5.3 these were adapted to OpenCL as well thanks to Tomaž Vajngerl (Collabora). In addition Michael Meeks (Collabora) added a test spreadsheet that is re-calculated using OpenCL whenever the driver changes - in order to catch badness in driver implementation - somewhat sad that these are necessary; it would be hoped that just asking a (software!) OpenCL driver for its version (to see if we can black-list it) would not hard-crash, but apparently that is too much to ask. Anyhow - now we cope well with lots of sub-optimal situations, and as such removed the legacy 'Test' button for manual CL driver selection.

The VCL / OpenGL backend got a large number of performance improvements from Tomaž Vajngerl (Collabora) including batch rendering of pixels, lines, rectangles and polylines and deferred texture rendering - these help to push more work to the GPU in one go improving performance.

Miscellaneous

QA / bugzilla

The QA team do a noble job struggling against great odds; you can see those doing the hugely valuable triage and fixing work in the QA stats for 5.3.0 wiki page. During this release cycle in August we had the pleasure of having Xisco Fauli join TDF - funded by our generous donors - to support and contribute to the QA team; their hard work together has got our UNCONFIRMED bug count way down - accelerating a vital feedback loop between users and developers.

Graph showing bug triage work backlog trends

Pootle upgrades

Pootle was upgraded this cycle bringing a wealth of improvements for our translators. From little things like allowing special characters in usernames, to major performance improvements allowing us to extract translations into our builds much more quickly. Other great features such as improved notifications with feedback on changes, off-line translation memory are much appreciated. Thanks to Dwayne Bailey (TranslateHouse) and our sysadmin team.

Updated bundled libraries

LibreOffice depends on many 3rd party libraries. To stay up-to-date, they have to be updated from time to time, their patches re-evaluated (ideally up-stream merged them) and they need re-testing. For 5.3, the following were updated:

Getting involved

Lots of fun stuff going on at LibreOffice - quite apart from the visible feature development. Its a great place to find a home and contribute. If you want to get involved there are plenty of great people to meet and work alongside - come and see us at FOSDEM for example this weekend. As you can see individuals ('Assigned') make a huge impact to the diversity of LibreOffice.

Graph showing individual code committers per month

In terms of diversity of code commits, we love to see the unaffiliated volunteers contribution by volume, though clearly the volume and balance changes with the season, release cycle, and volunteers vacation / business plans:

Graph showing individual code committers per month

Naturally we maintain a list of small, bite-sized tasks which you can use to get involved at our Easy Hacks page, with simple build / setup instructions. It is extremely easy to build LibreOffice, each easy-hack should have code pointers and be a nicely self contained task that is easy to solve. In addition some of them are really nice-to-have features or performance improvements. Please do consider getting stuck in with something.

Another thing that really helps is running pre-release builds and reporting bugs just grab and install a pre-release and you're ready to contribute alongside the rest of the development team.

Conclusion

LibreOffice 5.3 is great; it is made by a set of developers having fun, working together, and building an increasingly attractive and beautiful Free Software Office suite, I hope you enjoy using it. Thanks for reading, don't forget to checkout the user visible feature page and thank you for supporting LibreOffice.

Raw data and commit stats built using our gitdm-config - are available for many of the above graphs.

My content in this blog and associated images / data under images/ and data/ directories are (usually) created by me and (unless obviously labelled otherwise) are licensed under the public domain, and/or if that doesn't float your boat a CC0 license. I encourage linking back (of course) to help people decide for themselves, in context, in the battle for ideas, and I love fixes / improvements / corrections by private mail.

In case it's not painfully obvious: the reflections reflected here are my own; mine, all mine ! and don't reflect the views of Collabora, SUSE, Novell, The Document Foundation, Spaghetti Hurlers (International), or anyone else. It's also important to realise that I'm not in on the Swedish Conspiracy. Occasionally people ask for formal photos for conferences or fun.

Michael Meeks (michael.meeks@collabora.com)

Made with Pyblosxom