Stuff Michael Meeks is doing |
Older items: 2021: ( J F M A M J J A S O N D ), 2019, 2019, 2018, 2017, 2016, 2015, 2014, 2013, 2012, 2011, 2010, 2009, 2009, 2008, 2007, 2006, 2005, 2004, 2003, 2002, 2001, 2000, 1999, legacy html
Today we release LibreOffice 5.3.0, the next step in our journey: rich in features indeed - but (understandably) the media like to focus on the things you can see. What about the things you cannot ? the increasingly awesome underpinnings on which we're building the next round of improvements. Again - to see the pretty things people made and (more importantly) who did the heavy lifting checkout the user visible features from many great hackers, translators, UX designers etc. Here I am going to focus on the under-sung heros of making everything else better. There is an official 5.3 wiki page, but I expand on this and dive in more deeply here.
First apologies if I missed your good work; (and this is
a very incomplete view based on a few hours of analysis) -
please do add yourself to the wiki page and potentially mail
me a unified diff to this; thanks ! It is only really possible to
skim 9750 commits and nearly one hundred thousand lines of git log
message across core and online - with:
26,080 files changed, 1,698,565 insertions(+), 361,298 deletions(-)
One thing that perplexes me about LibreOffice 5.3 is how we can have
managed to translate 50,000 lines of German comments to English, but the last
3700 lines continue to defy us. Is it that people don't like finishing things ?
Is it that the interest in comment translation is proportional to the number of
remaining comments ? Seems unlikely - many of the remaning comments are in writer
and calc. Either way - there is some lingering low-hanging glory for the person
who can rid us of the last comments here ! Many thanks to Johannes Berg,
Maarten Bosmans, Tor Lillqvist (Collabora), Julian Mehne and Albert Thuswaldner
for taking on a few comments and to Phillip Szelat for updating the
bin/find-german-comments
tool.
A short list of strings to whet your appetite; please do help finally dispatch this gasping dragon if you're a native German speaker (or competent context reading programmer).
Markus Mohrhard's great work integrating a Breakpad based crash reporter has been extraordinarily useful. You can browse the crash reports here by version, and we've had over thirty crashreporter related fixes in 5.3 targetting particular issues. It is great to see commits like this that can quantify the win (20% of daily crashes in 5.2.0.4). We managed to nail lots of the most frequently reported issues soon after deploying the reporter.
Unfortunately, as Tomaz fixed another very long standing issue that was deadlocking the Windows clipboard code we managed to create a big spike in crashes (when exiting with a live clipboard) - this immediately jumped out of the stats:
Which allowed us to somewhat accelerate the next release containing a fix (hence 5.2.5.1 vs. 5.2.5.2). Markus also nailed a whole clutch of evil badness around processing of events during very late shutdown of LibreOffice that we hope will improve things still further.
Another curious issue is that Windows, even Windows 64bit has a fairly hard (for an ISV) limit of 10,000 gdi objects (should be enough for anybody) handles per process. This was leading to lots of extremely diverse symptoms - with inexplicable failure cases above the window abstraction. Thanks to Markus Morhard for adding the ability to report the number of open GDI handles to crash reports, and Kohei Yoshida (Collabora) for fixing a gratuitous consumer of these handles for 5.3.1.
As always lots of work from many people
check-elf-dynamic-objects
was implemented - to ensure that
we don't get unexpected and unhelpful new library dependencies in#
release and bibisect builds on GNU/Linux thanks to Michael Stahl
(RedHat)As always we continue to grow our unit test count, and number of assertions (and assert calls). The tests are run by the Jenkins CI infrastructure on commits before they are merged. Thanks to so many for adding tests and stopping things from regressing particuarly those who had more than 20 commits to the unit tests: Zdeněk Crhonek, Miklos Vajna (Collabora), Stephan Bergmann (RedHat), Caolán McNamara (RedHat), Justin Luth (SIL), Eike Rathke (RedHat), Noel Grandin (Collabora), Armin Le Grand (CIB), Michael Stahl (RedHat)
Thanks too to Michael Stahl (RedHat) for tackling a number of horrible threading races and mis-designs causing intermittent failure in our unit tests.
One notable improvement in 5.3 has been the work from Zdeněk Crhonek adding 280 or so nicely organized unit test sheets for each Calc function and some of its corner-cases to the tree, and even more encouraging to see these extended when bugs are fixed.
Another area that got a number of nice unit tests was the old-style Parser and new XFastParser implementations thanks to Mohammed Abdul Azeem laying the ground for our incremental adoption of the XFastParser for LibreOffice 5.4 for ODF formats. Great to write tests before the code.
Thanks to Markus Mohrhard and our generous donors we
have a shiny new UI
testing framework in LibreOffice 5.3. We even have tutorials (
part 1 and
part 2)
to help people add new tests. Checkout the code in uitest/
.
It is always good to see a whole new class of testing enabled, and provides a nice place for people comfortable with python to make a really important contribution.
There were really a lot of changes in the LibreOfficeKit APIs and in the online code to interface with them; these two modules are tightly coupled. For details and credits I wrote most of this up for the CODE 2.0 release, around the time of the feature freeze.
A number of important fixes and performance improvements made their way into 5.3 (many of which are back-ported to 5.2 as well). We have used guards - which are created and destroyed in pairs around any OpenGL block - to catch buggy & crashing drivers inside those execution blocks, in order to disable OpenGL. In 5.3 these were adapted to OpenCL as well thanks to Tomaž Vajngerl (Collabora). In addition Michael Meeks (Collabora) added a test spreadsheet that is re-calculated using OpenCL whenever the driver changes - in order to catch badness in driver implementation - somewhat sad that these are necessary; it would be hoped that just asking a (software!) OpenCL driver for its version (to see if we can black-list it) would not hard-crash, but apparently that is too much to ask. Anyhow - now we cope well with lots of sub-optimal situations, and as such removed the legacy 'Test' button for manual CL driver selection.
The VCL / OpenGL backend got a large number of performance improvements from Tomaž Vajngerl (Collabora) including batch rendering of pixels, lines, rectangles and polylines and deferred texture rendering - these help to push more work to the GPU in one go improving performance.
osl::Condition
API in 5.3 in favour of the much more
sensible (Unix like) std::condition_variable - which makes it easier
to write safe code. Thanks to Stephan Bergmann (RedHat) for
fixing a related Thread Pool thread-safety issue. Also to Kohei
Yoshida (Collabora) for unwinding an underlying threading issue
in our ZIP file handling hurting threaded XLSX import.
The QA team do a noble job struggling against great odds; you can see those doing the hugely valuable triage and fixing work in the QA stats for 5.3.0 wiki page. During this release cycle in August we had the pleasure of having Xisco Fauli join TDF - funded by our generous donors - to support and contribute to the QA team; their hard work together has got our UNCONFIRMED bug count way down - accelerating a vital feedback loop between users and developers.
Pootle was upgraded this cycle bringing a wealth of improvements for our translators. From little things like allowing special characters in usernames, to major performance improvements allowing us to extract translations into our builds much more quickly. Other great features such as improved notifications with feedback on changes, off-line translation memory are much appreciated. Thanks to Dwayne Bailey (TranslateHouse) and our sysadmin team.
LibreOffice depends on many 3rd party libraries. To stay up-to-date, they have to be updated from time to time, their patches re-evaluated (ideally up-stream merged them) and they need re-testing. For 5.3, the following were updated:
Lots of fun stuff going on at LibreOffice - quite apart from the visible feature development. Its a great place to find a home and contribute. If you want to get involved there are plenty of great people to meet and work alongside - come and see us at FOSDEM for example this weekend. As you can see individuals ('Assigned') make a huge impact to the diversity of LibreOffice.
In terms of diversity of code commits, we love to see the unaffiliated volunteers contribution by volume, though clearly the volume and balance changes with the season, release cycle, and volunteers vacation / business plans:
Naturally we maintain a list of small, bite-sized tasks which you can use to get involved at our Easy Hacks page, with simple build / setup instructions. It is extremely easy to build LibreOffice, each easy-hack should have code pointers and be a nicely self contained task that is easy to solve. In addition some of them are really nice-to-have features or performance improvements. Please do consider getting stuck in with something.
Another thing that really helps is running pre-release builds and reporting bugs just grab and install a pre-release and you're ready to contribute alongside the rest of the development team.
LibreOffice 5.3 is great; it is made by a set of developers having fun, working together, and building an increasingly attractive and beautiful Free Software Office suite, I hope you enjoy using it. Thanks for reading, don't forget to checkout the user visible feature page and thank you for supporting LibreOffice.
Raw data and commit stats built using our gitdm-config - are available for many of the above graphs.
My content in this blog and associated images / data under
images/
and data/
directories are (usually)
created by me and (unless obviously labelled otherwise) are licensed under
the public domain, and/or if that doesn't float your boat a CC0
license. I encourage linking back (of course) to help people decide for
themselves, in context, in the battle for ideas, and I love fixes /
improvements / corrections by private mail.
In case it's not painfully obvious: the reflections reflected here are my own; mine, all mine ! and don't reflect the views of Collabora, SUSE, Novell, The Document Foundation, Spaghetti Hurlers (International), or anyone else. It's also important to realise that I'm not in on the Swedish Conspiracy. Occasionally people ask for formal photos for conferences or fun.
Michael Meeks (michael.meeks@collabora.com)