This month I’ve been working on the following projects: Certify Boost.Beast Boost.Build BeastLounge Certify Certify did not have any platform-independent means of caching certificate status (i.e. revoked, valid, unknown), so I implemented one. For now it has to be manually filled, but I’ll add a way to import a static blacklist (somewhat similar to the builtin blacklist in Chrome) and query the status of a certificate. Unfortunately there is no way to handle OCSP stapling within the verification callback invoked by OpenSSL which is quite detrimental to usability. Additionally, OpenSSL doesn’t have a way of starting and waiting for an asynchronous operation within callbacks (without blocking). Don’t forget to star the repository: https://github.com/djarek/certify! Boost.Beast When working on making sure Beast is std::launder-correct, I ran into a number of previously undiagnosed bugs in Beast. All of them have been fixed in v254. I was quite confused why these issues weren’t found by CI previously. I’ve been able to track it down to old toolchain versions in Travis. Additionally, the test matrix lacks a few fairly important variants. Considering the fact that Trusty is no longer supported and the switch to Xenial is inevitable, I’ve decided to port over the CI to Azure Pipelines, because it offers better concurrency which allows the Beast CI to afford a larger test matrix. In the process I’ve also decided to use as many default b2 options as possible, to make future changes to the CI easier. There’s still an issue with Valgrind in Xenial to be resolved (doesn’t support the RDRAND instruction). Boost.Build While working on the AzP CI for Beast, I found out that the coverage feature in b2 doesn’t actually set build flags. coverage=all will now properly cause tests to produce gcno and gcda files for consumption by the lcov tool. BeastLounge When experimenting with the BeastLounge application running on Heroku I found out that Heroku’s router has a builtin 55s timeout which dropped websocket connections. I solved the issue by making the websocket ping timeouts configurable.
There are four main areas where I spend my time. Libc++, where I am the “code owner” WG21, where I am the chair of the Library Working Group (LWG) Boost Speaking at conferences Libc++ The next big milestone for libc++ is the LLVM 9.0 release this summer. We’re working towards that, implementing new features and fixing bugs. As the “code owner” for libc++, I also have to review the contributions of other people to libc++, and evaluate and fix bugs that are reported. That’s a never-ending task; there are new contributions ever day. LWG papers implemented this month. P0811 Add std::midpoint and std::lerp for C++20 LWG issues resolved this month 2960 nonesuch is insufficiently useless 2977 unordered_meow::merge() has incorrect Throws: clause 2164 What are the semantics of vector.emplace(vector.begin(), vector.back())? LLVM features implemented this month (certainly incomplete) Fixed the implementations of list::remove_if and list::unique to deal with values or predicates that are elements in the list. Same for forward_list. We did this for remove already, but now we do it for the other operations as well. Added a bunch of new tests for things that we were missing ** list::sort and forward_list::sort are required to be stable. ** You can’t use match_results until you’ve done a regex search. Our tests did this in several places; now we have assertions to prevent that. ` LLVM bugs resolved this month (probably incomplete) Bug 41323 Race condition in steady_clock::now for _LIBCPP_WIN32API Bug 41130 operator/ of std::chrono::duration and custom type. Bug 41577 test/std/utilities/optional/optional.object/optional.object.ctor/move.fail.cpp has wrong assumption. I spent a fair amount of time on Bug 39696 “Workaround “error: ‘(9.223372036854775807e+18 / 1.0e+9)’ is not a constant expression”; which turned out to be a GCC bug on PowerPC machines. Also, there was a series of general cleanups in the libc++ tests to improve portability and readability. I added a bunch of updates for debug-mode, and there were several places where we assumed that string::compare returned -1/0/1 instead of what was specified, which is \<0/0/\>0. Also, I added tests for std::any_cast and array types. The current status of libc++ can be found here: C++20 status C++17 status C++14 status (Complete) Libc++ open bugs WG21 There were no WG21 meetings in April. However, LWG held three teleconferences this month, reviewing papers in advance of the July meeting. We’ll have more teleconferences in May. I am working on more “cleanup” papers similar to P1458 - Mandating the Standard Library: Clause 16 - Language support library, and my P0805 - Comparing Containers needs an update. The goal of the July meeting is to have a “Committee Draft” (CD) of the proposed C++20 standard that can be sent out for review. Also on my TODO list is to attempt to implement some of the proposals that are coming up for a vote in July (flat_map, text formatting, etc). Boost We released Boost 1.70 on the 12th of April. Once again, I was the release manager, which involved a bunch of “process management”; things like assembling the release candidates, packaging up release notes, deciding which problems that came up would be fixed (and which ones would not), and updating the web site (and so on, and so on). Conferences This was a big travel month. I gave two presentations: At the LLVM European Developer’s conference in Brussels, I gave a 30 minute overview of the changes that were coming to the standard library for C++20. At ACCU in Bristol, England, I gave a talk titled “Navigating the development and evolution of a library” In May, I will be speaking at: CppNow, May 5-10 in Aspen, CO I have submitted a talk for CppCon in September, but I will not hear back about this for a month or two.
This month I’ve been working on the following projects: Certify Boost.Beast Boost.Build BeastLounge Certify Certify now properly verifies the hostname of a TLS server according to RFC 2818 or TLS-DANE if available. Additionally, initial support for CRLSets has been merged, although it’s still missing integration into the verification code. I’ve also invested a fair bit of time into researching what other open source libraries do to perform certificate status checking. I’ve looked into BoringSSL, mbedTLS, Botan and even the Go standard library. It’s interesting that no library has a default way of performing the status check of a certificate and it’s left up to the user. The Windows implementation of the certificate store in Certify will now properly use the entire chain passed by the peer, which resolves certificate failures in less common cases. Don’t forget to star the repository: https://github.com/djarek/certify! Boost.Beast Most of the work this month involved making Beast compile faster and use less memory by expanding the code that can use split compilation and reducing redundant dependencies in a few places. Boost.Build I’ve worked on implementing 2 improvements that make it less painful to work with b2: support for finding OpenSSL support for sanitizers in gcc and clang Both are currently still in review. BeastLounge The project lacked functioning CI so I implemented one. Since the project was previously only compiled on MSVC, this proved to be quite challenging, because MSVC accepts code that is not valid C++11. I’ve also created a deplyoment docker image, which allows running the application in popular cloud environments, like Heroku. A development version of the app is available at https://beast-lounge.herokuapp.com/.
There are four main areas where I spend my time. Libc++, where I am the “code owner” WG21, where I am the chair of the Library Working Group (LWG) Boost Speaking at conferences This month, I spent far more time reviewing other people’s code and preparing talks for conferences than the previous few months. The Boost release process consumed a fair chunk of time as well. Libc++ The big news is: we released LLVM 8 this month! (March 20th). You can get the sources and pre-built binaries from the LLVM download page, or wait for your system vendor to provide you with an update. As the “code owner” for libc++, I also have to review the contributions of other people to libc++, and evaluate and fix bugs that are reported. That’s a never-ending task; there are new contributions ever day. LWG papers implemented this month. P0811 std::midpoint for integral and pointer types. This turned out to be quite involved, and spawned a clang bug report. On the plus side, now I have a topic for a talk for CppCon this fall. Still to do, std::midpoint for floating point types. This is done, but it needs better tests. LWG issues implemented this month I didn’t actually commit any LWG issue fixes this month. I worked with others on several bug fixes that landed, but not under my name. LLVM features implemented this month (certainly incomplete) Add noexcept to operator for array and deque Mark vector::operator and front/back as noexcept Mark front() and back() as noexcept for array/deque/string/string_view Make to_chars/from_chars work back to C++11. This lets us use them in to_string. LLVM bugs resolved this month (probably incomplete) Bug 35967 <regex> syntax_option_type is not a proper bitmask No bug # Fix a minor bug with std::next and prev not handling negative numbers. No bug # Cleanup of requirements for optional - we no longer allow optional<const in_place_t> Bug 41130 operator/ of std::chrono::duration and custom type. Also, there was a series of general cleanups in the libc++ tests to improve portability and readability. Eric and I (mostly Eric) revamped the debug-mode support, and there will be more activity there in the future. Also, we’re moving towards using more of the ASSERT_XXXX macros for readability, and I revamped about 30 of the tests to use them. Only several thousand to go! The current status of libc++ can be found here: C++20 status C++17 status C++14 status (Complete) Libc++ open bugs WG21 The “winter” WG21 meeting was held in Kona, HI on February 18-24. This was the last meeting for new features for C++20, and as such, it was both contentious and very busy. Between now and the next meeting (July), LWG will be working on reviewing papers and issues to be adopted in July. We have had three teleconferences since Kona, and a fourth is scheduled for mid-April. I am working on more “cleanup” papers similar to P1458 - Mandating the Standard Library: Clause 16 - Language support library, and my P0805 - Comparing Containers needs an update. The goal of the July meeting is to have a “Committee Draft” (CD) of the proposed C++20 standard that can be sent out for review. Boost It’s time for another Boost release (1.70), and I am acting as the release manager again. The release calendar is available (as always) on the Boost website. The cut-off for contributions for the release is 3-April, with a release candidate to follow close behind, and the actual release to happen on the 10th. Once the release is over, I’ll be putting some serious time into Boost.Algorithm; there are a bunch of C++17/20 algorithms that can be added to the library (among other things). Conferences I had submitted talk proposals to three conferences, and all three were accepted. I will be speaking at: LLVM European Developer’s Conference, April 8-9 in Brussels ACCU, April 10-13 in Bristol CppNow, May 5-10 in Aspen, CO
Certify - X509 certificate validation I always knew that validating a certificate chain presented by a peer is not an easy procedure, but my recent work in Certify to port over the procedure from Chromium has only proven that I underestimated the complexity of it. Certificate revocation seems to be a particularly hard issue, with 2 main categories of solutions - offline and online validation. Online validation - OCSP OCSP is a protocol designed to allow checking the revocation status of a certificate by sending a request over a subset of HTTP/1.1. At first glance, it seems it solves the status checking problem on its own. However, OCSP has problems, inherent to online checking. First of all, the validation server might not be currently available - so a lack of response is most definitely not a state in which a chain can be trusted. Secondly, the check may be slow, after all, it requires connecting to a separate service. Additionally, the native Windows API for certificate verification does the status check synchronously, which means potentially blocking a user’s thread that typically services asynchronous operations. There is a feature that alleviates most of these issues, at least from the point of view of a TLS client, OCSP stapling. Sadly, it’s not very widespread and actually few large services support it, due to the fact that it increases bandwidth requirements. Certify will, at some point support both OCSP status checks on the client side and support for OCSP stapling. The problem here is that OCSP requires a fairly functional HTTP client and ASN.1 parsing. A lot of this functionality is already present in OpenSSL, however, integrating it with ASIO and Beast may be tricky. Offline validation - CRLs and Google CRLSets The traditional method of checking the status of a certificate involves looking up revocation lists installed in the OS’s store, or downloaded by the application from the CA. Unfortunately CRLs have issues - an example would be an incident from a few years ago when CloudFlare performed a mass revocation which blew up the size of the CRLs by a few orders of magnitude, resulting in a requirement to download multiple megabytes of data, turning CAs into a major performance bottleneck. Google came up with a different mechanism, called CRLSets, which involves a periodic download of a revocation list which is created by Google’s crawler querying certificate status over OCSP. This verification method is fairly attractive for applications that run on systems that already have Google products, since this database is shared, which is why I’ve chosen to provide an opt-in implementation in Certify. For now, updating the database will be out of scope, because that requires a few utilties that are missing from Boost at this time (XML, JSON and an HTTP Client). Don’t forget to star the repository: https://github.com/djarek/certify!
Monthly update (or, what Marshall did in January and February) There are four main areas where I spend my time. Libc++, where I am the “code owner” WG21, where I am the chair of the Library Working Group (LWG) Boost Speaking at conferences Libc++ The LLVM “branch for release” occurred in January, and there was a bit of a rush to get things into the LLVM 8 release. Now that is over, and we’re just watching the test results, seeing if anyone finds any problems with the release. I don’t anticipate any, but you never know. As the “code owner” for libc++, I also have to review the contributions of other people to libc++, and evaluate and fix bugs that are reported. That’s a never-ending task; there are new contributions ever day. After the branch, I started working on new features for the LLVM 9 release (for this summer). More calendaring stuff, new C++20 features, and some C++17 features that haven’t been done yet. LWG papers implemented in Jan/Feb P0355: Extending to Calendars and Time Zones. You may remember this from last month's update; this is a huge paper, and I am landing it in stages. P1024: tuple-like interface to span P1227: Signed ssize() functions P1357: Traits for [Un]bounded Arrays LWG issues implemented in Jan/Feb (certainly incomplete) LWG3101: span’s Container constructors need another constraint LWG3144: span does not have a const_pointer typedef Enabled a memcpy optimization for const vectors that was surprisingly missing LLVM bugs resolved in Jan/Feb (probably incomplete) Bug 28412 std::vector incorrectly requires CopyConstructible, Destructible and other concepts Bug 39183 tuple comparison operators return true for tuples of different sizes Bug 24411 libFuzzer outputs that crash libc++’s regex engine Bug 34330 error: use of undeclared identifier ‘isascii’ while compiling strstream.cpp Bug 38606 no_sanitize(“unsigned-integer-overflow”) annotation for decremented size_type in __hash_table Bug 40533 std::minmax_element is 3 times slower than hand written loop Bug 18584 SD-6 Feature Test Recommendations Bug 40566 Libc++ is not Implicit Integer Truncation Sanitizer clean Bug 21715 128-bit integers printing not supported in stl implementation Bug 38844 __cpp_lib_make_unique not defined in <memory> Bug 40495 is_invokable_v<void> does not compile Bug 40270 std::basic_stringstream is not working with std::byte Bug 39871 std::tuple_size should be a struct Bug 38052 std::fstream still good after closing and updating content Also, there was a series of general cleanups in the libc++ tests to improve portability. The current status of libc++ can be found here: C++20 status C++17 status C++14 status (Complete) Libc++ open bugs WG21 The “winter” WG21 meeting was held in Kona, HI on February 18-24. This was the last meeting for new features for C++20, and as such, it was both contentious and very busy. The Modules TS and the Coroutines TS were both adopted for C++20, along with a slew of language features. Here are some trip reports: Herb Sutter Bryce Adelstein Lelbach Guy Davidson My part in this was (as always) to chair the Library Working Group (LWG), the group responsible for the description of the library features in the standard (~1000 pages). We adopted several new features for C++20: P0339R6 polymorphic_allocator<> as a vocabulary type P0340R3 Making std::underlying_type SFINAE-friendly P0738R2 I Stream, You Stream, We All Stream for istream_iterator P0811R3 Well-behaved interpolation for numbers and pointers P0920R2 Precalculated hash values in lookup P1001R2 Target Vectorization Policies from Parallelism V2 TS to C++20 P1024R3 Usability Enhancements for std::span P1164R1 Make create_directory() Intuitive P1227R2 Signed ssize() functions, unsigned size() functions P1252R2 Ranges Design Cleanup P1357R1 Traits for [Un]bounded Arrays I wrote five substantive papers for the Kona meeting, all were adopted. Five of them were very similar, all about improving the wording in the standard, rather than proposing new features. P1458 Mandating the Standard Library: Clause 16 - Language support library P1459 Mandating the Standard Library: Clause 18 - Diagnostics library P1462 Mandating the Standard Library: Clause 20 - Strings library P1463 Mandating the Standard Library: Clause 21 - Containers library P1464 Mandating the Standard Library: Clause 22 - Iterators library I was also the nominal author of P1457 “C++ Standard Library Issues to be moved in Kona”, but that was just a list of issues whose resolutions we adopted. Between now and the next meeting (July), LWG will be working on reviewing papers and issues to be adopted in July. I’m planning regular teleconferences (in fact, we had the first one on 1-March). The goal of the July meeting is to have a “Committee Draft” (CD) of the proposed C++20 standard that can be sent out for review. Boost It’s been a quiet couple of months for Boost, since we’re between releases, and I have been busy with libc++ and WG21 activities. There have been a few bugs to chase down, and the dealing with change requests for the libraries whose maintainers have “moved on” takes some time. However, it’s time for another Boost release (1.70), and I will be acting as the release manager again. The release calendar is available (as always) on the Boost website. The beta release is schedule for March 13th, and the final release for 10-April. Conferences I had submitted talk proposals to three conferences, and all three were accepted. Hence, I will be speaking at: LLVM European Developer’s Conference, April 8-9 in Brussels ACCU, April 10-13 in Bristol CppNow, May 5-10 in Aspen, CO
The Alliance engages Adler & Colvin to complete IRS Form 1023, Application for Recognition of Exemption Under Section 501(c)(3) of the Internal Revenue Code. Completing this form can be a daunting task because of the legal and tax technicalities you’ll need to understand. Adler & Colvin is a group of seasoned attorneys based in San Francisco, deeply committed to serving the legal needs of the nonprofit sector. The firm brings an unrivaled depth of expertise and passion to its representation of tax-exempt organizations and individual philanthropists.
Monthly update (or, what Marshall did in December) There are three main areas where I spend my time. Boost Libc++ WG21, where I am the chair of the Library Working Group (LWG) Boost: December was a big month for boost, and much of the first part of the month was taken up with the release process. I was the release manager for the 1.69.0 release, which went live on 12-December. The final release process was fairly straighforward, with only one release candidate being made/tested - as opposed to the beta, which took three. In any case, we had a successful release, and the I (and other boost developers) are now happily working on features/bug fixes for the 1.70 release - which will occur in March. Libc++: After the WG21 meeting in November, there was a bunch of new functionality to be added to libc++. The list of new features (and their status) can be seen on the libc++ website. My major contributions of new features in December were Consistent Container Erasure, char8_t: A type for UTF-8 characters and strings, and Should Span be Regular?, and a big chunk of [Extending to Calendars and Time Zones](https://wg21.link/P0355R7). This is all pointing towards the January 16th “branch for release”, and for the scheduled March release of LLVM 8.0. As the “code owner” for libc++, I also have to review the contributions of other people to libc++, and evaluate and fix bugs that are reported. That’s a never ending task; there are new contributions ever day. WG21 Being between meetings (November -> February) there was not any special WG21 work to be done in December. There’s an ongoing stream of bug reports, discussion, paper reviews that get done between meetings, and there is a series of papers that I need to finish for the pre-Meeting mailing deadline on 21-January. I have 1 1/2 done, and need to do 3-4 more.
WG21 San Diego Meeting Last week was the fall 2018 WG21 standard committee meeting. It was held in San Diego, which is my hometown. The fact that I helped organize it (while I was working at Qualcomm) had absolutely no affect on the location, I assure you. ;-) This was the largest WG21 meeting ever, with 180 attendees. The last meeting (in Rapperswil, Switzerland) had about 150 attendees, and that was the largest one until now. There were more than 270 papers in the pre-meeting mailing; meaning that people were spending weeks reading papers to prepare for the meeting. Herb Sutter (the convener) has been telling everyone that new papers received after the San Diego meeting were out of scope for C++20, and apparently people took him at his word. This was my first meeting representing the C++ Alliance (though hardly my first overall). The Alliance was well represented, with Rene, Glen, Vinnie, Jon and myself attending. For information about how WG21 is structured, please see isocpp.org. I spent all of my time in LWG, since that’s the group that I chair, and the one that has the most influence over libc++, the library that I work on. The big news from a library POV was that we voted to merge an updated paper based on the Ranges TS into the draft standard; which means that (barring catastrophe) that it will be part of C++20. This was a huge paper, weighing in at 220+ pages. We spent several days in LWG reviewing this (and a bunch of time at previous meetings as well). We also moved a bunch (around 25) of smaller papers; too many to list here. Detailed trip reports can be found around the web: Herb Sutter Reddit The next WG21 meeting is in Kona, HI February 18-23rd.
Initial work on Certify complete It’s been mentioned in my initial blog post that I’d be working on a TLS certificate store abstraction library, with the intent of submitting it for formal review for Boost, at some point in the (hopefully near) future. The initial setup phase (things that every Software Engineer hates) is more or less complete. CI setup was a bit tricky - getting OpenSSL to run with the boost build system on both Windows and Linux (and in the future MacOS) has provided a lot of “fun” thanks to the inherent weirdness of OpenSSL. The test harness currently consists of two test runners that loads certificates from a database (big name for a folder structure stored in git) that has the certificate chains divided into two groups. Chains that will fail due to various reasons (e.g. self-signed certificates, wrong domain name) and ones that will pass (when using a valid certificate store). I’m still working on checking whether the failure was for the expected reason. All the verification is done offline (i.e. no communication with external servers is performed, only chain verification). At this point it looks like I should consider, whether the current design of the verification code is a good approach. Using the verification callback from OpenSSL and asio::ssl is quite an easy way of integrating the platform-specific certificate store API it causes issues with error propagation (transporting a platform-specific error through OpenSSL) and may be fairly slow, because it requires certificates to be reencdoded into the DER format so that they can be fed into the platform-specific API. An alternative to this approach would be load the entire root certificate store, along with CRLs and OCSP configuration into an OpenSSL context. This is potentially a little bit harder to get right but may offer better performance (no reencoding required when veryfing certificate chains) and eliminates the issues related to error handling. Further investigation, as to which approach is better, is required. Don’t forget to star the repository: https://github.com/djarek/certify!
The Alliance is a Gold sponsor for CppCon 2018. This conference is the annual, week-long face-to-face gathering for the entire C++ community. The conference is organized by the C++ community for the community. Attendees enjoy inspirational talks and a friendly atmosphere designed to help individuals learn from each other, meet interesting people, and generally have a stimulating experience.
Damian Jarek joins the Alliance as Staff Engineer. Previously he worked on a number of embedded networking projects for a few major clients. As a Staff Engineer he’ll be working on an open-source companion library for Boost.Beast and Boost.Asio, which will abstract away the platform-specific details of acessing system proxy settings and performing TLS verification of a peer certificate chain using the operating system’s key store.
Marshall Clow joins the Alliance as a Staff Engineer. Previously, he worked at Qualcomm for many years. Most of his time is spent working on libc++, the C++ standard library implementation for LLVM. He is also a member of the C++ standards committee, currently serving as the chair of LWG, the library working group. Marshall has been contributing to the Boost libraries since 2001, and is the author of the Boost.Algorithm library. Furthermore he maintains several other boost libraries, and moderates some of the boost mailing lists. Finally, Marshall has graciously taken on the role of release manager for several Boost versions.
The Alliance engages The Law Firm for Non-Profits for legal representation and services. They are passionate about supporting, advocating for and partnering with non-profits and the people behind them. For more than three decades, those looking for assistance with non-profit law throughout the United States and around the world have relied on the attorneys of The Law Firm for Non-Profits for superior legal and business guidance.
The Alliance is member of the International Committee for Information Technology Standards. INCITS is the central U.S. forum dedicated to creating technology standards for the next generation of innovation. INCITS members combine their expertise to create the building blocks for globally transformative technologies. From cloud computing to communications, from transportation to health care technologies, INCITS is the place where innovation begins. Membership in INCITS allows voting in official WG21 meetings.
The Alliance engages Foundation Group, a non-profit formation and compliance services company. Foundation Group delivers a comprehensive 501(c)(3) registration service with a 100% IRS approval rate.