Certify - X509 certificate validation I always knew that validating a certificate chain presented by a peer is not an easy procedure, but my recent work in Certify to port over the procedure from Chromium has only proven that I underestimated the complexity of it. Certificate revocation seems to be a particularly hard issue, with 2 main categories of solutions - offline and online validation. Online validation - OCSP OCSP is a protocol designed to allow checking the revocation status of a certificate by sending a request over a subset of HTTP/1.1. At first glance, it seems it solves the status checking problem on its own. However, OCSP has problems, inherent to online checking. First of all, the validation server might not be currently available - so a lack of response is most definitely not a state in which a chain can be trusted. Secondly, the check may be slow, after all, it requires connecting to a separate service. Additionally, the native Windows API for certificate verification does the status check synchronously, which means potentially blocking a user’s thread that typically services asynchronous operations. There is a feature that alleviates most of these issues, at least from the point of view of a TLS client, OCSP stapling. Sadly, it’s not very widespread and actually few large services support it, due to the fact that it increases bandwidth requirements. Certify will, at some point support both OCSP status checks on the client side and support for OCSP stapling. The problem here is that OCSP requires a fairly functional HTTP client and ASN.1 parsing. A lot of this functionality is already present in OpenSSL, however, integrating it with ASIO and Beast may be tricky. Offline validation - CRLs and Google CRLSets The traditional method of checking the status of a certificate involves looking up revocation lists installed in the OS’s store, or downloaded by the application from the CA. Unfortunately CRLs have issues - an example would be an incident from a few years ago when CloudFlare performed a mass revocation which blew up the size of the CRLs by a few orders of magnitude, resulting in a requirement to download multiple megabytes of data, turning CAs into a major performance bottleneck. Google came up with a different mechanism, called CRLSets, which involves a periodic download of a revocation list which is created by Google’s crawler querying certificate status over OCSP. This verification method is fairly attractive for applications that run on systems that already have Google products, since this database is shared, which is why I’ve chosen to provide an opt-in implementation in Certify. For now, updating the database will be out of scope, because that requires a few utilties that are missing from Boost at this time (XML, JSON and an HTTP Client). Don’t forget to star the repository: https://github.com/djarek/certify!
Monthly update (or, what Marshall did in January and February) There are four main areas where I spend my time. Libc++, where I am the “code owner” WG21, where I am the chair of the Library Working Group (LWG) Boost Speaking at conferences Libc++ The LLVM “branch for release” occurred in January, and there was a bit of a rush to get things into the LLVM 8 release. Now that is over, and we’re just watching the test results, seeing if anyone finds any problems with the release. I don’t anticipate any, but you never know. As the “code owner” for libc++, I also have to review the contributions of other people to libc++, and evaluate and fix bugs that are reported. That’s a never-ending task; there are new contributions ever day. After the branch, I started working on new features for the LLVM 9 release (for this summer). More calendaring stuff, new C++20 features, and some C++17 features that haven’t been done yet. LWG papers implemented in Jan/Feb P0355: Extending to Calendars and Time Zones. You may remember this from last month's update; this is a huge paper, and I am landing it in stages. P1024: tuple-like interface to span P1227: Signed ssize() functions P1357: Traits for [Un]bounded Arrays LWG issues implemented in Jan/Feb (certainly incomplete) LWG3101: span’s Container constructors need another constraint LWG3144: span does not have a const_pointer typedef Enabled a memcpy optimization for const vectors that was surprisingly missing LLVM bugs resolved in Jan/Feb (probably incomplete) Bug 28412 std::vector incorrectly requires CopyConstructible, Destructible and other concepts Bug 39183 tuple comparison operators return true for tuples of different sizes Bug 24411 libFuzzer outputs that crash libc++’s regex engine Bug 34330 error: use of undeclared identifier ‘isascii’ while compiling strstream.cpp Bug 38606 no_sanitize(“unsigned-integer-overflow”) annotation for decremented size_type in __hash_table Bug 40533 std::minmax_element is 3 times slower than hand written loop Bug 18584 SD-6 Feature Test Recommendations Bug 40566 Libc++ is not Implicit Integer Truncation Sanitizer clean Bug 21715 128-bit integers printing not supported in stl implementation Bug 38844 __cpp_lib_make_unique not defined in <memory> Bug 40495 is_invokable_v<void> does not compile Bug 40270 std::basic_stringstream is not working with std::byte Bug 39871 std::tuple_size should be a struct Bug 38052 std::fstream still good after closing and updating content Also, there was a series of general cleanups in the libc++ tests to improve portability. The current status of libc++ can be found here: C++20 status C++17 status C++14 status (Complete) Libc++ open bugs WG21 The “winter” WG21 meeting was held in Kona, HI on February 18-24. This was the last meeting for new features for C++20, and as such, it was both contentious and very busy. The Modules TS and the Coroutines TS were both adopted for C++20, along with a slew of language features. Here are some trip reports: Herb Sutter Bryce Adelstein Lelbach Guy Davidson My part in this was (as always) to chair the Library Working Group (LWG), the group responsible for the description of the library features in the standard (~1000 pages). We adopted several new features for C++20: P0339R6 polymorphic_allocator<> as a vocabulary type P0340R3 Making std::underlying_type SFINAE-friendly P0738R2 I Stream, You Stream, We All Stream for istream_iterator P0811R3 Well-behaved interpolation for numbers and pointers P0920R2 Precalculated hash values in lookup P1001R2 Target Vectorization Policies from Parallelism V2 TS to C++20 P1024R3 Usability Enhancements for std::span P1164R1 Make create_directory() Intuitive P1227R2 Signed ssize() functions, unsigned size() functions P1252R2 Ranges Design Cleanup P1357R1 Traits for [Un]bounded Arrays I wrote five substantive papers for the Kona meeting, all were adopted. Five of them were very similar, all about improving the wording in the standard, rather than proposing new features. P1458 Mandating the Standard Library: Clause 16 - Language support library P1459 Mandating the Standard Library: Clause 18 - Diagnostics library P1462 Mandating the Standard Library: Clause 20 - Strings library P1463 Mandating the Standard Library: Clause 21 - Containers library P1464 Mandating the Standard Library: Clause 22 - Iterators library I was also the nominal author of P1457 “C++ Standard Library Issues to be moved in Kona”, but that was just a list of issues whose resolutions we adopted. Between now and the next meeting (July), LWG will be working on reviewing papers and issues to be adopted in July. I’m planning regular teleconferences (in fact, we had the first one on 1-March). The goal of the July meeting is to have a “Committee Draft” (CD) of the proposed C++20 standard that can be sent out for review. Boost It’s been a quiet couple of months for Boost, since we’re between releases, and I have been busy with libc++ and WG21 activities. There have been a few bugs to chase down, and the dealing with change requests for the libraries whose maintainers have “moved on” takes some time. However, it’s time for another Boost release (1.70), and I will be acting as the release manager again. The release calendar is available (as always) on the Boost website. The beta release is schedule for March 13th, and the final release for 10-April. Conferences I had submitted talk proposals to three conferences, and all three were accepted. Hence, I will be speaking at: LLVM European Developer’s Conference, April 8-9 in Brussels ACCU, April 10-13 in Bristol CppNow, May 5-10 in Aspen, CO
The Alliance engages Adler & Colvin to complete IRS Form 1023, Application for Recognition of Exemption Under Section 501(c)(3) of the Internal Revenue Code. Completing this form can be a daunting task because of the legal and tax technicalities you’ll need to understand. Adler & Colvin is a group of seasoned attorneys based in San Francisco, deeply committed to serving the legal needs of the nonprofit sector. The firm brings an unrivaled depth of expertise and passion to its representation of tax-exempt organizations and individual philanthropists.
Monthly update (or, what Marshall did in December) There are three main areas where I spend my time. Boost Libc++ WG21, where I am the chair of the Library Working Group (LWG) Boost: December was a big month for boost, and much of the first part of the month was taken up with the release process. I was the release manager for the 1.69.0 release, which went live on 12-December. The final release process was fairly straighforward, with only one release candidate being made/tested - as opposed to the beta, which took three. In any case, we had a successful release, and the I (and other boost developers) are now happily working on features/bug fixes for the 1.70 release - which will occur in March. Libc++: After the WG21 meeting in November, there was a bunch of new functionality to be added to libc++. The list of new features (and their status) can be seen on the libc++ website. My major contributions of new features in December were Consistent Container Erasure, char8_t: A type for UTF-8 characters and strings, and Should Span be Regular?, and a big chunk of [Extending to Calendars and Time Zones](https://wg21.link/P0355R7). This is all pointing towards the January 16th “branch for release”, and for the scheduled March release of LLVM 8.0. As the “code owner” for libc++, I also have to review the contributions of other people to libc++, and evaluate and fix bugs that are reported. That’s a never ending task; there are new contributions ever day. WG21 Being between meetings (November -> February) there was not any special WG21 work to be done in December. There’s an ongoing stream of bug reports, discussion, paper reviews that get done between meetings, and there is a series of papers that I need to finish for the pre-Meeting mailing deadline on 21-January. I have 1 1/2 done, and need to do 3-4 more.
WG21 San Diego Meeting Last week was the fall 2018 WG21 standard committee meeting. It was held in San Diego, which is my hometown. The fact that I helped organize it (while I was working at Qualcomm) had absolutely no affect on the location, I assure you. ;-) This was the largest WG21 meeting ever, with 180 attendees. The last meeting (in Rapperswil, Switzerland) had about 150 attendees, and that was the largest one until now. There were more than 270 papers in the pre-meeting mailing; meaning that people were spending weeks reading papers to prepare for the meeting. Herb Sutter (the convener) has been telling everyone that new papers received after the San Diego meeting were out of scope for C++20, and apparently people took him at his word. This was my first meeting representing the C++ Alliance (though hardly my first overall). The Alliance was well represented, with Rene, Glen, Vinnie, Jon and myself attending. For information about how WG21 is structured, please see isocpp.org. I spent all of my time in LWG, since that’s the group that I chair, and the one that has the most influence over libc++, the library that I work on. The big news from a library POV was that we voted to merge an updated paper based on the Ranges TS into the draft standard; which means that (barring catastrophe) that it will be part of C++20. This was a huge paper, weighing in at 220+ pages. We spent several days in LWG reviewing this (and a bunch of time at previous meetings as well). We also moved a bunch (around 25) of smaller papers; too many to list here. Detailed trip reports can be found around the web: Herb Sutter Reddit The next WG21 meeting is in Kona, HI February 18-23rd.
Initial work on Certify complete It’s been mentioned in my initial blog post that I’d be working on a TLS certificate store abstraction library, with the intent of submitting it for formal review for Boost, at some point in the (hopefully near) future. The initial setup phase (things that every Software Engineer hates) is more or less complete. CI setup was a bit tricky - getting OpenSSL to run with the boost build system on both Windows and Linux (and in the future MacOS) has provided a lot of “fun” thanks to the inherent weirdness of OpenSSL. The test harness currently consists of two test runners that loads certificates from a database (big name for a folder structure stored in git) that has the certificate chains divided into two groups. Chains that will fail due to various reasons (e.g. self-signed certificates, wrong domain name) and ones that will pass (when using a valid certificate store). I’m still working on checking whether the failure was for the expected reason. All the verification is done offline (i.e. no communication with external servers is performed, only chain verification). At this point it looks like I should consider, whether the current design of the verification code is a good approach. Using the verification callback from OpenSSL and asio::ssl is quite an easy way of integrating the platform-specific certificate store API it causes issues with error propagation (transporting a platform-specific error through OpenSSL) and may be fairly slow, because it requires certificates to be reencdoded into the DER format so that they can be fed into the platform-specific API. An alternative to this approach would be load the entire root certificate store, along with CRLs and OCSP configuration into an OpenSSL context. This is potentially a little bit harder to get right but may offer better performance (no reencoding required when veryfing certificate chains) and eliminates the issues related to error handling. Further investigation, as to which approach is better, is required. Don’t forget to star the repository: https://github.com/djarek/certify!
The Alliance is a Gold sponsor for CppCon 2018. This conference is the annual, week-long face-to-face gathering for the entire C++ community. The conference is organized by the C++ community for the community. Attendees enjoy inspirational talks and a friendly atmosphere designed to help individuals learn from each other, meet interesting people, and generally have a stimulating experience.
Damian Jarek joins the Alliance as Staff Engineer. Previously he worked on a number of embedded networking projects for a few major clients. As a Staff Engineer he’ll be working on an open-source companion library for Boost.Beast and Boost.Asio, which will abstract away the platform-specific details of acessing system proxy settings and performing TLS verification of a peer certificate chain using the operating system’s key store.
Marshall Clow joins the Alliance as a Staff Engineer. Previously, he worked at Qualcomm for many years. Most of his time is spent working on libc++, the C++ standard library implementation for LLVM. He is also a member of the C++ standards committee, currently serving as the chair of LWG, the library working group. Marshall has been contributing to the Boost libraries since 2001, and is the author of the Boost.Algorithm library. Furthermore he maintains several other boost libraries, and moderates some of the boost mailing lists. Finally, Marshall has graciously taken on the role of release manager for several Boost versions.
The Alliance engages The Law Firm for Non-Profits for legal representation and services. They are passionate about supporting, advocating for and partnering with non-profits and the people behind them. For more than three decades, those looking for assistance with non-profit law throughout the United States and around the world have relied on the attorneys of The Law Firm for Non-Profits for superior legal and business guidance.
The Alliance is member of the International Committee for Information Technology Standards. INCITS is the central U.S. forum dedicated to creating technology standards for the next generation of innovation. INCITS members combine their expertise to create the building blocks for globally transformative technologies. From cloud computing to communications, from transportation to health care technologies, INCITS is the place where innovation begins. Membership in INCITS allows voting in official WG21 meetings.
The Alliance engages Foundation Group, a non-profit formation and compliance services company. Foundation Group delivers a comprehensive 501(c)(3) registration service with a 100% IRS approval rate.