Google tells us how broken the traditional security certificate revocation system is, that we should not use it, and that Chrome's unique CRLSet solution provides all the protection we need . . . |
ZDNet's Larry Seltzer: As a Google spokesperson said to me, “...[W]hen we identify a bad cert, we update our revocation list, and Chrome users are automatically protected.”
This page answers the question: Is that true?
What is Chrome's CRLSet?
A “CRL” is a Certificate Revocation List, a list of the serial numbers of unexpired security certificates which have been revoked by their issuer and should no longer be trusted. Each issuing “certificate authority” (CA) maintains and publishes its own CRL containing the current list of certificates it once signed and vouched for, but which should no longer be trusted. CRLs are in a continual state of change as old certificates expire off the lists (no longer trusted anyway due to their age) and serial numbers of newly revoked certificates are added.
The Chromium project calls its custom implementation of a CRL a “CRLSet” because it contains a carefully selected collection of revoked-certificate serial numbers published by many different certificate authorities. In other words, the CRLSet consists of a set of individually curated CA CRLs. The Chromium project defines the intent and goals for their CRLSet clearly on this page. To keep the CRLSet manageable, its size is strictly limited to 256k bytes. Consequently, an oft repeated admission by Google's engineers and spokespeople is that their CRLSet “does not contain all revoked certificates”. This page carefully examines that admission.
We believe that this examination is important because the topic of certificate revocation is quite complex. In a factual vacuum sound-bites are powerful. This creates a great deal of confusion within the industry, resulting in articles such as “Chrome does certificate revocation better”, recently written by industry veteran Larry Seltzer, published April 21st, 2014 by ZDNet. We'll come back to Larry's article, and the Google spokesperson he quotes, once we've established some context for evaluating his and Google's claims. (Larry also wrote another piece following up after this page was published. See the end of this page for more.)
What, exactly, does Chrome's CRLSet contain?
In order to evaluate the real world performance of Chrome's CRLSet, we need to understand exactly what CRLSets are, and how they operate. When you reach the end of this page you will understand everything about Chrome's CRLSets.
Chrome's CRLSet file format is well documented, straightforward and easily understood by non-programmers. It is divided into two parts:
Below is Chrome's CRLSet #1596, the first 1,000 lines of 24,275, retrieved from the “clients2.google.com” server on April 23rd, 2014. (The entire file is available, contained within our DIY Kit referred to below.) It is displayed within a scrolling window so you may browse around to explore the file's contents.
{
"Version":0,
"ContentType":"CRLSet",
"Sequence":1596,
"DeltaFrom":0,
"NumParents":53,
"BlockedSPKIs":
[
"GvVsmP8EPvkr6/9UzrtN1nolupVsgX8+bdPB5S61hME=",
"PtvZrOY5uhotStBHGHEf2iPoWbL79dE31CQEXnkZ37k=",
"Jdoa1Yu/z7In2HI7GFfUwY57qnQXtPnv+TZrXoafizk=",
"li5LVLuYp+5dX+uWM/mR08MwDpUU2t57DU+CjHlPjoc=",
"yP3cdcsb27WMB7TqhHKH9iZlndZrwQomrdm1dbOgo40="
],
"NotAfter":1398627981
}
019406d575cf285a3c2d8bbf8133e0cfae4839c99cc1815bdf244487259e7cd2
11270b1308d38971db8f728e2956c8d38bc7
11270b612ddbed52a12dfa3ab5c9317c486a
11270e0d85729204151e40e55374d140b395
11272184dfda95993c83e575142b0d209185
112723158fe05c8de8c2bb7357a23c13b9d0
11272874436692a4fbd793d737c7ba4fd494
11272f54911df5a61d9ba8d0e9837093a23a
1127367d2d5a6e0b40c12686185e6bee8bc2
11274f9fca0f134199d919bcd5c834ef4c99
11275179516d2006d1f4b6d69db7f8888550
11275928b0b00ec56b091c982884d43849a0
11275e9345cf322113a7734dbeb3812c802f
1127629afc9c0e78711b8d39482815158e04
11276feb7d814316ef9bfff72383cd70ddbe
11276ff7431188b4f2244b0f9fa5c1b8cc51
11277112e2615faf0f7b7c0680aa57b9d70b
11277235c0c4e3721ca04c546b9d0b17b9a7
112779fd75b9555ecd2bb0f39fa582cf4c33
112782f76421ed8856978276ba212d8bd86b
112783536ea08730d778bd14d7a9164ab451
112787d33c8f75e5b12dba7c68fd95d6f5e7
112787e72f3d224df439f0735dbb9c471bfa
11278e0cd72d0d031fd549f38f57b6b521d8
1127928e1ada9795875fb7ae78f78377a337
11279cff4f7c46a800f84f7dfe64febe1ed4
11279d0e8d78728756b63f5fdbe11827c453
11279fea6f39a7d0e67ec7fca60358f8dddf
1127aade5d4be26bc4629f392174c9e540fd
1127ab8541690e87629a1b932896e5ca87e5
1127acd012e79d32d096c872c61acb44385c
1127bde8c0f8015a2afcd8af67939b53044f
1127c1c465aef729cc2a53b9f6664da7f7a4
1127c8c2a337982c3be14ad5ca0ce0e6ff25
1127ca5e1e5d2afd9943606af13fca32ebdb
1127d0b9eee07a5c0f2ae14d82dfebda78da
1127e8f7e8ae2746340cfa497c0d2d3fcb2d
1127ea5a69561302be4a96d834775a6e42d5
1127f825909ab002b795e07e8425337580c3
1127fecd29259d1bff4e5509992e156cb824
051cf9fa95e40e9b83edaeda6961f6168c7879c4660172479cdd51ab03cea62b
15dab72c00000000320c
40388f43
40388f59
40388f5b
40388f67
40388f68
40388f6a
40388f73
40388f76
40388f77
40388f7a
40388f85
40388f8a
40388f8b
40388f8c
40388f8d
40388f8e
40388f8f
40388f90
40388f92
40388f95
40388f96
40388f98
40388f99
40388f9c
40388f9d
40388f9f
40388fa1
40388fa2
40388fa3
40388faa
40388fb0
40388fb9
40388fba
40388ff5
40388ff6
40388ff7
40388ff8
40388ff9
40388ffa
40388ffb
40388ffc
40388ffd
40388ffe
40388fff
40389000
40389001
40389002
40389003
40389004
40389005
40389006
40389007
40389008
40389009
4038900a
4038900b
4038900c
4038900d
4038900e
4038900f
40389010
40389011
40389012
40389013
40389014
40389015
40389016
40389017
40389018
40389019
4038901a
4038901b
4038901c
4038901d
4038901e
4038901f
40389020
40389021
40389022
40389023
40389024
40389025
40389026
40389027
40389028
40389029
4038902a
4038902b
4038902c
4038902d
4038902e
4038902f
40389030
40389031
40389032
40389033
40389034
40389035
40389036
40389037
40389038
40389039
4038903a
4038903b
4038903c
4038903d
4038903e
4038903f
40389040
40389041
40389042
40389043
40389044
40389045
40389046
40389047
40389048
40389049
4038904a
4038904b
4038904c
4038904d
4038904e
4038904f
40389050
40389051
40389052
40389053
40389054
40389055
40389056
40389057
40389058
40389059
4038905a
4038905b
4038905c
4038905d
4038905e
4038905f
40389060
40389061
40389062
40389063
40389064
40389065
40389066
40389067
40389068
40389069
4038906a
4038906b
4038906c
4038906d
4038906e
4038906f
40389070
40389071
40389072
40389073
40389074
40389075
40389076
40389077
40389078
40389079
4038907a
4038907b
4038907c
4038907d
4038907e
4038907f
40389080
40389081
40389082
40389083
40389084
40389085
40389086
40389087
40389088
40389089
4038908a
4038908b
4038908c
4038908d
4038908e
4038908f
40389090
40389091
40389092
40389093
40389094
40389095
40389096
40389097
40389098
40389099
4038909a
4038909b
4038909c
4038909d
4038909e
4038909f
403890a0
403890a1
403890a2
403890a3
403890a4
403890a5
403890a6
403890a7
403890a8
403890a9
403890aa
403890ab
403890ac
403890ad
403890ae
403890af
403890b0
403890b1
403890b2
403890b3
403890b4
403890b5
403890b6
403890b7
403890b8
403890b9
403890ba
403890bb
403890bc
403890bd
403890be
403890bf
403890c0
403890c1
403890c2
403890c3
403890c4
403890c5
403890c6
403890c7
403890c8
403890c9
403890ca
403890cb
403890cc
403890cd
403890ce
403890cf
403890d0
403890d1
403890d2
403890d3
403890d4
403890d5
403890d6
403890d7
403890d8
403890d9
403890da
403890db
403890dc
403890dd
403890de
403890df
403890e0
403890e1
403890e2
403890e3
403890e4
403890e5
403890e6
403890e7
403890e8
403890e9
403890ea
403890eb
403890ec
403890ed
403890ee
403890ef
403890f0
403890f1
403890f2
403890f3
403890f4
403890f5
403890f6
403890f7
403890f8
403890f9
403890fa
403890fb
403890fc
403890fd
403890fe
403890ff
40389100
40389101
40389102
40389103
40389104
40389105
40389106
40389107
40389108
40389109
4038910a
4038910b
4038910c
4038910d
4038910e
4038910f
40389110
40389111
40389112
40389113
4038911d
4038911e
4038911f
40389120
40389121
40389123
40389124
4038912b
4038912c
09451613534eb3495e503782f9c3951e57785f9bd95bd4756f594426e40fbdff
01000000000132825f3d92
010000000001339385b1effcd6db
01000000000133c6ebf4855d87a4
01000000000133c6f18ac50041ac
01000000000133cbe294998bdecd
010000000001356343f91435ce5f
01000000000135eebbcdb589fe85
010000000001362c710d535135f7
01000000000136c5ee63b356b43c
01000000000136eec29faadd765f
01000000000136eec328df50b27b
01000000000136eec4eb4fbeadf4
010000000001379a93eaaab2b390
01000000000137cca26bc6fb18c4
0100000000013907edfe91a009fb
0100000000013907eeafa591ada8
0100000000013907ef244a67b129
0100000000013907ef866e2c625f
0100000000013907efe8879a2df9
010000000001390cca3988bd517c
010000000001390ccb1157d28023
0100000000013954b83c8f8666e9
0100000000013958e6534c19c436
010000000001397351a7ff022656
010000000001397dd0fe52c3bc57
0100000000013a673fb5b0d8bdca
0100000000013b8bf1e79017bad0
0100000000013bb5497e1a515e69
0100000000013bbd7ce63e4d5195
0100000000013c26640466dc58cc
0100000000013c4f738409881820
0100000000013c4f7447929842a9
0100000000013c6f0ee0fe831fac
0100000000013c8ba946358f4231
0100000000013d69a499b97ae7c5
0100000000013d8d4c437ccf4563
0100000000013d8e949ed723499e
0100000000013db7115ced879634
0100000000013df5bf91e9462888
0100000000013df6a52a1681491b
0100000000013df6a63bd105d7fa
0100000000013df6bc279887cccf
0100000000013e0f2793d32f4512
0100000000013e13437867dfac9d
0100000000013e193f67f7911a4e
0100000000013e194262990b98f0
0100000000013e194570c530edee
0100000000013e1e1ff22164fe43
0100000000013e7c73897bbf8f82
0100000000013e7cde294ac08e14
0100000000013e81c199b43ae847
0100000000013ea8dac293f6d94d
0100000000013ec4d3f6cbabea8b
0100000000013ec8cd956760cc7f
0100000000013ec8d2a1af4adf17
0100000000013ed1f41163fa3659
0100000000013f148aeb5f577536
0100000000013f1c2071f32f1adf
0100000000013f1c213587e0fa72
0100000000013f378f4e7bb23d39
0100000000013f43ec169fc4aba0
0100000000013f5ee29fa6d5fdd6
0100000000013f5ee4afb3eefc15
0100000000013f5ee65e01f9c0af
0100000000013f5ee6d35b3b1c1d
0100000000013f5ee748d190f385
0100000000013f6358de63696994
0100000000013f635967611ed2a5
0100000000013f9e9507fb449919
0100000000013f9e9b5eef548a20
0100000000013fc0ce217cea86ea
0100000000013fc51f47b1a5c102
0100000000013ff33bb4fefaf1e9
0100000000013ff33d633a2d7166
010000000001402b7c312b3f9f75
010000000001402b7ca67c1d3cfe
01000000000140543a4841974bd4
0100000000014054b2f86a91b7d3
0100000000014054bf586baa9c4e
010000000001407e037546f2dffa
010000000001409c04be05e2f546
01000000000140a16f9a48745b1e
01000000000140a17ceea99bd002
01000000000140a6bbe359ce7531
01000000000140a7724b7d7833ca
01000000000140a779db4ca69854
01000000000140bf37c533d3b645
01000000000140dee6e2339edd25
0100000000014118e1b22dad6bd3
0100000000014127d3de6774ff20
010000000001412ab942efe68ed4
010000000001413b462fa506d87b
010000000001415664ff37c0ef13
010000000001416ea2f71a68a44b
010000000001416eaf6a5f32c9e2
010000000001417477d8e876f777
0100000000014198bf3100f9d2ef
01000000000141a1ea2577986d9b
01000000000141a2863817322a77
01000000000141a29b125c9be44e
01000000000141c1970ca6008a74
01000000000141da8c8420eadafa
01000000000141decac498950a4d
0100000000014208f09e5f6875e2
010000000001422ed6cc380297de
0100000000014236aba0d45f4781
010000000001425de11526689ab9
0200000000013326febf5384f271
020000000001339385ab65de37d3
02000000000133b2595738988647
020000000001350bbb79711c62d2
020000000001356343fe2a4270c7
020000000001361c42e1e08e6195
02000000000136543536a0b172b2
020000000001367e1bcf899b81dd
02000000000136c5ef97e82c129f
02000000000136eec881732ffb37
020000000001372cc14ef4a2518f
02000000000137a42dd7c891b8c9
02000000000137ebe9b5010b32f7
020000000001385c8dd47e0b3bf2
020000000001390bd0f816608e9a
020000000001390cb666f43402b5
02000000000139551b0576c8e865
020000000001397dbe30741e7ece
020000000001397dbef5ea9f181e
020000000001399b481cee43e7f7
0200000000013a07cafd8f1f2cbc
0200000000013b3e1f98272aa77f
0200000000013b41a05d69540f29
0200000000013b43e53e2468cca6
0200000000013b66ba2a101bb700
0200000000013b963ca9a9b0d54a
0200000000013bae7850eb21a744
0200000000013bd79fba80325370
0200000000013c1af86a85953a09
0200000000013c4a28e26ed895e7
0200000000013c4f4daf3e4d7ef3
0200000000013c737d27dd0c8d8c
0200000000013cad4eb822de600b
0200000000013d1bf17a69bdd14e
0200000000013d645ff1f2a334c0
0200000000013d653621d76eb3db
0200000000013d6931ec52e53051
0200000000013d8957a45bda8d5d
0200000000013dcbe8b61c8a8711
0200000000013df697ecfe4b5f48
0200000000013df698b090abb5b5
0200000000013df69a724b1916f3
0200000000013e1327597411252b
0200000000013e143f129a8bacd1
0200000000013e17effedd5c27dc
0200000000013e1917f44d75b8ab
0200000000013e191b161fa5122b
0200000000013e191c3b6478f055
0200000000013e191e4b72213501
0200000000013e1921327c8e15f1
0200000000013e1924a277ad5869
0200000000013e1925c7c0324d3b
0200000000013e1926ed30b5c283
0200000000013e23c6cca6746b2c
0200000000013e3878fc6bbde200
0200000000013e3dc898fcdc2aae
0200000000013e57ce7b9c814b04
0200000000013e58560fb47079ff
0200000000013e65f370bcceaef9
0200000000013e8afd535fb90f58
0200000000013ea3e41e55d35c1d
0200000000013ea3e5b907c2b31a
0200000000013ea477892856e19b
0200000000013ec4b25053be79e5
0200000000013ec8bcfb11e2ced9
0200000000013ed244a6d94aa341
0200000000013ef1ed99a3bd0137
0200000000013efc9cda7b565040
0200000000013f1ad8ea715da345
0200000000013f1c1fc8d0ae5edb
0200000000013f1c20c6fda6dc8a
0200000000013f1c2338ea6e70cc
0200000000013f3539130004082b
0200000000013f375f0ad4d6f8bb
0200000000013f5ee4d9b2383e57
0200000000013f5ee5629f106f43
0200000000013f5ee6d625cb14b1
0200000000013f5ee737f75e692a
0200000000013f5ee7ad5759bd2f
0200000000013f5ee95c874f3a04
0200000000013f5ee9d1f723e430
0200000000013f5eea33da7ddcbc
0200000000013f67674ea36d3d67
0200000000013f9a172b50ec4da8
0200000000013fd24c51f7d9b187
0200000000013fd4238f3ec741ad
0200000000013fe69a526b62cfd8
0200000000014006c5cfb8716869
0200000000014006c7dff63d65ee
0200000000014008ca6fca575e24
0200000000014008cb5a6ffd08f5
020000000001401bc17eba24c718
020000000001401c822a35cd5e43
020000000001402b7ef2d74328e7
0200000000014053117819860b7a
02000000000140534b5dc7343990
0200000000014053c9db6409306c
0200000000014053d66250ed450a
0200000000014053eec028645071
02000000000140541f2ea55bf601
02000000000140543010380fca2a
020000000001405435cafa1a98ab
0200000000014054449cf391b3a0
02000000000140544a4428fb2a91
0200000000014054bbda92ebadb8
0200000000014054ca98c8aeaff0
0200000000014083708d4c75207e
02000000000140a770e45901d659
02000000000140a774dd49b61119
02000000000140c07f96e5f96720
02000000000140c080cfcb8bb53f
020000000001410165472f40e7c0
0200000000014126dd006670fec8
02000000000141279b10db897488
020000000001416eb82f2aebf038
020000000001419cc32cf851df7b
02000000000141a18087bb594db0
02000000000141a263b0647749f0
02000000000141c3cf9883ffb9d7
02000000000141cba36aee89c309
02000000000141e70ddb1c4c2278
020000000001420cd2bab39b5852
02000000000142229e960175bfe6
020000000001422879a84ac9fefa
0200000000014229648decc12fbb
0200000000014229676176cc3826
02000000000144085d02a9d1587e
0c97420a5c8c51800720f65508f3c95adcbca45922811992aae6bc452f142699
7e240a4986f84ba7
0ee0d04926b8b9b693cda6f93018e1d8dbeb1240f3b75d243c47670d050f9d9d
011444d7fa7044453c6bd7a165fce70b
01eb4d84b2f2cfd2817249d4ee7fef61
0243352a12001fef31a99b0b16c30f7a
0246ed711cb180f92b0b4c03ba865fae
032f058e63abce632ee3271278361d48
033c1520fbd0c1c630cca3b55a27d3f6
03671f8ba5276f57b972aa9467eda14c
03f917eae05b2a952c45b2b728578e9e
044346af53f06e3192a7c976d140b28c
045c5c8349844af9d1f923c24f3d2c13
0474462702771ad4a63f3e999660c41c
048de9ddc53ef7ffd253f8e296046957
04bbebaf3d94526096fd5a644cb4d6a6
05f68f60c30e66e1b5de9d84b3135e93
06449fc6f92b397e24cb66933a804581
06c6f4503d93bf70d861a37af8559079
06da57a57bd63ef5be35ac43162ca1cf
071590952587be7a2b37e44e5ee361e8
07ac9be3d25475efd1a6ee5aa2cd17e9
07f7095350df13121ef00158772d73cb
098fac5de86ea51b8342bde5a5cb12d8
09fbefffbe3034cd00fa03daf5ab2990
0a419f4f9196e8804c1adceb6e29f8d6
0a9ef4166ae8a26f5c52b0014996eb9f
0af8ba45194b2b765a75edabe813c7f8
0b0b6e055947268e41c0caad185f79ea
0b15e143be1788af83916ed3967662a7
0bc4c59ae662999defd7342f8dfae0c7
0bcd8d5c6e6930335e19e16d531b48fe
0bf447c3a803284eb2758309f1ad3195
0bfd5e13fca843e73456aaf758718380
0c4930845f1765e652c5a2c4f90fc11d
0c8270de4f8731d7bdf79be97746e46d
0cc8ebea550e5223a493033cd785b5c0
0e7a1b052623137135d676497d44be87
0e7f498919a466274b7e878bdf10e464
0e9bac5262bb0ef3bc3e56a5ae112fa4
0f3ec633d8b7b3f1b3ba1438f5eedb98
0fb349cd8d26d0d803b26d8c95ca2f61
0ff2bb486a62c7db824f1bd6a940316a
1055dd788182e46f84ce68e340ceb3b9
1075c9f76bc155d6fe9f5f82abec1f89
109e7d126aeca06e4f0416ac8b3ea65f
10bb1c8947a45412da0ba22601205094
110f2ab26d4f7e69f83dd9386916e658
11469cc16f133fccf4c4ec1579d92c0f
11d34dbc3ac6f5612ec8d262393f3e35
122c16da3996271019a387ea02abcfe5
125d31f099a9119143d4cff29d19623e
12c7d823c688dba5d446628a411195ae
12c7ef9cd052b491ab9b5958ccc161c8
12e2cecea1dcc35f2ffa5c5107a6aeb2
13b90280a57432f4cfda87e124653f9a
13e164ae7a9a164d1562993db38a3556
148f1dac7a1a56aa48acd4f2739480e6
14a8fbad33fe3f7bf165ea718c04c836
14c98efb21ab557ea7037bc84bdb5d90
14d5019881267235f8971d64e8d5daa3
15b5aa9c6fef2c614152d06306a00786
15f07b07ba023d8ee8b2844b646a8421
1628ab8474af95d4ce24d39b770a7808
165b8fa08d9de04bcd931bff924d70bf
170c3af70195145fd6ed4c166d7d1584
171d0b83baed760aceddf831bbe26fa5
177f7bfdd9c1baf40ef116d1561e2fad
17bcc83bcc699ae576823e187c3ba541
17c81503006e153a868555325360e743
17d7170f6d60405333624bf669a11a0d
180a520e4b4e0b08270b3034c67c0241
180ed0ebd9838a85add8469cd85e6364
182c62111a835ebc311467b49b21c8e8
183f9c1be17539fab36aa3f0eb9ab32f
188aea9b4807ed1c0cf53ba526f358ba
18bfa63a45bac3de78708ae6f16940b8
1908986446940a2afb495c211d1a335f
199fabf08197c33b9b5825717b09f5c2
1a40283e63e05d351f2e12d8353a6b8b
1a4d4ea963f7825e850377e369e10123
1a91cc553510fe2400269702a47a4717
1a94769fe93ea351c6aeb68eebb04d6c
1aa27e98a10546489054ba35d208d1fb
1aa64d4815194d38fb4d463eabf86628
1aa8cf10f0b9a355d2e0c908d1b23c3c
1b192c9e0a318346dbdf514c3a684531
1b3d44f60a3065bc3e3990bdd06ffce2
1c119e4f12ee1ceef28c1ee5cc197735
1c61b2b17a9206ba272eb0a281107f8c
1e4d5f25f9a4427f6951f85e1ad5d462
1e62e9942fdbc7448f385541b3be93e7
1f13cb784ca161daa02772f2a9332dc2
1f3a52a9f16b937ee6be3c82bf00cb0b
2008207f05d26f5feb18e874feea89c8
2017227e137f723291d07ceea69ececd
20448b1a678ecaec722d4e0efaad0bc5
20779e46efbc82c4f3a1d5a4d8547960
21878f3deb48aa30e677a0bc5b8610e8
219435fdd5174af03587f00b84daf5a4
2203088686117bf249401138b4e2631d
2210e965e15d4e8e960efe3e3f482657
227e60247b88c9516e6d5ba89c7b758b
2356da3a5c0251f81152c87b4b18b956
235b709a64d073bc9bb01094293d214f
23e8c2ef8f7344afbcdeba72a956e4ee
2442ee9443c759b63ae97a9f07826595
250841e1712d727099064010fd562251
2513a6291085784bb2abd13103b51a19
26b434ecb6df042244604b5a3be5fded
273de69b25acc9ae9950a09bcd49747b
2785977725c666504505bb713d1b4967
27c8f76674a802b41f0c56422fee276f
282c88d34d8fbdc4b315fa18efc170d0
2857c004aa0e0ca33f20b65b03381437
28f58f4c34983056e6fd323bcdd2edba
2918a8b67ff6cf5474cbc0af03e54d3d
29a7c2d77ef0053cfa8042e5fd21a06b
29dbe551650ed26e309cd2650c8bc736
29e0811b9c9a17b96b3f29c45306a70c
2ad301eabb3048774bee7d1af8c121e9
2b857c7d258e3836ea3d83d2255200ca
2b96dc6990a46a250c958da93d8a608c
2bd63501d165548982cb047069bdd0f2
2c0ec7f77c660e3808ef5cb61c4de85c
2caff2473aa441051394a0f1d857f4d5
2cee285860ba8899dd0f648bd3967543
2cfa67777a60b42075a72075d88e8e3f
2d31d98aaa7a15db3b9015a2cd64cf44
2d4ad01dc2675ce44a80d8dd85f7e748
2e34d255e364997fa439ebe2e638721f
2f71b417c857026ccb5abae10424a21f
2f99328217a025289cc65b3703b85a8b
30664f1e6df68f387376106ecf7475be
30fba900b263d7328c4f73a22ce56749
31980603dea9dfe743b8961669e5ee2f
31ba7b863598f6501ab9b7a0488c1c35
32ebbadb708b3239693af8c9c6bf017c
33e724bde4c2f5fb1ec83ba64f4e06b2
34268631936fc0e9a5a9c7f790e9d97a
344cf23913c2c490d545c48f70928ea0
34549f318ab80416fe26a5abdf353b85
3486101d9131f0cc7bb40b75df16be65
349a9818abe3cf1dec8a3c3e02f0fbff
349d6c8fb1f01ddd160ff4f02f1b2e19
34d48b0b3e27f38f9b5a634b48dc4867
34e8dd18af32cc7b96711e94c9efe4ad
350c17924ca62dc1bc2e10299e4e2880
35223d2129d3de29dcbcc1de0e175d1b
35c8a0312cc175c142fbe17ddd4b715a
360a0df121739273e4cd2aab969d67dc
36bb17555bf3def0a5694ee085bf011b
388fbb4981188c8122d46d98018d5500
38924baca5f21b60955ba507e1c9628b
3945f8594da353b262b5c914506c5981
3947025d202eb30327f34bea804588e8
3a78099d9280945ce553366d5593f707
3ab37c7eac7aec0114359b694c92288a
3af0645a5362657c18272bf6f2fc025f
3b2333ea3936edab7ebd26395b34551a
3b7caac5aaf695041e402fd046374f69
3c1ae78bff841dc9fbf7c4a91459e125
3c8e4249ef142786e9b56601191ad424
3d09c14c080c2664e4993dac274e5411
3d2622bf1a04079f86e350a08d6b5f36
3e0ca0282ca32194290cf18238f1d949
3f56cbaafa810d003dbb585f9f6dd1bd
3f5a567c2c1635994d4bd884d930f355
40053fe7934d1e28fbfd4bc932c96d45
405d2b2b47b9c0abfad3c7e430eeac56
40c81277d901a592fbc0b771d7417db8
410fda288acf35eef957778022f6733a
426effdcdba456988e9f98667de68a34
42d6835df03b65557ab48a5fad4fa6e1
43c38ca7b6d89d43b040293c5fe53084
4488005d506b2a5c59b02f15179f73ab
44bed28abe75286a2ea0e74654ac6513
44c405edfa5d02a9742dd8532dee4676
454460c09b689b5aa765efa050884ae6
466224db2e5cec56ae1a9fb57db83969
47757909a6008c2cce0706723208e3c3
493667ea6218ceb59f3ba54240b0a49d
4994e8fcc0a121f732d21478d026af27
49dddd641ce8941ad9ab258e18ba1038
4a2ebc5e51290a3f76579f5a0bb5781c
4a38d3280a5c0c75e39f18a5ae943c33
4aab4948d7455e065802199e41b94163
4acb98faa730715965201c5de234a75f
4b4db6ee2d2ffda92ddbd1ac0cd03548
4b9c95aeabacb75e5726cf8bdb368db6
4bafe6bbaae04b640a44b56910aa9d74
4be19fedb6c8ece012bf7743c33d5f9c
4c264b11237cbfa937c9fc1af8128321
4c87795576af05327d258d061bebecf7
4da6a7a70766b0afa47bc2e08f0396c5
4da7193f58f61f769525d8eb6d249870
4daf282c20b6d47e2ac4694345eabbed
4de33e5d12a2e2550aba87986f152667
4e517525511b42dec0442eb821b09aed
4e639a56bc10a4664b2876c26df03df1
4ea50cdcd9046d04e9716945e34148c5
4f1955b52ad021965a659e90bddb80de
4f30d79a1e899e81e30bf084c9027c53
4f32804c0ed749daa1a21ae7d25aca12
4f3d13a64184151c5f97ab942510c268
4f5e826493633d0072f9e6da81cec632
4f8e55122df71469bc2fe7cccdcd1606
51434414ed379477cd91ba648bc011d7
519c628c5b8df952ba4601026464334b
51cce734ba5d2d012e1911c96b50face
52111405542fbcf12c82f12cf0c00582
526d03fa0cac900541bfbf1ffad23b16
526e1df56122a829d0ef2c4e0136474b
537beb024f9d46175bf2e1cf7575f86b
53fc64c3d81e30a461a5abd7c5504de6
5413b950e0851a56c34afd32e9db1448
546b76c3e98cc943dfdd6a8aa48b858b
54d8c910519fe20eb1064cd1e6dbc0d8
552e9df12a0a3e26d92145499f2fcdee
5536c833e99539190b632a6ccc712055
558596f0708c61acbb377e82b806012c
55e2a5605d5a91838229ff8b6729b531
55f73d3cc76679ab69fa4336e3e93a97
55ffe37a94f80ec87d0933bac3fd6363
56048fd56ae5ba2ab60e441981d7c8b9
56238baeb37cb9143d914c36bdba1f8f
5703fa0e43719d2c9eb176b5800018f1
57dce7bebd7cca21021b0cf634707ed9
57f20dde3851d0449d497afdf5e42ec3
58223457a2c1f86f88571b75309548db
584ec09d01289577a687e7b1bac1c9cc
5895462d8bb46e6f1832ab937e7ccec7
594bc76107ccdf3fe014e8b18159955a
597f365dbd78b46c5a8a613ea726b69a
59d8b2570f94d04138ddda1c1be6dd02
5a08e81c7c168e08935eb0564c0e4c22
5aa95f2d5b359df58e8574f1e1bc0b12
5bf74017b1e14f9c126dda9c70701da6
5d487986e597f44054c7841a23945c76
5ddbef0aaa802f33e7ef94fdcb9e1850
5e2b73f9d36eb346be9c77327c17477a
5e3416f143b277b9bd27d07a40323eec
5e394787dc4437cd36eb0cea892e5e23
5ed3b72bd3abadd8fdc1c63f576310d5
5ee137c48fa8befe771cd13bdb936b03
5eee69793bc721810aab1fee2e83e2a5
5f808536172ca2b40ad916eb8ae6ebbf
60195ec309123c1d703fe7c6a5e7ebf3
606f0feace4ddfae08b18037a6ed3493
609091fcf9ac0bb720b22be37851c785
60a07b5a69c7978dbe8c1d7c505a38a9
60a1a47ab7773540b0a3f1be3dfb21d8
6137fbfe99405a4c6fc03ec04428d04c
61dbe19741460b40cd8a02dbf5417e97
627f4b0b3c71ce6a19d9499b8c4567f7
62bf12a978c15673b74dcaaf3a5b7b2e
62c6f656ceb5b9f8f50c2f58c74d6196
6349fc6003aea4d2224049fd016bef4f
63f7a840e91201c81cc6e386dfe0871f
64510c9b8580f4f1ec8e335847bd742d
649e2b5c681f25c105a8f41d05e66708
65852e60b24a2fa63b8b075717987a2f
65b85cbcd9559c6a806c622447df089e
6660a0b8ff4f4a1b1285355c5474c760
66662878f61a505331f04ba5dbfed5d8
6673cfa5b13e2e048b3c6d4e2c7c9f0b
678f231bd5996a7e650705725c5e2785
67c8c0204354c409ece0b56a6f80b39a
67c97da56bbd0cd68a7d42eb1514bf7a
681f617d1babb4e70542ff2ab8b51cc1
68af86a478eed00c4284b221836dda0f
68da4b0e45510bb9aa0a5f7af760317a
68effa320fa74779c0e7270c9ccab69b
69690f90799aeb398bc63ea651874787
6a63e20e73bb4115e5d3490b82e44b9f
6b07732933eb37aeadd3c43f58b266c7
6c4c007e203d356971e2954614cb13df
6ce4b1c2477a4342f8e81768e7c1f972
6cffed95647a23b6bd802ef3f2b53692
6d9fea49db5300c1f035898b76a4a974
6db039df27841220fb7a65874f74d86d
6de5b77740d5a200b3aee50198e6ad09
6e1b5baa8ad5bc0ccf4bdd9efc0dea2d
6e268f19e88612d1a0e030cc26b60034
6e7979d57209752a982950a23dc4b481
6e7dcb4914f3a331fc60df4fd3f34438
6e9ae7a008054423dbb70f55673cf02f
6f59a96e9f1b91dc989992c4f6253a30
6f65d2bc9022a8047455c18eedd3390a
6fda53dc72b88b83e9c9e999428fd3ef
6feba3f747593431e629743308159567
701a8884729a1aa4306d27b2965128fc
70ace8db41ed7171c8422d883507a006
70e428b3987aea50eb52676ec85c1b7a
70f03cffa6f6efeae0ea8ba95455d9ae
7126f793d6b14255425cb0046b0a480a
7127d3b417582faaf3054033b91d7f55
7131c25e5c3649785bc557fce7a82c2a
724feff7b4a1e52be4b1cb20e0660e39
728e928290cb5bb8242f1d97604ec84f
72d1ceaf172c8b15c4d2022fa8444557
73203e6a488a006b9216819b40d9a1e6
732dcf309bdbec724a887c1c66d856c7
73b3cdc5e9ccfec6438dae9999c20a8f
73dee78b55e813221b835b152734bb99
74a13d08dce8b01cbd493eedc99d01a5
74aee724c459324ae2912fa1bc11944f
74bfa1f51843ceaae2b8fb585a1b76a1
74f5bc244233966c50fca313b25b35a1
7615859c410f68bdb20dad9901f0abec
765c0993515112e16684c6ec2d898f7b
76a83ce2a444ab45177d8889804895df
783e69223432b6aeca46703d3c63fb8a
78683fab09d92efef92a9952c3e5ec40
789e91750a6371c15dcaf6a1a9209f50
79706345766c0b7a0b64ea5ef365d46d
79a547780408ea77d77cccf2abf16131
79ae8fa83aee118af998bdd5bd5da1a9
7a2ead9ea031ae61d8979f512cb2ce21
7a3d3e7d78d79db4be410cd789beed2b
7a907628e707ff0893ce34e34cb7b213
7b21d32904d3dd51d1f51adbd266adda
7b3fa1123366c497895d0ff943a2af1c
7b63b271963d909b0eb1c36ddeeeb3e7
7c0a6fe56598e4d6ac639a7bc952488e
7c13ff4aadc9a53b278797d41aea5355
7c41becb222fb014fc88e13fce440fba
7c66dc1361753f734779bd6a3c89699b
7cf1af48c60c5fcec5042584888bb4a4
7d2fb015dc0572b2a2650491a2465633
7e1f2e938b13ccb77e828f395000352e
7e7105e37bdaa09b64d03f58da4608fd
7f458440c7d31dce12c233f4dd3aa34b
7f9f2084bc39a1e34d38494bc1a2778e
7fbd6c8b4ce60f42baa33597cb578cc1
7ff67c56a3055b1624a954da635d02c4
808bb3995968192c29e0ddc69cd44166
81a0b84c73057e97a5199b529d41d4df
81b26f1313208fafba04552cb9d9f3f7
81bf4e36c5ec152237635e72cf8ef0b7
821b7dd5cf27ea24049aecd9ac2acb02
826332738db15d82229610c1b895b24c
826b23c18ece07eb90b90c4ae453434d
82ac6548dbbfaf6fd3dc359e13b17845
82d6030b11fff3affa94c1e83c5af9c7
843bcdce77bf62f85027d5006df5f897
849c20fbb522f729c075122ce387ceb8
84cac522c7a7048585e60db6694f3b0b
84e7a677e5e34687c4368266d1d3a680
861535ed93decea3d9ea8a0ff59c551a
861f653c69faa71c4b7ee5969fb0b536
8647b8be37b0abbd7b9776827ecd325f
866edf67adff94f9e0a7aea1eae3e241
867487d0a5074f09d665df4130e3e306
8683f5d5c43299c550c1d3ff997083b8
872624e12a7d67d475f383c9515ad72c
87f3d84aac9aba1aedb9441de7956d26
8830f6a6ace9e7a97ba924b0a40b9a84
8853bcd7110749441f22833efb52db3a
887450f33fd07e3d679337d9946f33ab
88b0eada7375ded151e4018461cb4947
8910757c5bce2781e994649355579ffd
8921e8ade41d23ede958fb7dfad4148d
89b16d369feb37c240329f2ca8dd9827
8a1e552dabbcb3526576b054557b9861
8a2fa3a8eae7fff276ad943dbda27747
8a62699540a680ddb398904820843049
8af46d329d02743f05b49f9d08fb5575
8b3ffcbb510a715e4587ea7fa1255240
8b544f463d587bfda148c34df1e36bbd
8b8b9687ee7e8f9a51e36f5fd3ba60b1
8d14919a573ad9f35880dc8dab175e46
8dffbe074314ab4fa55e748859c2299c
8e7d0dd9914d6a504cc2c18677d10086
8e9cd992b604bb239c9bf7e7ff8d0120
8e9f582d11c88c6b3647a0183869ca4e
8ec7f560347625e2038f373786c7862a
8f40f8d6c12870c9593334ecfe9cf7c3 |
What do these characteristics mean for the CRLSet's effectiveness?
The header indicates that the body of the file contains 53 sets of revoked certificates (NumParents:53). In other words, the entire current CRLSet contains sets of revoked certificates signed by 53 certificate authorities.
Let's look at some numbers:
Apple's site provides their list of 211 root certificate authorities which are built into and trusted by Apple products, and Microsoft's site provides their listings (on four pages) of 353 root certificate authorities built into and trusted by Microsoft products.
| Apple | Microsoft | |
| Number of vendor-trusted CAs: | 211 | 353 |
| Number of CAs represented in CRLSet: | -53 | -53 |
| Number of CAs missing from CRLSet: | 158 | 300 |
| Percentage of CAs missing from CRLSet: | 75% | 85% |
If you are using Chrome on an Apple product, which trusts the certificates signed by 211 certificate authorities, the limitations inherent in Chrome's CRLSet blinds it to ALL of the certificates which have been revoked by 158 certificate authorities trusted by Apple's products. To rephrase that: Chrome will blindly trust every revoked certificate that was originally signed by three quarters of the certificate authorities Apple trusts.
If you are using Chrome on a Microsoft system, which trusts the certificates signed by 353 certificate authorities, the limitations inherent in Chrome's CRLSet blinds it to ALL of the certificates which have been revoked by 300 certificate authorities trusted by Microsoft's systems. To rephrase that: Chrome will blindly trust every revoked certificate that was originally signed by more than four fifths of the certificate authorities Microsoft trusts.
Now we know that Chrome's CRLSet ignores the revocation warnings published by a large majority of the Internet's certificate issuers.
How does it perform in sheer number of revoked certificates?
Examining CRLSet #1596, we find 24,259 lines of certificate identifiers, one per line. Since we know that 53 of those lines are the certificate signatures of each section's “parent,” we subtract 53 from the total body line count to obtain 24,206 revoked serial numbers contained within CRLSet #1596.
But . . . compared to what? That's the question.
In July, 2013, Websense harvested data from their “ThreatSeeker Intelligence Cloud” as described in detail in this Websense Security Labs Blog article titled “Digging Into Certificate Revocation Lists.” During approximately one hour of sampling the Internet for certificate traffic, Websense gathered 10,000 records. From those 10,000 records, 9,066 contained the URL of a certificate authority's online certificate revocation list (CRL). Within that set they found 819 unique URLs (there were many repeats).
Here is Websense's graph of the Top 30 requested CRL URLs:
Credit: Websense blog, July 10th, 2013
Websense notes that the Globalsign CRL (the lower line on the chart above) is the most popular, having been referenced almost twice as often as most of the other top 30, not to mention the other 481 URLs that didn't make it into the Top 30 list. We'll come back to Globalsign a bit later.
Websense then compiled the certificate revocation list content from 511 of the 819 URLs that responded to their queries. Given the nature of the ad hoc URL collection, and their unknown provenance, Websense was not surprised by non-responses. Other facets of Websense's research is further discussed on our OCSP Must-Staple page.
What's significant here, is that by pulling from the Internet's readily available certificate revocation lists, Websense gathered records of a total of 2,232,845 revoked certificates. This means that Chrome's CRLSet, which currently lists the serial numbers of 24,206 revoked certificates, reflects only 1.08% of the revoked certificates collected by Websense in one hour.
The 2,232,845 number should probably be regarded as a worst-case high-end count. Certificates are used and revoked for purposes other than protecting web traffic. So for the sake of argument and scrupulous fairness, lets assume that only half of the revoked certificates collected by Websense are relevant to web security. That's probably very conservative, as additional data will shortly demonstrate. But it allows us to very conservatively conclude . . .
| Chrome's CRLSet contains at most 2% of the Internet's currently revoked certificates, leaving Chrome's users 98% unprotected. |
So that we're not basing our entire case upon a single research study, here are two additional data points from Internet monitoring companies to further characterize the limited reach of Chrome's list of 24,206 revoked serial numbers:
Credit: Websense blog, July 10th, 2013
The apparent decline in 2013 is due to the fact that data for only the first five months of 2013 was available and shown. Websense notes that during those first five months, 766,451 certificate revocations were seen. A calculation of the average monthly revocation rate based upon those first five months will significantly underestimate the year's total, because the historical trend has clearly been increasing. But it can give us a working minimum estimate.
766,451 revocations in five months means an average of 153,290 revocations per month through those first five months. Extrapolating that for the year, which, again, won't account for any rate increase, gives us a low-end estimate of 1,839,482 newly published certificate revocations occurring only during 2013.
Standard web certificates have a lifetime of either two or three years (24 or 36 months) though most are issued for three years to obtain a discounted price. If we assume that revocation events are uniformly (randomly) spread across the lifetime of certificates, this means that the average revoked certificate will remain on its CRL for half its lifetime, or 18 months. Then it will be expired off the list due to age. For the sake of obtaining a scrupulously fair low-end estimate, we will ignore the effects of an accelerating revocation rate which would cause more certificates to be added every month than expire. At any given time, assuming a low-end revocation rate of 153,290 revocations per month, the Internet's collected revocation lists would contain an average of 18 months of certificate revocations, or 2,759,220 listed revoked certificates. Remember that this figure assumes a constant revocation rate based upon the observed revocation rate during the first five months of 2013. We know the real number is significantly higher.
From this data, even after making every conservative assumption, Chrome's list of 24,206 revoked certificates, accounts for just 0.877% of the Internet's currently published non-expired and possibly fraudulent certificates.
Activity on certificate revocation lists peaked at a rate of 3,900 revocations per hour on the day the Heartbleed bug was announced (Monday April 7, 2014). On a typical Monday, we would expect to see a total of around 22,000-30,000 SSL certificates being revoked over the course of the day. On the Monday that the Heartbleed bug was announced to the public, there were 29,000 revocations. On the next day (Tuesday), 33,000 certificates were revoked, followed by 32,000 on Wednesday. These were both above average, suggesting that around 5,000 certificates were revoked in direct response to the Heartbleed bug each day. Note that fewer revocations usually take place over weekends.
The crucial takeaway from Netcraft's rather matter-of-fact baseline, is that they state that on a typical Monday before the Heartbleed incident, they would expect to see between 22,000 and 30,000 certificates revoked. With most certificates having either a three year (36 month) life, we'd expect the average certificate to remain on revocation lists for half that time, 18 months or 548 days. Some will expire sooner, and some later. But Chrome's CRLSet #1596 contains a grand total of 24,206 revoked certificate serial numbers. Given Netcraft's observations, that implies that Chrome's CRLSet might be able to handle a single day's Internet certificate revocations. But given the average revoked certificate CRL presence of 548 days, Chrome can only retain approximately 1/548th of the Internet's revoked CRLs.
Since signatures are made from hash functions which are deliberately “one-way,” it's not possible to take a signature hash and “decode it” to recover the value that was originally hashed. The only way to determine which certificate authorities are represented in Chrome's CRLSet is to take candidate CA signing certificates, generate their signature, and see whether that value matches any of the values in Chrome's CRLSet.
Let's first take a look at Globalsign:
As we saw above, during the single hour of Internet traffic capture which Websense analyzed last summer (long before anyone knew about the critical vulnerability in OpenSSL and had imagined or named “Heartbleed”), references to Globalsign's CRL were made nearly twice as often as most of the other CRL's among the Top 30, to say nothing of the remaining 481 responding CRLs in the data set. In other words, as reflected by this sample . . .
Internet users are actively establishing connections secured by Globalsign's certificates much more often than any other certificate authority.
In other words, the integrity of Globalsign's certificates matters to Internet users. This makes the importance of intercepting and blocking any attempted abuse of these certificates clear.
As a consequence of the Heartbleed incident, hundreds of thousands of possibly-compromised certificates were revoked and replaced across the Internet. This chart, compiled by Netcraft, shows the regular daily cycle of Internet certificate revocations per hour in the days following the Heartbleed revelation:
Credit: Netcraft Security Blog, April 18th, 2014
It also shows, separately in green, the April 16th to 17th period during which the large web performance and security company, CloudFlare, which was vulnerable to Heartbleed as were so many sites, revoked all of its web certificates issued by Globalsign.
To obtain an estimate of the total number of CloudFlare's certificates revoked by Globalsign, we next turn to the SANS Technology Institute's certificate revocation tracker:
Credit: SANS Technology Institute, Internet Storm Center, SSL CRL Activity monitor
At the time of this writing, the SANS Internet Storm Center is monitoring the CRLs of 81 Internet certificate authorities including Globalsign's. Although this is only a subset of all trusted certificate authorities, it is perfect for our purposes, because their monitoring of fewer means that skewing of the Globalsign data will be reduced.
The two data points of interest occur on April 16th and 17th. Examining the chart, we note that the midpoint of the line connecting the data for the two days falls almost exactly across the 70,000 certs-per-day scale line. This means that the value for April 16th is as far above 70,000 as the value for the 17th is below 70,000. In other words, the average of the two days is slightly below 70,000, so the total for the two days is slightly less than twice that, or 140,000.
Therefore, we can confidently conclude that just short of 140,000 possibly-compromised and now revoked certificates were originally issued by Globalsign, the most heavily used certificate authority (based upon Websense's traffic analysis).
What does Chrome's CRLSet have
to say about that 140,000?
The Globalsign CRL in question, which was monitored by Netcraft and SANS for their reports, is http://crl.globalsign.com/gs/gsorganizationvalg2.crl. This handy page provides the intermediate CA certificate whose private key was used to sign the Organization Validated (OV) certificates whose subsequent revocations were listed in the Globalsign OV CRL. Since this certificate is the “parent” certificate of all those revoked, its signature will introduce a set of revoked serial numbers within Chrome's CRLSet.
We generate any certificate's “signature” with the following concatenation of OpenSSL commands:
openssl x509 -pubkey -noout -in gsov.cer | openssl base64 -d | openssl dgst -sha256 We provide a complete DIY Crypto Kit to allow anyone to duplicate and verify this research and findings. |
This extracts the Subject's Public Key Info (SPKI) record, converts it from base64 into binary, obtains its SHA256 hash and outputs the final result in hexadecimal, which is the format Chrome's CRLSet uses for each of its certificate set's headers. When this command is run against the Globalsign OV Intermediate certificate we obtain this signature:
3aaa93962bc8e2d693083244d68abad6d02c767490ff5dc86e9d10284d47eed7 |
Now we perform a simple text search for that string of characters in the ASCII-converted CRLSet. And it is not found. In other words . . .
Following the Heartbleed incident, none of the approximately 140,000 certificates revoked by Globalsign after the confirmed possibility that they could have been compromised, are present in Chrome's CRLSet.
But . . .
First, a brief bit of history to set the stage . . .
Certificate revocation has not been receiving sufficient attention from the Internet community. And it seemed clear that the news of the “Heartbleed” vulnerability would result in a “tsunami” of revocations of possibly-compromised certificates. So the day after Heartbleed went public, work on GRC's revocation testing “https://revoked.grc.com” website began. It was secured by a certificate that had been revoked before the site was ever made public. Its intent was clearly stated: To allow users to test the “revocation awareness” of their various browsers, devices, and computing platforms.
As expected, the revoked.grc.com site immediately and vividly demonstrated that “the revocation problem” has been a well-kept secret among industry insiders. The Internet's many millions of web surfers have been encouraged to believe that revocation is working and that their web browsers are protecting them from fraudulent web sites. (They're not.)
Because few people understand the reality of today's broken revocation system, Chrome's users assumed that their web browser's willing display of the revoked.grc.com demonstration page must be a bug . . . and reported it as such to the Chromium developers: Issue 362696: Missing warning on revoked certificate. (For additional background, read the thread there. It's enlightening.)
And sure enough, though it took a few days, we started receiving reports that Chrome was now “properly” blocking the special revoked.grc.com test site . . . as expected.
Wanting to take a closer look, we grabbed a copy of Adam's Langley's CRLSet fetch & dump utility from his github repository. (This is how we originally obtained and created the complete CRLSet listing above.) Knowing how the CRLSet worked, we simply searched it for the serial number of the certificate of our now-blocked revoked.grc.com site . . .
0519bb07ad6f78216b1a001fa72223af
. . . assuming, of course, that it would be there somewhere. But it wasn't!
The serial number of GRC's “revoked.grc.com”
certificate wasn't in the CRLSet file anywhere.
How could that be? Chrome had begun blocking access to revoked.grc.com once people obtained the updated CRLSet. Adam Langley himself, answering in the dialog following the Issue 362696 bug report, wrote: “CRLSets push 1576 (I think) contains the revocation for revoked.grc.com. (See chrome://components.)” (Adam's reference to “chrome://components” is a means to have Chrome display the versions of its various components, among them the last retrieved and loaded CRLSet.)
So we had a mystery: We have Adam Langley publicly affirming that “CRLSet push 1576 contains the revocation for revoked.grc.com.” And, sure enough, after loading any CRLSet at or after #1576, Chrome knows not to allow the revoked.grc.com page to display. Yet we know how the CRLSets operate . . . and our serial number is definitely NOT listed anywhere in the file!Was our CA, Digicert, missing the same way Globalsign was?
We decided to see whether our parent's (Digicert) intermediate certificate was present as one of the headers for a set of its revoked certificates. So just as we did with the Globalsign certificate, we ran the OpenSSL incantation on the certificate for the key that signed “revoked.grc.com”:
openssl x509 -pubkey -noout -in digi.cer | openssl base64 -d | openssl dgst -sha256 |
It dutifully dumped out the signature of the certificate for the key that signed “revoked.grc.com”:
7a6ad88298cba65b176ba3a7ab25ee8f90604033da6a98ac07959f566fa3ac54 |
And, once again, we searched the CRLSet for that string . . . and it was not present in the file. That meant that no certificates signed by Digicert's High Assurance CA-3 intermediate certificate were present in the file, including ours.
But it had to be since Chrome had started blocking the site.
A bunch of us in the grc.securitynow newsgroup, who had been collaborating on this research, puzzled over this intractable mystery for a day or two. Finally, one of the brighter bulbs in the box decided to pursue a wild hunch . . . which shocked us all, but instantly solved the mystery:
“revoked.grc.com” was in the CRLSet file header. {
"Version":0,
"ContentType":"CRLSet",
"Sequence":1596,
"DeltaFrom":0,
"NumParents":53,
"BlockedSPKIs":
[
"GvVsmP8EPvkr6/9UzrtN1nolupVsgX8+bdPB5S61hME=",
"PtvZrOY5uhotStBHGHEf2iPoWbL79dE31CQEXnkZ37k=",
"Jdoa1Yu/z7In2HI7GFfUwY57qnQXtPnv+TZrXoafizk=",
"li5LVLuYp+5dX+uWM/mR08MwDpUU2t57DU+CjHlPjoc=", |
By adding one more term to the growing OpenSSL invocation, we convert its hexadecimal signature output into base64 encoding, which is the format used within the “BlockedSPKI” header block:
openssl x509 ‑pubkey ‑noout ‑in rgrc.cer | openssl base64 ‑d | openssl dgst ‑binary ‑sha256 | openssl base64 ‑e |
Rather than using just our certificate's serial number, our entire certificate is fed into this invocation. It extracts the Subject's Public Key Info (SPKI) record and returns a base64 encoding of it's SHA256 hash:
yP3cdcsb27WMB7TqhHKH9iZlndZrwQomrdm1dbOgo40= |
. . . which is exactly what's listed as the last entry in the list of certificates to be blocked by the certificate's public key (not its serial number). You'll note that we also discovered that one other high profile recently revoked domain certificate had also recently been added into the CRLSet header: cloudflarechallenge.com
We have no idea what certs those first three “BlockedSPKI” entries are blocking, and no one (except the Chrome Devs) would ever know what those last two were, if this mystery hadn't been solved. Now we all know.
In order to block the two known high profile revoked sites, which Chrome would have otherwise continued to display, the Chromium developers added a special-case to Chrome's CRLSet. Not because Chrome's CRLSet works . . . but because it doesn't.
Thus, by any reasonable definition of “broken” . . .
You'll recall that near the top of this page we referred to Larry Seltzer's recent ZDNet column with that title. We now have sufficient context, truth and fact to evaluate Larry's conclusion. In his column, Larry quotes an unnamed Google spokesperson saying:
As a Google spokesperson said to me, “...[W]hen we identify a bad cert, we update our revocation list, and Chrome users are automatically protected.”
Somebody, somewhere, has been drinking some serious Google Kool-Aid.
As another point of interest, it's worth noting that in practice, updates to Chrome's CRLSet appear to often take several days and may be unreliable. In the online dialog following the developers' addition of the cloudflarechallenge.com and revoked.grc.com certificates, participants can be seen continuing to report that the sites were not being blocked. They were jumping through hoops, rebooting and restarting their browsers, forcing update refreshes, and doing everything they could think of to get the blocking to work. The actual process seemed to go a bit less smoothly in practice than in theory.
Larry then goes on to quote Adam Langley:
“That's why I claim that online revocation checking is useless — because it doesn't stop attacks. Turning it on does nothing but slow things down. You can tell when something is security theater because you need some absurdly specific situation in order for it to be useful.”
Let's address Adam's last sentence first: If anything has ever been deserving of the title “security theater” it's a certificate revocation detection system that is completely blind to all of the certificates revoked by 4/5ths of the Internet's certificate authorities, at least 98% of revoked certificates overall, doesn't even attempt to deal with the consequences of major Internet events such as Heartbleed, and must have known-revoked certificates, which would be embarrassing to not block, manually added by its maintainers. As we all now know, that describes Chrome's CRLSet to a tee. “Security theater” . . . by design.
What about Adam's first assertion?:
“That's why I claim that online revocation checking is useless — because it doesn't stop attacks. Turning it on does nothing but slow things down.”
Unfortunately, in our view, Adam is allowing the perfect to be the enemy of the good. He is absolutely correct in asserting that using most of today's technology there are situations in which online revocation checking can fail and/or be blocked by a determined and properly positioned attacker. But we believe he's wrong to completely ignore the many situations in which it cannot be blocked and is guaranteed to succeed in protecting its user from online fraud.
One example is Firefox's available option to refuse any secure connection whose certificate cannot be positively verified as valid. Although this can result in Firefox blocking when it should not, it provides extremely good protection to users who have enabled it. (We have always had it enabled and have never received a single false-positive block.) This page is not the place for a detailed pro and con discussion of this issue, but we do have such a page: For a thorough discussion, including the examination of worst-case attack scenarios, please see our OCSP Must-Staple page.
Given Adam's continual assertions (here and here and here) that “revocation checking is useless” one quick example of certificate revocation working valuably and perfectly is in order. For brevity and clarity, and because we haven't discussed the newer OCSP protocol here, we'll look at the case of classic CRL lists which have been supported for decades:
In the entirely feasible real world scenario described above, you'll note that there was absolutely nothing any attacker could do at the time of the attack to block the browser's awareness of the revoked certificate. Thanks to background CRL fetching and caching, the user's device already knew of the certificate's revoked status. And note that since no additional Internet queries and retrievals were required, in this instance nothing was slowed down . . . except for the attacker, who was stopped cold.
It is unfortunate that anyone would advise everyone to disable this long-standing feature of Internet security using the argument that there are some scenarios in which it can be defeated. No one argues that. But that certainly doesn't render it useless, and it appears to be far more generally useful than Google's own CRLSet which presumes to replace it.
The system known as “OCSP Must-Staple” is
the solution . . . and we're close to having it.
With OCSP Must-Staple, the same web server that serves the security certificate also “staples” to that certificate a freshly signed assertion from the issuing certificate authority attesting to the certificate's current status. The web browser doesn't need to make any additional fetches, so no delays or performance penalty, ever. The web server periodically obtains a fresh OCSP assertion from its certificate's authority and provides that to every browser.
“Stapling” already exists and it's supported by all major web servers (Apache, nginx, LiteSpeed, IIS and others):
Firefox has supported stapling since version 26 and Microsoft with Internet Explorer has been well ahead of the curve.
The final piece of the puzzle is the enforcement of stapling, known is must-staple, and we're close to having that too.
Let's get this fixed!
Note: Our next and final page in this series will describe and characterize the current “revocation awareness” state of the industry's various OS and web browser platforms . . . stay tuned!
 | Gibson Research Corporation is owned and operated by Steve Gibson. The contents of this page are Copyright (c) 2024 Gibson Research Corporation. SpinRite, ShieldsUP, NanoProbe, and any other indicated trademarks are registered trademarks of Gibson Research Corporation, Laguna Hills, CA, USA. GRC's web and customer privacy policy. |  |
| Last Edit: May 08, 2014 at 14:07 (3,805.15 days ago) | Viewed 104 times per day |
![[Home]](/mb-home.gif){kind=small}
![[Products]](/mb-products.gif){kind=small}
![[Services]](/mb-services.gif){kind=small}
![[Freeware]](/mb-freeware.gif){kind=small}
![[Research]](/mb-research.gif){kind=small}
![[Other]](/mb-other.gif){kind=small}
cloudflarechallenge.com
"yP3cdcsb27WMB7TqhHKH9iZlndZrwQomrdm1dbOgo40="