The Jepsen analysis of Radix was published today at:
https://jepsen.io/analyses/radix-dlt-1.0-beta.35.1
Some parts of the report are uncomfortable to read.
What are your thoughts on the report and RDX Works response?
3 Likes
Iâm glad they published it as they didnât have to, it shows a great degree of transparency. The wording of the report comes across very damning in parts, but the new Gateway service hopefully resolves most of the issues highlighted. It would be great for this to be verified by Jepsen in future.
I think Radixâs response was good at adding some context around the findings, but some more clarity would be useful, especially around for example the throughput tests which were found to be much less than the 50tps weâre expecting from Olympia.
As for the issue highlighted in the case of a single validator going down, that was all too evident in the past week, so I hope the proposed solution that the team have alluded to will be coming sooner rather than later.
1 Like
Itâs indeed a bit uncomfortable to read. Seems almost as if there was a bit of friction between the team and Jepsen. The report has some completely unnecessary sarcastic undertone to it. I havenât read other reports by them, so donât know if thatâs normal.
Anyway, are we really surprised on the findings? There nothing catastrophic in there, right?
The technical outcome of the report was that the old Archive architecture was not fit for purpose. Radix responded by prioritising its replacement and most of the concerns have been addressed by the 1.1.0 release. Itâs great to see that the team are re-using the test harness that Jepsen used and have recreated tests to address the concerns in the report.
Unfortunately the report was overshadowed by tearing apart some of the marketing claims that are made by Radix. I donât think the tone was necessarily sarcastic - just lots of exclamation marks expressing incredulity at claims made versus observed results. Itâs natural to be defensive but it is really not a good look to be asking to remove unflattering sections of the report and I respect Jepsen hugely for not doing so. The statement that âblockchain/DLT readers would normally understand present-tense English statements like these to be referring to potential future behavior, rather than the presentâ deserves the mockery it received on Hacker News.
I think it took real courage by Radix to engage Jepsen to do a report. You donât hire them for a soft, warm and fuzzy analysis for marketing purposes. Their analysis is the best in class and its findings are closely scrutinised. I genuinely believe that the Radix system will be all the better for having done so.
6 Likes
The request to remove content was specific to using the terms âLiveness Breakâ and âSafety Breakâ in the âIntroductory Abstractâ section of the report until they had been presented later in the report with a comparison between the definition in a DLT world versus the academic definition. Not to remove any findings related to these points.
2 Likes
To Jepsenâs surprise, RDX Works asserted that phenomena such as aborted read, intermediate read, and lost writes do not constitute safety violations (in the DLT sense). RDX Works claims that to describe these errors as safety violations would not be understood by readers from a DLT background; this report is therefore âfactually incorrectâ. On these grounds, RDX Works requested that Jepsen delete any mention of our findings from the abstract of this report.
Jepsen respectfully declines to do so.
Easy to miss the word âabstractâ I suppose. Still not a good look though.
1 Like
The tricky thing about these reports is that you can spin them two ways: âRadix has fixed the issues, the most important stuff workedâ, or âSee? So many issues⌠Told you Radix sucks!â
Something like the supposed double-spend issue doesnât help. In the end, it was, again, an API issue (and no on-ledger double-spend) from what Iâm reading, but itâs really easy to grab that info and create FUD, unfortunately.
Either way, Iâm glad they got someone who tried to break what they built. I like that 
Someone also linked to these âtweetsâ or whatever they are from Jepsenâs site:
6 Likes