doc: programming-2022: Add the latest reviews.
* doc/programming-2022/programming-reviews.txt, doc/programming-2022/programming-response.txt: New files.
This commit is contained in:
parent
44b8ad6f3f
commit
c6b0f7c176
|
@ -0,0 +1,203 @@
|
|||
-*- Org -*-
|
||||
|
||||
Dear reviewers,
|
||||
|
||||
This resubmission is my attempt at taking into account the comments and
|
||||
suggestions that you made, in particular: clarifying the scientific
|
||||
context early on, decoupling the model from the tool, and elaborating on
|
||||
the lessons learned.
|
||||
|
||||
The reviews were detailed, which I appreciated. I modified parts of the
|
||||
paper that appeared to be unclear or left some questions open. Please
|
||||
find below replies to individual questions and comments.
|
||||
|
||||
* Reviewer A
|
||||
|
||||
Thank you for the thorough review and for your comments and suggestions.
|
||||
I tried to address most issues in the document and to answer the many
|
||||
questions below.
|
||||
|
||||
- You write: “when Guix has been out and [sic] about for 3-5 years
|
||||
[…]”. Guix has been out for 10 years, as shown for example by
|
||||
reference 12. I have now made it explicit in the introduction.
|
||||
|
||||
- On the paper title: Whether of not Guix is popular, I believe the
|
||||
title is appropriate as the paper describes a solution designed in
|
||||
that framework. In other words, it’s about how Guix provides
|
||||
building blocks to build a secure supply chain—that is what I meant.
|
||||
|
||||
- You write “We don’t know what Guix is”—while it is nowhere near as
|
||||
popular as Linux (but no free software package is as popular as
|
||||
Linux), Guix is not unknown in its domain; it’s the 8th biggest
|
||||
GNU/Linux distribution according to <https://repology.org/> and
|
||||
there has been a number of peer-reviewed publications about it, some
|
||||
of which are listed at <https://guix.gnu.org/en/publications/>.
|
||||
|
||||
- What are the “IDEAS” (sic) behind Guix? The “Background” section
|
||||
provides answers (Reviewers B and C appear to agree), as well as
|
||||
references 12 and 27. The aggressive tone of this comment is
|
||||
regrettable.
|
||||
|
||||
- I added a sentence in the intro stating the threat model for Guix
|
||||
updates upfront, as suggested.
|
||||
|
||||
- You write: “this seems like a rather naive statement, given that C
|
||||
does not behave the same across systems”. The patronizing tone is,
|
||||
again, uncalled for. Please define “C”; what is C, at run time, if
|
||||
not the C library and kernel?
|
||||
|
||||
I have clarified this by adding: “as evidenced by more than twenty
|
||||
years of experience with the functional deployment model [12, 18]”.
|
||||
This is the extent of the experience these two projects have
|
||||
gathered, in production. If you have references to substantiate
|
||||
that this is a “naive statement”, I can revise it and include the
|
||||
references.
|
||||
|
||||
- You write: “my understanding was that there is no *possibility* of
|
||||
there being any difference”. The paper makes no such claim. The
|
||||
first sentence of the paragraph you’re referring to says that build
|
||||
isolation _helps_ achieve bit-for-bit reproducible builds. I added
|
||||
a couple of sentences discussing non-deterministic build processes.
|
||||
|
||||
- I clarified as suggested the meaning of arrows in the caption of
|
||||
Figure 4.
|
||||
|
||||
- You write: “As the paper already notes, it breaks pull-requests […]
|
||||
Ironically, this workflow was created precisely to support the
|
||||
creation of systems like Linux, which Guix is clearly trying to
|
||||
support”. I am confused as to whether this comment is about systems
|
||||
or about workflows.
|
||||
|
||||
The Linux *project* uses Git for its source code repository, but it
|
||||
does *not* use the pull-request workflow.
|
||||
|
||||
The Linux *kernel* is used by GNU Guix System, the standalone
|
||||
GNU/Linux distribution.
|
||||
|
||||
- About the stolen laptop scenario, you write: “(a) "immediately" is
|
||||
not always realistic; and (b) without my laptop, how do I even track
|
||||
all the projects that I'm contributing to?”; using the same
|
||||
condescending tone as before, you add: “this strikes me as extremely
|
||||
simplistic to the point of being naive.”
|
||||
|
||||
You are right that “immediately” is unrealistic; I changed it to
|
||||
“quickly”, with the understanding that it’s “best effort”.
|
||||
|
||||
How to handle compromised developer keys points at a social issue;
|
||||
it has to do with how the project grants commit rights. Guix has
|
||||
about 40 authorized committers. Like most free software projects,
|
||||
it has a procedure to grant commit access, limiting it to
|
||||
experienced contributors who know each other. In that sense, it’s
|
||||
not unreasonable to expect a committer to notify their peer; but
|
||||
it’s true that it’s part of the social contract among committers,
|
||||
and not something that is “enforced” or “guaranteed”.
|
||||
|
||||
- In the flatmap-stream scenario, the original event-stream developer
|
||||
willingly granted access to someone who turned out to be malicious.
|
||||
It’s of course possible for an authorized developer to be malicious,
|
||||
but as you write, it’s beyond our threat model.
|
||||
|
||||
- As you note, the ‘channels.scm’ configuration file is Scheme code,
|
||||
not data. That file, when it exists, consists of a few lines
|
||||
written by the user (see
|
||||
<https://guix.gnu.org/manual/en/html_node/Specifying-Additional-Channels.html>
|
||||
for the format), making it an unlikely attack vector. In any case,
|
||||
that file is not involved in the checkout authentication mechanism.
|
||||
|
||||
- Regarding downgrade attacks, by virtue of preventing downgrades of
|
||||
Guix itself, we are effectively preventing downgrades of packages.
|
||||
I believe this concern is addressed by the paragraph that starts
|
||||
with “Doing a non-fast-forward update”.
|
||||
|
||||
- You write: “Suppose, say, GitHub were to adopt this idea. You'd hit
|
||||
1000s of commits every few seconds”. GitHub’s source code (if
|
||||
that’s what you mean) is proprietary so we do not know its commit
|
||||
rate. It’s certainly orders of magnitude below what you write
|
||||
though.
|
||||
|
||||
Nixpkgs, which is a package distribution stored in Git but is one of
|
||||
the most active distributions, receives less than 4,000 commits per
|
||||
month, according to
|
||||
<https://github.com/NixOS/nixpkgs/graphs/commit-activity>; Brew (in
|
||||
the same domain) gets less than 300 commits per months according to
|
||||
<https://github.com/Homebrew/brew/graphs/commit-activity>. The
|
||||
Linux kernel (perhaps the most active free software project) gets
|
||||
less than 6,000 commits per month according to
|
||||
<https://github.com/torvalds/linux/graphs/commit-activity>. I added
|
||||
these figures to this paragraph for clarity.
|
||||
|
||||
- You write: “I'd like to know more about the user experience if this
|
||||
scales up.” I expanded the last paragraph of the “Evaluation”
|
||||
section a bit.
|
||||
|
||||
- You write: “A useful point of comparison to what Guix does is signed
|
||||
commits in (say) GitHub”. I believe Section 4 and the relevant
|
||||
paragraph of the “Related Work” section together answer this
|
||||
question, though I’m of course open to suggestions.
|
||||
|
||||
Do GitHub/Lab “verified” labels mean that commits and their parents
|
||||
were made by authorized developers? No. Does GitHub/Lab supports a
|
||||
notion of cryptographic “authorization”? No. Is the project user
|
||||
account information for GitHub/Lab available in-band? No. Are
|
||||
GitHub/Lab’s user OpenPGP keys available in-band? No.
|
||||
|
||||
- Regarding your last question, <https://guix.gnu.org/en/security/>
|
||||
lists the Guix security response team—people who respond to reports
|
||||
of security vulnerabilities in Guix itself or, more commonly, in the
|
||||
packages that Guix provides. Other people currently on the security
|
||||
team did not contribute to this document. They participated in the
|
||||
public review process of the authentication mechanism, visible at
|
||||
<https://issues.guix.gnu.org/22883>, but were not directly involved
|
||||
in its design and implementation, nor in the authoring of this
|
||||
article.
|
||||
|
||||
I am thankful for the thoroughness of your review. However, I regret
|
||||
its aggressive tone, which I find unnecessary and detrimental to our
|
||||
work together.
|
||||
|
||||
* Reviewer B
|
||||
|
||||
Thank you for your comments. I fixed the typos you reported.
|
||||
|
||||
* Reviewer C
|
||||
|
||||
Thank you for your comments and suggestions. Below are answers to some
|
||||
of the comments you made:
|
||||
|
||||
- This new revision of the article is an attempt to more clearly
|
||||
decouple the model from the implementation in Section 4. The
|
||||
introduction clarifies the context by mentioning how signed commits
|
||||
alone are insufficient when it comes to authenticating a complete
|
||||
checkout.
|
||||
|
||||
- The background section goes to great lengths explaining Guix, as you
|
||||
write. I deemed it necessary because the functional deployment
|
||||
model that Guix implements is very different from the mainstream
|
||||
model of “binary distributions”; related work such as TUF and
|
||||
in-toto targets binary distributions and is not applicable as-is to
|
||||
Guix. It is hard to understand why something different from TUF is
|
||||
necessary without first getting an overview of the functional
|
||||
deployment model.
|
||||
|
||||
Furthermore, I wanted to give a holistic view of the supply chain.
|
||||
Related work in this area takes the same approach: for instance,
|
||||
in-toto can be thought of as a generalization of TUF. When looking
|
||||
at the whole supply chain, reproducible builds and “boostrapping”
|
||||
are key aspects.
|
||||
|
||||
- I have clarified in Section 3 why simply checking for commit
|
||||
signatures is of no use. (Sections 4 and 9 already answered that
|
||||
question, but stating it earlier is certainly helpful.)
|
||||
|
||||
- Regarding the pull-request workflow: the proposed mechanism does not
|
||||
prevent its use, but it prevents delegation of merges to the server,
|
||||
as is done when clicking on “Merge” buttons in the web interface. I
|
||||
have clarified it in Section 4.
|
||||
|
||||
- You write that “attackers may learn about the identity of
|
||||
authorizers”. Indeed, authorizations are public, just like the
|
||||
repository and its history is public. I don’t think this has any
|
||||
impact on the effectiveness of the mechanism though. Also, the
|
||||
mechanism is also usable on a private repository; no information is
|
||||
leaked since all the information used by the authentication
|
||||
mechanism is available in the repository itself.
|
|
@ -0,0 +1,291 @@
|
|||
From: "Programming 7.1 HotCRP" <noreply@programming71.hotcrp.com>
|
||||
Subject: [Programming 7.1] Submission #22 "Building a Secure Software Supply Chain..." | Notification
|
||||
To: Ludovic Courtès <ludovic.courtes@inria.fr>
|
||||
Cc: jeremy.gibbons@cs.ox.ac.uk
|
||||
Date: Sun, 3 Apr 2022 17:04:40 +0000 (UTC) (19 hours, 35 minutes ago)
|
||||
Reply-To: robert.hirschfeld@hpi.de
|
||||
|
||||
Dear Ludovic --
|
||||
|
||||
Thank you for your submission to The Art, Science, and Engineering of
|
||||
Programming, Volume 7 Issue 1 (Programming 7.1).
|
||||
|
||||
Title: Building a Secure Software Supply Chain with GNU Guix
|
||||
Authors: Ludovic Courtès (Inria)
|
||||
Site: https://programming71.hotcrp.com/paper/22
|
||||
|
||||
The reviewers would like to see a "major" MINOR REVISION of your
|
||||
submission.
|
||||
|
||||
Let me expand on the "major" MINOR REVISION by directly quoting from
|
||||
the reviewers' discussion of your submission: "This is a major
|
||||
revision on the grounds that there may be a fair amount of editing AND
|
||||
the editing is essential, but a minor revision in that we don't see
|
||||
any problem with it being done, especially because we are not asking
|
||||
for new research or results, nor is there any danger that the author
|
||||
provides us some information that negates a key result."
|
||||
|
||||
Because of the substantial amount of editing expected to be done, I
|
||||
would like to suggest the following:
|
||||
|
||||
#1 - Please submit a revised version of your paper by the deadline for
|
||||
the next issue (June 1, 2022); then your paper will be reviewed for
|
||||
inclusion in Volume 7 Issue 2 of the journal.
|
||||
|
||||
#2 - However, if you manage to finish revisions in time, you may
|
||||
consider submitting your revised paper already by May 1, 2022 so that
|
||||
your submission can still be examined for inclusion in Volume 7 Issue 1.
|
||||
|
||||
For both cases: Supplementing the revised paper, please also submit
|
||||
(1) a cover letter that outlines your changes and how you considered
|
||||
the points made by the reviewers and (2) a latex diff (using
|
||||
https://ctan.org/pkg/latexdiff) or a similarly annotated copy that
|
||||
clearly indicates the changes you made.
|
||||
|
||||
In addition to the detailed reviews below please pay attention to the
|
||||
author-visible note, which summarizes the reviewers' discussion and
|
||||
contains further advice on how to improve your submission.
|
||||
|
||||
I hope you find the reviews helpful.
|
||||
|
||||
Note: Starting with Volume 7, The Programming Journal offers Artifact
|
||||
Evaluation (AE). You indicated during submission that you do not
|
||||
intent to submit an artifact, but since you might have reconsidered,
|
||||
our AE Chairs will contact you about that in a separate e-mail.
|
||||
|
||||
Please do not hesitate to ask if you have any questions.
|
||||
|
||||
Best regards,
|
||||
Robert Hirschfeld
|
||||
|
||||
--
|
||||
Robert Hirschfeld
|
||||
The Programming Journal, Associate Editor, Volume 7
|
||||
robert.hirschfeld@hpi.uni-potsdam.de
|
||||
https://www.hpi.uni-potsdam.de/swa/
|
||||
|
||||
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
|
||||
|
||||
Review #22A
|
||||
===========================================================================
|
||||
|
||||
Paper summary (for authors)
|
||||
---------------------------
|
||||
This paper is about the role of package managers in the software supply chain problem. Specifically, it (rightly) notes that package managers are both a source of problems in supply chain security and can play a useful role in mitigating these problems. It describes a new Nix-like package manager, GNU Guix, for source distribution. The main contribution is the use of OpenPGP signatures as part of the repository to enable a kind of chain-of-trust verification. The paper talks about the design and some consequences for monitoring and preventing certain kinds of attacks. Guix seems to be in production.
|
||||
|
||||
Assessment and comments (for authors)
|
||||
-------------------------------------
|
||||
[The following is my review as originally written. I feel it's useful for authors to be able to see a reviewer's state of mind at the time of authoring. At the end, I have provided updated comments listing what I would like to see done in a revision.]
|
||||
|
||||
# Original Review
|
||||
|
||||
Overall I am happy to read about a paper in this space. We have not paid enough attention to the problems of supply-chains, and it seems necessary to rethink their security architecture. Guix seems to be focused on doing that in a specific space. The paper is pretty easy reading and the system seems to have "legs". I liked all that about it.
|
||||
|
||||
I am, however, not convinced that this paper is ready for publication. The paper is submitted in the Engineering track, which means it provides "knowledge and technical skills acquired through designing and building large systems and through calculated application of principles in building those systems. Examples include measurements of artifacts’ properties, development processes and tools, and quality assurance methods." That doesn't really describe the paper as submitted. Now, I'll admit that the <Programming> classification system can be very difficult for some papers; however, this paper doesn't seem to quite fit the other tracks, either. That is not inherently a problem — a paper can still be great while falling outside the lines, and <Programming> of all venues should be open to that — but in this case I think it points to some difficulty with the paper, in that it's difficult to know what the real contribution is. The core scientific idea is not very sophisticated but I think also perhaps problematic (as discussed below). The evaluation is too limited. There aren't enough lessons learned.
|
||||
|
||||
What I would like to see in a revision is a clear focus on deepening the contribution and reducing some of the less important details. The paper sometimes reads like a user manual. These parts can be removed, I think, to make room for teaching us more of a lesson about what you have seen and learned. (I am also sensitive to the possibility that perhaps it is just a bit early to be writing this paper: maybe when Guix has been out and about for 3-5 years, we may learn enough Engineering lessons, but not quite before then.)
|
||||
|
||||
Below I provide various comments noted while reading the paper, so they are in the order in which they show up in the prose:
|
||||
|
||||
The paper title is poorly chosen. We don't know what Guix *is*. If it were really well-known ("Building a Secure Software Supply Chain with Linux") that would be okay. Otherwise the title is unhelpful and frustrating. Also, this is a research publication. Therefore, the title should as much as possible focus on IDEAS. What is the big IDEA behind Guix, that one can read and perhaps replicate in a different tool? That's what the title should emphasize.
|
||||
|
||||
I'm curious why the paper doesn't use the structured abstract asked for by the journal. Did the author find it too limiting?
|
||||
|
||||
"Guix is a software deployment tool" - "Guix is first and foremost source code" seems like a type error. Maybe the paper means to say that it's first and foremost *for* source code? In general, I found several places where the paper could benefit from a bit of proofreading.
|
||||
|
||||
*What is the threat model?* It takes a long time to find out — only on pg 7 do we get what seems to be one. This needs to be up front and center in a paper with "secure" in the title.
|
||||
|
||||
"Guix packages stand alone: they provide all the user-land software they need, down to the C library; this guarantees they behave the same on any system." — this seems like a rather naive statement, given that C does not behave the same across systems.
|
||||
|
||||
"For example, the command guix build -- -check hello rebuilds the hello package locally and prints an error if the build result differs from that already available." — I don't understand this. Maybe I don't know what "already available" refers to; the phrase is too vague to denote unambiguously. But up to this point in the paper, my understanding was that there is no *possibility* of there being any difference — that's the whole point behind a functional build. So one could almost ask, philosophically, why this command even needs to exist. I would have liked a little more discussion about this. (I understand that *should be* doesn't mean *is*; this is presumably also why the `challenge` command exists. But I think the paper would be enhanced by listing the ways in which the "should be" *fails* to be "is", necessitating these operations.)
|
||||
|
||||
In figure 4, please explain what the *direction* of the arrow represents. It's equally reasonable to view the tail of the arrow as the older commit and the head as newer, which left me very confused, until I realize you mean it the other way around. (Never assume the meaning of a picture is obvious! It may be…but it could be obviously *different*.)
|
||||
|
||||
The authorization process, as the paper says, is indeed simple to state and enforce. It also strikes me as far too weak.
|
||||
|
||||
1. As the paper already notes, it breaks pull-requests; the paper waves this away as not being in "the needs of the Guix project". Ironically, this workflow was created precisely to support the creation of systems like Linux, which Guix is clearly trying to support (given the extensive discussion of the trusting-trust paper). Without a clear sense of "the needs" (i.e., the *limitations*) of Guix, this comment cannot really be judged and hence entertained.
|
||||
|
||||
2. As the paper again notes, anyone who has commit access has full power over the repo *including decommissioning other users*. The paper notes the case of a compromised account like a "stolen laptop", "they can notify other committers so they immediately remove the key, thereby preventing it from being used to push new commits". However, (a) "immediately" is not always realistic; and (b) without my laptop, how do I even track all the projects that I'm contributing to? I have dozens and dozens of repos on my machine; when I move to a new machine, other than 2-3 key repos, the rest I only remember when I need to touch them; the others I basically forget about. So this strikes me as extremely simplistic to the point of being naive. (There's also the question of how I find these other developers to get in touch, etc.) This is not an unrealistic case: e.g., the Copay wallet attack was based on the flatmap-stream code that the developer was basically no longer maintaining, i.e., was effectively in the forgotten category.
|
||||
|
||||
3. Furthermore, there's the more insidious case of a developer not being aware they are under attack. Going back to a flatmap-stream-like scenario, the developer may not even notice that commits are being made (in their name) to a repo they are no longer watching. In that case, the real developer has no reason to notify anyone of anything.
|
||||
|
||||
Admittedly, this last situation somewhat falls outside the scope of the threat model of the paper. However, that does not mean these aren't real issues. And the real problem here is not that I'm falling outside the threat model, but rather that the permission model seems far too simplistic. For one whose whole foundation is security, it seems to need to be more sophisticated. I really needed much more discussion on this, and defense of the design, since this is the core of the paper.
|
||||
|
||||
"configuration file, which contains a declarative Scheme code snippet" — this (and the .scm name) suggests that the configuration file is itself evaluated, i.e., it's not a passive s-expression (like the authorizations file). What are the permissions of this program? Can this not be used to create harm? Does it actually use raw Scheme, or a variant like Rees's Security Kernel based on Scheme? (Apparently not, it uses Guile. Which leaves this issue very open. Or is it not a problem because it's run in a container? Either way, please address this threat and its handling explicitly.)
|
||||
|
||||
Downgrade attacks: This seems to address the case of Guix itself being upgraded, but what about downgrade attacks on individual packages? I think the same logic holds, but I find it curious that the paper does not talk about this.
|
||||
|
||||
Section 8.3 talks about the performance of authentication and its sufficiency against the current commit frequency. However, this is not very satisfying. It may be that this is sufficient *for the number of current users*. But the general problem here is much broader than Guix. Suppose, say, GitHub were to adopt this idea. You'd hit 1000s of commits every few seconds, I expect. How parallelizable is this? How costly is this? There needs to be some discussion here about deploying this idea at (large) scale.
|
||||
|
||||
For an Engineering paper, I would have liked to hear a lot more about what was learned from building and deploying this system. Right now this is mostly limited to the evaluation section (8.5), other than a tiny bit of prose outside it about, e.g., performance of authentication. I would have liked to read more detail here, and maybe some hard numbers. I would also have liked fewer fluffy statements like "been adopting authentication support without hassle" (especially since nobody can say "without hassle" and OpenPGP in the same breath — as even the paper notes, referring to the "complexity" of OpenPGP in 8.1).
|
||||
|
||||
Related to my question in 8.3, I'd like to know more about the user experience if this scales up. We can certainly agree that using OpenPGP will not be "without hassle" when you consider the vast number of non-technical people who use a system like GitHub. Setting that aside, what other things will they run into? For instance, artists use GitHub to share Figma designs with each other. What happens when they try to use this system? How simple and transparent will it be? What is the user experience?
|
||||
|
||||
Finally, I am unclear from the paper on the exact functioning of Guix in an important way. A useful point of comparison to what Guix does is signed commits in (say) GitHub. The related work engages with this, but the difference is not as persuasive as I would like. Moreover, given the focus on source, I would have thought that what we would get in Guix is not just an analog to that, but also something like a hash chain (say a Merkle DAG) of all the source that led to this point. That way I can feel confident that the source I'm getting is in fact the source I was expecting to get. Is Guix doing that? If not, why not?
|
||||
|
||||
As an aside, I'm curious — this does not need to be in the paper — why the Guix security page [https://guix.gnu.org/en/security/] lists three names but only one is an author of this paper.
|
||||
|
||||
# Comments After Discussion
|
||||
|
||||
In general, I would like the author to follow the usual journal response process. Depending on the author's experience, if they are not used to writing these, I encourage consultation with the journal chair. I would be happy to provide feedback on drafts, too.
|
||||
|
||||
After discussion with Reviewer B, I am much more positively disposed to the paper. Still, I feel the presentation is not where I would it to be. Hopefully the only thing that needs fixing is that, not the content proper (but improving the presentation may yet lay bare some content issues…).
|
||||
|
||||
While I would like to see as many things as possible in my review responded to and/or addressed and fixed, the issues I most care about that will keep me from being in favor of the paper are:
|
||||
|
||||
- Make the threat model explicit early on. It's very difficult to evaluate a security-related paper without a clear sense of the threat model.
|
||||
|
||||
- Clarify issues like "guarantees same behavior", "build result differs", etc.
|
||||
|
||||
- Respond to and as best as possible, address the concerns about the authentication model.
|
||||
|
||||
- Explain the nature of the configuration file and, if it is executable, make clear why that is not a problem.
|
||||
|
||||
- Discuss downgrade attacks.
|
||||
|
||||
- Explain authentication at scale.
|
||||
|
||||
- In general, provide more data on actual use, given that that is the main thing that is attractive about the work that this paper is about.
|
||||
|
||||
- Address loose and sloppy remarks like "without hassle": either justify them with hard evidence, or remote them (and indeed, acknowledge those hassles explicitly).
|
||||
|
||||
I look forward to the revision.
|
||||
|
||||
|
||||
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
|
||||
|
||||
|
||||
Review #22B
|
||||
===========================================================================
|
||||
|
||||
Paper summary (for authors)
|
||||
---------------------------
|
||||
This paper provides an overview of how Guix supports reproducible and
|
||||
accountable builds of an operating environment from source, and then
|
||||
focuses on a particular piece of the system: how trust is established
|
||||
for the initial sources and their evolution. Sources are drawn from
|
||||
Git, and the key to establishing trust is to associate a signature
|
||||
with each commit. Trusted signatures are also stored in the
|
||||
repository, which means that a single mechanism can be used for
|
||||
authenticating changes and authors of the changes.
|
||||
|
||||
Assessment and comments (for authors)
|
||||
-------------------------------------
|
||||
The key idea seems simple, and as the paper explains, it's not
|
||||
specific to Guix. The idea applies to any context where a Git-style
|
||||
content-addressable graph is used to communicate a history of changes
|
||||
from an agreed-on starting point. The paper addresses both the
|
||||
mechanism and the social process that is used to establish the
|
||||
starting points. At the same time, the fact that the idea is
|
||||
implemented in Guix shows the relevance of the problem and provides
|
||||
substantial evidence that it works. Although the presentation is
|
||||
clear, I am nowhere near expert enough in security and trust to vet
|
||||
the idea in this paper --- but, very much in spirit of the idea here,
|
||||
I feel inclined to trust the protocol because it has been implemented
|
||||
and vetted by others who care.
|
||||
|
||||
The paper is well written, and I am at a loss to recommend
|
||||
improvements to the presentation. A small, self-contained model of
|
||||
`guix git authenticate` might be nice, but I don't think it's
|
||||
necessary.
|
||||
|
||||
Typos:
|
||||
|
||||
page 8: "computer" -> "computed"
|
||||
|
||||
page 15: "Git 'protocols', via" - comma and italics intended?
|
||||
|
||||
|
||||
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
|
||||
|
||||
|
||||
Review #22C
|
||||
===========================================================================
|
||||
|
||||
Paper summary (for authors)
|
||||
---------------------------
|
||||
The paper describes an authentication mechanism built into GNU Guix to protect against attacks that focus on git access like the introduction of malicious changes and downgrade attacks. The mechanism has been deployed in production two years ago, and the paper briefly reports on experiences on the provided support.
|
||||
|
||||
Assessment and comments (for authors)
|
||||
-------------------------------------
|
||||
Strengths
|
||||
- Supply chain attacks, in particular, git attacks, is a relevant
|
||||
topic - Proposed mechanism seems sound and it is used in production
|
||||
|
||||
Weaknesses
|
||||
- I liked the tutorial-like style of the paper, but the paper is not written in a scientific way that contextualizes the work
|
||||
- It is unclear the relevance of the contributions to state-of-the-art
|
||||
in the verification of git updates
|
||||
Detailed Comments:
|
||||
|
||||
- The introduction would best describe a conceptual research question
|
||||
rather than a research question focused on a concrete tool. As far
|
||||
as I understand, the work focuses on studying mechanisms for
|
||||
verifiable git updates to prevent unauthorized modifications of
|
||||
source code. Even if the solution is integrated into GNU Guix, it
|
||||
may help to first describe the issues the work aims to tackle
|
||||
decoupled from GNU Guix. Having the description of the solution
|
||||
directly explained in the context of the tool makes it difficult to
|
||||
place the work in a broader scientific context. Since the problems
|
||||
and solution are described for GNU Guix, it is also difficult to
|
||||
assess how relevant the work is beyond GNU Guix.
|
||||
- The background section focuses too much on describing GNU Guix. The
|
||||
paper would be stronger by reducing details that are not needed for
|
||||
understanding the contributions of the authentication mechanism
|
||||
(e.g. section 2.3 could be removed) and adding a more detailed
|
||||
explanation on the kind of attacks the work aims to prevent (this is
|
||||
now briefly explained in two paragraphs in section 3). A more
|
||||
detailed description may also clarify how the attacks can be
|
||||
performed when the attacker is not impersonating an authorized
|
||||
developer. The article does not focus on protecting “against cases
|
||||
where an attacker is able to impersonate an authorized developer”,
|
||||
but how is it possible otherwise to conduct an introduction of
|
||||
malicious changes (an attack targeted by the work)?
|
||||
- Section 3 could be improved if it describes what measurements exist
|
||||
today to deal with the targeted attacks so that the paper more
|
||||
clearly conveys shortcomings in current work that this solution can
|
||||
overcome. For example, as mentioned in section 9, it is possible in
|
||||
git to cryptographically sign a commit using a GPG key. However,
|
||||
this is criticized as being a limited form of verification but it is
|
||||
unclear why and which of the targeted attacks could then not be
|
||||
prevented with such support for verified git updates.
|
||||
- The fact that the designed tool supports off-line checkout authentication is said to be a novelty but, unfortunately, the paper does not properly contextualize the work and clarifies the relevance of such off-line support.
|
||||
|
||||
- As far as I understand, the devised authentication mechanism does
|
||||
not work well with the pull-request-style way of working popular in
|
||||
git repositories, which may limit the applicability of the approach
|
||||
beyond GNU Guix.
|
||||
- The fact that .guix-authorizations is just a file stored in the
|
||||
repository, seems to be a weak point of the solution as attackers
|
||||
may learn about the identity of authorizers.
|
||||
Minor typographic comments:
|
||||
- Page 3: its salient feature are => is
|
||||
- Page 5, Niewenzhuizen et al. misses the actual citation.
|
||||
|
||||
Comments after PC discussion
|
||||
|
||||
Overall, I am positive about the paper but I don’t think it is ready yet for publication in its current form. I believe the paper needs some reorganization of the material and additions as mentioned in the reviews. Besides the issues mentioned in the list by reviewer #a, I would also like to see the following things incorporated into the paper:
|
||||
|
||||
-revise the scientific context early on so that readers can understand
|
||||
what limitations current git authentication suffers from, which
|
||||
motivates this work.
|
||||
-decouple the model from the tool in the paper. The abstract does explain that the main contribution is a model and a tool to authenticate new Git revisions, but those two aspects are coupled in the main body, which makes it difficult to assess the relevance of the “model”.
|
||||
|
||||
-elaborate on the lesson learned from having the solution in production; there is not enough attention given to that aspected in the submitted version while it is a strength of the work.
|
||||
|
||||
|
||||
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
|
||||
|
||||
Comment @A1
|
||||
---------------------------------------------------------------------------
|
||||
Overall, we are positive about the paper but don’t think it is ready yet for publication in its current form. We believe the paper needs some reorganization of the material and additions as mentioned in the reviews. Besides the issues mentioned in the list by reviewer #a, we would also like to see the following things incorporated into the paper:
|
||||
|
||||
- Revise the scientific context early on so that readers can understand what limitations current git authentication suffers from, which motivates this work.
|
||||
|
||||
- Decouple the model from the tool in the paper. The abstract does explain that the main contribution is a model and a tool to authenticate new Git revisions, but those two aspects are coupled in the main body, which makes it difficult to assess the relevance of the “model”.
|
||||
|
||||
- Elaborate on the lesson learned from having the solution in production; there is not enough attention given to that aspected in the submitted version while it is a strength of the work.
|
||||
|
||||
|
||||
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
|
||||
|
Loading…
Reference in New Issue