guix/data-service
2
0
Fork 0
mirror of git://git.savannah.gnu.org/guix/data-service.git synced 2023-12-14 03:23:03 +01:00
Code Issues Releases Wiki Activity
data-service/guix-data-service/web/controller.scm

1035 lines
45 KiB
Scheme
Raw Normal View History

Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
;;; Guix Data Service -- Information about Guix over time
;;; Copyright © 2016, 2017, 2018, 2019 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2019 Christopher Baines <mail@cbaines.net>
;;;
;;; This program is free software: you can redistribute it and/or
;;; modify it under the terms of the GNU Affero General Public License
;;; as published by the Free Software Foundation, either version 3 of
;;; the License, or (at your option) any later version.
;;;
;;; This program is distributed in the hope that it will be useful,
;;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
;;; Affero General Public License for more details.
;;;
;;; You should have received a copy of the GNU Affero General Public
;;; License along with this program. If not, see
;;; <http://www.gnu.org/licenses/>.
(define-module (guix-data-service web controller)
#:use-module (ice-9 match)
Add a packages comparison page The primary use I have in mind for this is producing a list of strings suitable for building a limited Cuirass job with.
2019-02-26 00:44:32 +01:00
#:use-module (ice-9 vlist)
Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
#:use-module (ice-9 pretty-print)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-11)
#:use-module (srfi srfi-26)
#:use-module (web request)
#:use-module (web uri)
Allow specifying the fields on the packages page This is mostly for the JSON output, as it allows much more information to be included.
2019-05-16 23:28:16 +02:00
#:use-module (texinfo)
#:use-module (texinfo html)
#:use-module (texinfo plain-text)
Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
#:use-module (squee)
Allow specifying the fields on the packages page This is mostly for the JSON output, as it allows much more information to be included.
2019-05-16 23:28:16 +02:00
#:use-module (json)
Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
#:use-module (guix-data-service comparison)
Extract the database connection handling from the server Previously, one of the first things that happened when responding to a request was a database connection was made, even when serving the CSS. This is unnecessary, so move the database connection handling in to the controller. Also, to allow for separating it out from the assets, separate the assets out from the parts of the controller that require a database connection.
2019-06-06 21:39:06 +02:00
#:use-module (guix-data-service database)
Start to handle information about Git branches Add some new pages /branches and /branch/... as well as a new git_branches table. Also extend the email processing to enter the branch information in to the database.
2019-05-05 21:06:28 +02:00
#:use-module (guix-data-service model git-branch)
Switch to storing Git repositories in a table Rather than just storing the URL in the guix_revisions and load_new_guix_revision_jobs tables. This will help when storing more information like tags and branches in the future.
2019-05-05 14:35:48 +02:00
#:use-module (guix-data-service model git-repository)
Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
#:use-module (guix-data-service model guix-revision)
Add a few new pages For showing more information about builds, revisions and derivations.
2019-03-06 23:59:27 +01:00
#:use-module (guix-data-service model package)
Many changes A large proportion of these changes relate to changing the way packages relate to derivations. Previously, a package at a given revision had a single derivation. This was OK, but didn't account for multiple architectures. Therefore, these changes mean that a package has multiple derivations, depending on the system of the derivation, and the target system. There are multiple changes, small and large to the web interface as well. More pages link to each other, and the visual display has been improved somewhat.
2019-03-11 23:11:14 +01:00
#:use-module (guix-data-service model package-derivation)
#:use-module (guix-data-service model package-metadata)
Start to visualise derivations
2019-03-07 09:43:16 +01:00
#:use-module (guix-data-service model derivation)
Improve the compare derivations page Add support for filtering the results, and add the system and target to the output.
2019-03-17 23:44:09 +01:00
#:use-module (guix-data-service model build-status)
Add a few new pages For showing more information about builds, revisions and derivations.
2019-03-06 23:59:27 +01:00
#:use-module (guix-data-service model build)
Display counts of lint warnings on the revision page
2019-08-31 13:42:54 +02:00
#:use-module (guix-data-service model lint-checker)
Add a page to show the lint warnings for a revision
2019-09-01 13:59:45 +02:00
#:use-module (guix-data-service model lint-warning)
Show when revisions are missing, but queued for processing
2019-02-24 17:47:29 +01:00
#:use-module (guix-data-service jobs load-new-guix-revision)
Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
#:use-module (guix-data-service web render)
Allow specifying the fields on the packages page This is mostly for the JSON output, as it allows much more information to be included.
2019-05-16 23:28:16 +02:00
#:use-module (guix-data-service web sxml)
Add a new query-parameters module The query parameters feed in to the results shown, but also forms on pages. Validation is important to avoid errors and security issues, but it's also important to provide appropriate feedback to the user. This module provides some utilities and structure around handling query parameters.
2019-05-11 17:44:17 +02:00
#:use-module (guix-data-service web query-parameters)
Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
#:use-module (guix-data-service web util)
#:use-module (guix-data-service web view html)
#:export (controller))
Cache the pages which don't really change for a day
2019-05-18 21:25:34 +02:00
(define cache-control-default-max-age
(* 60 60 24)) ; One day
(define http-headers-for-unchanging-content
`((cache-control
. (public
(max-age . ,cache-control-default-max-age)))))
Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
(define-syntax-rule (-> target functions ...)
(fold (lambda (f val) (and=> val f))
target
(list functions ...)))
(define (render-with-error-handling page message)
(apply render-html (page))
;; (catch #t
;; (lambda ()
;; (receive (sxml headers)
;; (pretty-print (page))
;; (render-html sxml headers)))
;; (lambda (key . args)
;; (format #t "ERROR: ~a ~a\n"
;; key args)
;; (render-html (error-page message))))
)
Improve the compare derivations page Add support for filtering the results, and add the system and target to the output.
2019-03-17 23:44:09 +01:00
(define (assoc-ref-multiple alist key)
(filter-map
(match-lambda
((k . value)
(and (string=? k key)
value)))
alist))
(define (with-base-and-target-commits query-parameters conn f)
(let* ((base-commit (assoc-ref query-parameters "base_commit"))
(target-commit (assoc-ref query-parameters "target_commit")))
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
(f base-commit
(commit->revision-id conn base-commit)
target-commit
(commit->revision-id conn target-commit))))
Make some pages around revisions more generic So that they can also be used for the /branch/foo/latest-processed-revision pages. The content is the same, but the title, link, and some of the links on the page are different.
2019-06-16 11:27:14 +02:00
(define* (render-view-revision mime-types
conn
commit-hash
#:key path-base
(header-text
`("Revision " (samp ,commit-hash))))
Add a JSON variant of the revision page
2019-05-12 10:51:00 +02:00
(let ((packages-count
(count-packages-in-revision conn commit-hash))
Show the branches for a revision
2019-05-18 13:36:03 +02:00
(git-repositories-and-branches
(git-branches-with-repository-details-for-commit conn commit-hash))
Add a JSON variant of the revision page
2019-05-12 10:51:00 +02:00
(derivations-counts
Display the jobs related to a revision on the revision page
2019-07-22 21:00:11 +02:00
(count-packages-derivations-in-revision conn commit-hash))
(jobs-and-events
Display counts of lint warnings on the revision page
2019-08-31 13:42:54 +02:00
(select-jobs-and-events-for-commit conn commit-hash))
(lint-warning-counts
(lint-warning-count-by-lint-checker-for-revision conn commit-hash)))
Add a JSON variant of the revision page
2019-05-12 10:51:00 +02:00
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
`((packages_count . ,(caar packages-count))
(derivations_counts . ,(list->vector
(map (match-lambda
((system target derivation_count)
`((system . ,system)
(target . ,target)
(derivation_count . ,derivation_count))))
Display counts of lint warnings on the revision page
2019-08-31 13:42:54 +02:00
derivations-counts)))
(lint_warning_counts . ,(map (match-lambda
((name description network-dependent count)
`(,name . ((description . ,description)
(network_dependent . ,(string=? network-dependent "t"))
(count . ,(string->number count))))))
lint-warning-counts)))
Cache the pages which don't really change for a day
2019-05-18 21:25:34 +02:00
#:extra-headers http-headers-for-unchanging-content))
Add a JSON variant of the revision page
2019-05-12 10:51:00 +02:00
(else
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (view-revision
commit-hash
packages-count
git-repositories-and-branches
Make some pages around revisions more generic So that they can also be used for the /branch/foo/latest-processed-revision pages. The content is the same, but the title, link, and some of the links on the page are different.
2019-06-16 11:27:14 +02:00
derivations-counts
Display the jobs related to a revision on the revision page
2019-07-22 21:00:11 +02:00
jobs-and-events
Display counts of lint warnings on the revision page
2019-08-31 13:42:54 +02:00
lint-warning-counts
Make some pages around revisions more generic So that they can also be used for the /branch/foo/latest-processed-revision pages. The content is the same, but the title, link, and some of the links on the page are different.
2019-06-16 11:27:14 +02:00
#:path-base path-base
#:header-text header-text)
Cache the pages which don't really change for a day
2019-05-18 21:25:34 +02:00
#:extra-headers http-headers-for-unchanging-content)))))
Add a JSON variant of the revision page
2019-05-12 10:51:00 +02:00
Allow specifying the fields on the packages page This is mostly for the JSON output, as it allows much more information to be included.
2019-05-16 23:28:16 +02:00
(define (texinfo->variants-alist s)
(let ((stexi (texi-fragment->stexi s)))
`((source . ,s)
(html . ,(with-output-to-string
(lambda ()
(sxml->html (stexi->shtml stexi)))))
(plain . ,(stexi->plain-text stexi)))))
Add missing revision handling As previously the pages would render, but with no data.
2019-05-19 23:31:07 +02:00
(define (render-unknown-revision mime-types conn commit-hash)
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
'((unknown_commit . ,commit-hash))
#:code 404))
(else
(render-html
#:code 404
#:sxml (unknown-revision
commit-hash
(select-job-for-commit
Improve the Unknown revision page Make it similar to the revision page, as some of the information, like the link to the relevant CGit instance might be available.
2019-06-19 23:03:55 +02:00
conn commit-hash)
Display the jobs related to a revision on the revision page
2019-07-22 21:00:11 +02:00
(git-branches-with-repository-details-for-commit conn commit-hash)
(select-jobs-and-events-for-commit conn commit-hash))))))
Add missing revision handling As previously the pages would render, but with no data.
2019-05-19 23:31:07 +02:00
Make some pages around revisions more generic So that they can also be used for the /branch/foo/latest-processed-revision pages. The content is the same, but the title, link, and some of the links on the page are different.
2019-06-16 11:27:14 +02:00
(define* (render-revision-packages mime-types
conn
commit-hash
query-parameters
#:key
(path-base "/revision/")
(header-text
`("Revision " (samp ,commit-hash)))
(header-link
(string-append "/revision/" commit-hash)))
Add a textual search to the packages page
2019-05-12 22:17:08 +02:00
(if (any-invalid-query-parameters? query-parameters)
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
`((error . "invalid query"))))
(else
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (view-revision-packages commit-hash
query-parameters
'()
'()
Add some missing arguments to view-revision-packages
2019-07-07 17:42:00 +02:00
#f
#:path-base path-base
#:header-text header-text
#:header-link header-link))))
Add a textual search to the packages page
2019-05-12 22:17:08 +02:00
(let* ((search-query (assq-ref query-parameters 'search_query))
Add an all results option to the packages page
2019-05-19 22:28:01 +02:00
(limit-results (or (assq-ref query-parameters 'limit_results)
99999)) ; TODO There shouldn't be a limit
Allow specifying the fields on the packages page This is mostly for the JSON output, as it allows much more information to be included.
2019-05-16 23:28:16 +02:00
(fields (assq-ref query-parameters 'field))
Add a textual search to the packages page
2019-05-12 22:17:08 +02:00
(packages
(if search-query
(search-packages-in-revision
Add some basic pagination to the packages page Lower powered devices will have problems displaying all ~9000+ packages, so return a smaller number by default.
2019-05-12 20:51:44 +02:00
conn
commit-hash
Add a textual search to the packages page
2019-05-12 22:17:08 +02:00
search-query
#:limit-results limit-results)
(select-packages-in-revision
conn
commit-hash
#:limit-results limit-results
#:after-name (assq-ref query-parameters 'after_name))))
Allow specifying the fields on the packages page This is mostly for the JSON output, as it allows much more information to be included.
2019-05-16 23:28:16 +02:00
(git-repositories
(git-repositories-containing-commit conn
commit-hash))
Add a textual search to the packages page
2019-05-12 22:17:08 +02:00
(show-next-page?
(and (not search-query)
(>= (length packages)
limit-results))))
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
Allow specifying the fields on the packages page This is mostly for the JSON output, as it allows much more information to be included.
2019-05-16 23:28:16 +02:00
`((revision
. ((commit . ,commit-hash)))
(packages
. ,(list->vector
(map (match-lambda
((name version synopsis description home-page
location-file location-line
location-column-number licenses)
`((name . ,name)
,@(if (member "version" fields)
`((version . ,version))
'())
,@(if (member "synopsis" fields)
`((synopsis
. ,(texinfo->variants-alist synopsis)))
'())
,@(if (member "description" fields)
`((description
. ,(texinfo->variants-alist description)))
'())
,@(if (member "home-page" fields)
`((home-page . ,home-page))
'())
,@(if (member "location" fields)
`((location
. ((file . ,location-file)
(line . ,location-line)
(column . ,location-column-number))))
'())
,@(if (member "licenses" fields)
`((licenses
. ,(if (string-null? licenses)
#()
(json-string->scm licenses))))
'()))))
Cache the pages which don't really change for a day
2019-05-18 21:25:34 +02:00
packages))))
#:extra-headers http-headers-for-unchanging-content))
Add a textual search to the packages page
2019-05-12 22:17:08 +02:00
(else
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (view-revision-packages commit-hash
query-parameters
packages
git-repositories
Make some pages around revisions more generic So that they can also be used for the /branch/foo/latest-processed-revision pages. The content is the same, but the title, link, and some of the links on the page are different.
2019-06-16 11:27:14 +02:00
show-next-page?
#:path-base path-base
#:header-text header-text
#:header-link header-link)
Cache the pages which don't really change for a day
2019-05-18 21:25:34 +02:00
#:extra-headers http-headers-for-unchanging-content))))))
Add a JSON variant of the packages page
2019-05-12 10:56:24 +02:00
Make some pages around revisions more generic So that they can also be used for the /branch/foo/latest-processed-revision pages. The content is the same, but the title, link, and some of the links on the page are different.
2019-06-16 11:27:14 +02:00
(define* (render-revision-package mime-types
conn
commit-hash
name
version
#:key
(header-text
`("Revision "
(samp ,commit-hash)))
(header-link
(string-append
"/revision/" commit-hash)))
Add initial JSON output for the package page
2019-05-12 11:11:14 +02:00
(let ((metadata
(select-package-metadata-by-revision-name-and-version
conn
commit-hash
name
version))
(derivations
(select-derivations-by-revision-name-and-version
conn
commit-hash
name
Store and display the location of packages Store the location a package can be found at, and display this on the package page. If available, link off to the git repository containing the package.
2019-05-13 22:02:53 +02:00
version))
(git-repositories
(git-repositories-containing-commit conn
Add lint warnings to the package page
2019-09-01 19:26:46 +02:00
commit-hash))
(lint-warnings
(select-lint-warnings-by-revision-package-name-and-version
conn
commit-hash
name
version)))
Add initial JSON output for the package page
2019-05-12 11:11:14 +02:00
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
`((name . ,name)
(version . ,version)
,@(match metadata
(((synopsis description home-page))
`((synopsis . ,synopsis)
(description . ,description)
(home-page . ,home-page))))
(derivations . ,(list->vector
(map (match-lambda
((system target file-name status)
`((system . ,system)
(target . ,target)
(derivation . ,file-name))))
Cache the pages which don't really change for a day
2019-05-18 21:25:34 +02:00
derivations))))
#:extra-headers http-headers-for-unchanging-content))
Add initial JSON output for the package page
2019-05-12 11:11:14 +02:00
(else
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (view-revision-package-and-version commit-hash
name
version
metadata
derivations
Make some pages around revisions more generic So that they can also be used for the /branch/foo/latest-processed-revision pages. The content is the same, but the title, link, and some of the links on the page are different.
2019-06-16 11:27:14 +02:00
git-repositories
Add lint warnings to the package page
2019-09-01 19:26:46 +02:00
lint-warnings
Make some pages around revisions more generic So that they can also be used for the /branch/foo/latest-processed-revision pages. The content is the same, but the title, link, and some of the links on the page are different.
2019-06-16 11:27:14 +02:00
#:header-text header-text
#:header-link header-link)
Cache the pages which don't really change for a day
2019-05-18 21:25:34 +02:00
#:extra-headers http-headers-for-unchanging-content)))))
Add initial JSON output for the package page
2019-05-12 11:11:14 +02:00
Add a page to show the lint warnings for a revision
2019-09-01 13:59:45 +02:00
(define* (render-revision-lint-warnings mime-types
conn
commit-hash
query-parameters
#:key
(path-base "/revision/")
(header-text
`("Revision " (samp ,commit-hash)))
(header-link
(string-append "/revision/" commit-hash)))
Support filtering lint warnings by linter
2019-09-01 15:11:50 +02:00
(define lint-checker-options
(map (match-lambda
((name description network-dependent)
(cons (string-append name ": " description )
name)))
(lint-checkers-for-revision conn commit-hash)))
Add a page to show the lint warnings for a revision
2019-09-01 13:59:45 +02:00
(if (any-invalid-query-parameters? query-parameters)
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
`((error . "invalid query"))))
(else
(render-html
#:sxml (view-revision-lint-warnings commit-hash
query-parameters
'()
Support filtering lint warnings by linter
2019-09-01 15:11:50 +02:00
lint-checker-options
Add a page to show the lint warnings for a revision
2019-09-01 13:59:45 +02:00
#:path-base path-base
#:header-text header-text
#:header-link header-link))))
(let* ((package-query (assq-ref query-parameters 'package_query))
(linters (assq-ref query-parameters 'linter))
(message-query (assq-ref query-parameters 'message_query))
(fields (assq-ref query-parameters 'field))
(git-repositories
(git-repositories-containing-commit conn
commit-hash))
(lint-warnings
(lint-warnings-for-guix-revision conn commit-hash
#:package-query package-query
#:linters linters
#:message-query message-query)))
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
`((revision
. ((commit . ,commit-hash)))
(lint_warnings
. ,(list->vector
(map (match-lambda
((id lint-checker-name lint-checker-description
lint-checker-network-dependent
package-name package-version
file line-number column-number
message)
`((package . ((name . ,package-name)
(version . ,package-version)))
,@(if (member "message" fields)
`((message . ,message))
'())
,@(if (member "location" fields)
`((location . ((file . ,file)
(line-number . ,line-number)
(column-number . ,column-number))))
'()))))
lint-warnings))))
#:extra-headers http-headers-for-unchanging-content))
(else
(render-html
#:sxml (view-revision-lint-warnings commit-hash
query-parameters
lint-warnings
git-repositories
Support filtering lint warnings by linter
2019-09-01 15:11:50 +02:00
lint-checker-options
Add a page to show the lint warnings for a revision
2019-09-01 13:59:45 +02:00
#:path-base path-base
#:header-text header-text
#:header-link header-link)
#:extra-headers http-headers-for-unchanging-content))))))
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(define (render-compare-unknown-commit mime-types
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
conn
base-commit
base-revision-id
target-commit
target-revision-id)
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
'((unknown_commit . #t))))
(else
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (compare-unknown-commit base-commit
target-commit
(if base-revision-id #t #f)
(if target-revision-id #t #f)
(select-job-for-commit conn
base-commit)
(select-job-for-commit conn
target-commit))))))
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(define (render-compare mime-types
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
conn
base-commit
base-revision-id
target-commit
target-revision-id)
(let-values
(((base-packages-vhash target-packages-vhash)
(package-data->package-data-vhashes
(package-differences-data conn
base-revision-id
target-revision-id))))
(let* ((new-packages
(package-data-vhashes->new-packages base-packages-vhash
target-packages-vhash))
(removed-packages
(package-data-vhashes->removed-packages base-packages-vhash
target-packages-vhash))
(version-changes
(package-data-version-changes base-packages-vhash
Move the derivations off the compare page To the compare/derivations page. Previously, the compare/derivations page was comparing more than the derivations, notably the package metadata. This change avoids that, and also reduces the information overload on the compare page.
2019-06-20 14:29:56 +02:00
target-packages-vhash)))
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
`((new-packages . ,(list->vector new-packages))
(removed-packages . ,(list->vector removed-packages))
Fix the JSON compare output
2019-07-29 20:48:57 +02:00
(version-changes . ,(list->vector
(map
(match-lambda
((name data ...)
`((name . ,name)
,@data)))
version-changes))))
Cache the pages which don't really change for a day
2019-05-18 21:25:34 +02:00
#:extra-headers http-headers-for-unchanging-content))
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(else
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (compare base-commit
target-commit
Link to cgit on the comparison page
2019-07-29 21:40:56 +02:00
(guix-revisions-cgit-url-bases
conn
(list base-revision-id
target-revision-id))
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
new-packages
removed-packages
Move the derivations off the compare page To the compare/derivations page. Previously, the compare/derivations page was comparing more than the derivations, notably the package metadata. This change avoids that, and also reduces the information overload on the compare page.
2019-06-20 14:29:56 +02:00
version-changes)
Cache the pages which don't really change for a day
2019-05-18 21:25:34 +02:00
#:extra-headers http-headers-for-unchanging-content))))))
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(define (render-compare/derivations mime-types
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
conn
Update the derivation comparison implementation This adds more query parameter validation, and uses form-horizontal-control to neaten up the view code.
2019-05-11 21:38:16 +02:00
query-parameters)
Many changes A large proportion of these changes relate to changing the way packages relate to derivations. Previously, a package at a given revision had a single derivation. This was OK, but didn't account for multiple architectures. Therefore, these changes mean that a package has multiple derivations, depending on the system of the derivation, and the target system. There are multiple changes, small and large to the web interface as well. More pages link to each other, and the visual display has been improved somewhat.
2019-03-11 23:11:14 +01:00
(define (derivations->alist derivations)
(map (match-lambda
Improve the compare derivations page Add support for filtering the results, and add the system and target to the output.
2019-03-17 23:44:09 +01:00
((file-name system target buildstatus)
Many changes A large proportion of these changes relate to changing the way packages relate to derivations. Previously, a package at a given revision had a single derivation. This was OK, but didn't account for multiple architectures. Therefore, these changes mean that a package has multiple derivations, depending on the system of the derivation, and the target system. There are multiple changes, small and large to the web interface as well. More pages link to each other, and the visual display has been improved somewhat.
2019-03-11 23:11:14 +01:00
`((file_name . ,file-name)
Improve the compare derivations page Add support for filtering the results, and add the system and target to the output.
2019-03-17 23:44:09 +01:00
(system . ,system)
(target . ,target)
Fix comparison in render-compare/derivations
2019-03-18 00:13:57 +01:00
(build_status . ,(if (string=? buildstatus "")
Many changes A large proportion of these changes relate to changing the way packages relate to derivations. Previously, a package at a given revision had a single derivation. This was OK, but didn't account for multiple architectures. Therefore, these changes mean that a package has multiple derivations, depending on the system of the derivation, and the target system. There are multiple changes, small and large to the web interface as well. More pages link to each other, and the visual display has been improved somewhat.
2019-03-11 23:11:14 +01:00
"unknown"
buildstatus)))))
derivations))
Update the derivation comparison implementation This adds more query parameter validation, and uses form-horizontal-control to neaten up the view code.
2019-05-11 21:38:16 +02:00
(if (any-invalid-query-parameters? query-parameters)
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
'((error . "invalid query"))))
(else
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (compare/derivations
query-parameters
(valid-systems conn)
build-status-strings
'()))))
Update the derivation comparison implementation This adds more query parameter validation, and uses form-horizontal-control to neaten up the view code.
2019-05-11 21:38:16 +02:00
(let ((base-commit (assq-ref query-parameters 'base_commit))
(target-commit (assq-ref query-parameters 'target_commit))
(systems (assq-ref query-parameters 'system))
(targets (assq-ref query-parameters 'target))
(build-statuses (assq-ref query-parameters 'build_status)))
Improve derivation comparison to show more changes In cases where the version is changed for example, the relevant derivations will now show up, whereas previously they did not.
2019-08-06 10:55:03 +02:00
(let*
((data
(package-differences-data conn
(commit->revision-id conn base-commit)
Support filtering by systems and targets when comparing derivations
2019-08-06 11:22:57 +02:00
(commit->revision-id conn target-commit)
#:systems systems
#:targets targets))
Improve derivation comparison to show more changes In cases where the version is changed for example, the relevant derivations will now show up, whereas previously they did not.
2019-08-06 10:55:03 +02:00
(names-and-versions
(package-data->names-and-versions data)))
(let-values
(((base-packages-vhash target-packages-vhash)
(package-data->package-data-vhashes data)))
(let ((derivation-changes
(package-data-derivation-changes names-and-versions
base-packages-vhash
target-packages-vhash)))
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
derivation-changes
#:extra-headers http-headers-for-unchanging-content))
(else
(render-html
#:sxml (compare/derivations
query-parameters
(valid-systems conn)
build-status-strings
derivation-changes)
#:extra-headers http-headers-for-unchanging-content)))))))))
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(define (render-compare/packages mime-types
Add a packages comparison page The primary use I have in mind for this is producing a list of strings suitable for building a limited Cuirass job with.
2019-02-26 00:44:32 +01:00
conn
base-commit
base-revision-id
target-commit
target-revision-id)
(define (package-data-vhash->json vh)
Many changes A large proportion of these changes relate to changing the way packages relate to derivations. Previously, a package at a given revision had a single derivation. This was OK, but didn't account for multiple architectures. Therefore, these changes mean that a package has multiple derivations, depending on the system of the derivation, and the target system. There are multiple changes, small and large to the web interface as well. More pages link to each other, and the visual display has been improved somewhat.
2019-03-11 23:11:14 +01:00
(delete-duplicates
(vhash-fold (lambda (name data result)
(cons `((name . ,name)
(version . ,(car data)))
result))
'()
vh)))
Add a packages comparison page The primary use I have in mind for this is producing a list of strings suitable for building a limited Cuirass job with.
2019-02-26 00:44:32 +01:00
(let-values
(((base-packages-vhash target-packages-vhash)
(package-data->package-data-vhashes
(package-differences-data conn
base-revision-id
target-revision-id))))
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
`((base
. ((commit . ,base-commit)
(packages . ,(list->vector
(package-data-vhash->json base-packages-vhash)))))
(target
. ((commit . ,target-commit)
(packages . ,(list->vector
Cache the pages which don't really change for a day
2019-05-18 21:25:34 +02:00
(package-data-vhash->json target-packages-vhash))))))
#:extra-headers http-headers-for-unchanging-content))
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(else
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (compare/packages
base-commit
target-commit
base-packages-vhash
Cache the pages which don't really change for a day
2019-05-18 21:25:34 +02:00
target-packages-vhash)
#:extra-headers http-headers-for-unchanging-content)))))
Add a packages comparison page The primary use I have in mind for this is producing a list of strings suitable for building a limited Cuirass job with.
2019-02-26 00:44:32 +01:00
Start to visualise derivations
2019-03-07 09:43:16 +01:00
(define (render-derivation conn derivation-file-name)
(let ((derivation (select-derivation-by-file-name conn
derivation-file-name)))
(if derivation
(let ((derivation-inputs (select-derivation-inputs-by-derivation-id
conn
(first derivation)))
(derivation-outputs (select-derivation-outputs-by-derivation-id
conn
Continue improving pages and linking things together
2019-03-08 00:50:51 +01:00
(first derivation)))
(builds (select-builds-with-context-by-derivation-id
conn
(first derivation))))
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (view-derivation derivation
derivation-inputs
derivation-outputs
Cache the pages which don't really change for a day
2019-05-18 21:25:34 +02:00
builds)
#:extra-headers http-headers-for-unchanging-content))
Properly handle not finding a matching derivation
2019-08-05 20:45:10 +02:00
(render-html
#:sxml (general-not-found
"Derivation not found"
"No derivation found with this file name.")
#:code 404))))
Start to visualise derivations
2019-03-07 09:43:16 +01:00
(define (render-store-item conn filename)
Continue improving pages and linking things together
2019-03-08 00:50:51 +01:00
(let ((derivation (select-derivation-by-output-filename conn filename)))
(match derivation
(()
Handle the 404 case when viewing store items
2019-05-20 09:17:56 +02:00
(render-html
#:sxml (general-not-found
"Store item not found"
"No derivation found producing this output")
#:code 404))
Many changes A large proportion of these changes relate to changing the way packages relate to derivations. Previously, a package at a given revision had a single derivation. This was OK, but didn't account for multiple architectures. Therefore, these changes mean that a package has multiple derivations, depending on the system of the derivation, and the target system. There are multiple changes, small and large to the web interface as well. More pages link to each other, and the visual display has been improved somewhat.
2019-03-11 23:11:14 +01:00
(derivations
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (view-store-item filename
derivations
(map (lambda (derivation)
(match derivation
((file-name output-id rest ...)
(select-derivations-using-output
conn output-id))))
Cache the pages which don't really change for a day
2019-05-18 21:25:34 +02:00
derivations))
#:extra-headers http-headers-for-unchanging-content)))))
Start to visualise derivations
2019-03-07 09:43:16 +01:00
Add a new page listing jobs
2019-06-17 12:21:58 +02:00
(define (render-jobs mime-types conn)
(render-html
#:sxml (view-jobs
(select-jobs-and-events conn))))
Iterate the log viewing Replace the Guile-side HTML escaping with a less complete, but hopefully faster PostgreSQL side HTML escaping approach. Also, allow reading part of the log, by default, the last 1,000,000 characters, as this should render quickly.
2019-06-24 21:30:47 +02:00
(define (render-job mime-types conn job-id query-parameters)
Add a job page Which shows the output for that job.
2019-06-22 01:14:57 +02:00
(render-html
#:sxml (view-job
job-id
Iterate the log viewing Replace the Guile-side HTML escaping with a less complete, but hopefully faster PostgreSQL side HTML escaping approach. Also, allow reading part of the log, by default, the last 1,000,000 characters, as this should render quickly.
2019-06-24 21:30:47 +02:00
query-parameters
(log-for-job conn job-id
#:character-limit
(assq-ref query-parameters 'characters)
#:start-character
(assq-ref query-parameters 'start_character)))))
Add a job page Which shows the output for that job.
2019-06-22 01:14:57 +02:00
Update the derivation comparison implementation This adds more query parameter validation, and uses form-horizontal-control to neaten up the view code.
2019-05-11 21:38:16 +02:00
(define (parse-commit conn)
(lambda (s)
(if (guix-commit-exists? conn s)
s
(make-invalid-query-parameter
s "unknown commit"))))
(define (parse-system s)
s)
(define (parse-build-status s)
s)
Extract the database connection handling from the server Previously, one of the first things that happened when responding to a request was a database connection was made, even when serving the CSS. This is unnecessary, so move the database connection handling in to the controller. Also, to allow for separating it out from the assets, separate the assets out from the parts of the controller that require a database connection.
2019-06-06 21:39:06 +02:00
(define (controller request method-and-path-components mime-types body)
(match method-and-path-components
Fix a couple of GET's in the controller match expression These should be symbols, not just placeholders.
2019-09-01 19:48:05 +02:00
(('GET "assets" rest ...)
Extract the database connection handling from the server Previously, one of the first things that happened when responding to a request was a database connection was made, even when serving the CSS. This is unnecessary, so move the database connection handling in to the controller. Also, to allow for separating it out from the assets, separate the assets out from the parts of the controller that require a database connection.
2019-06-06 21:39:06 +02:00
(or (render-static-asset (string-join rest "/")
(request-headers request))
(not-found (request-uri request))))
Fix a couple of GET's in the controller match expression These should be symbols, not just placeholders.
2019-09-01 19:48:05 +02:00
(('GET "healthcheck")
Add a healthcheck endpoint This indicates currently if the database is accessible and a simple query can be performed. I'm mostly looking at using this when writing a system test for this service.
2019-06-06 22:08:34 +02:00
(let ((database-status
(catch
#t
(lambda ()
(with-postgresql-connection
Associate a name with database connections This helps when working out which connection to the database is doing what.
2019-07-12 20:45:41 +02:00
"web healthcheck"
Add a healthcheck endpoint This indicates currently if the database is accessible and a simple query can be performed. I'm mostly looking at using this when writing a system test for this service.
2019-06-06 22:08:34 +02:00
(lambda (conn)
(number?
(string->number
(first
(count-guix-revisions conn)))))))
(lambda (key . args)
#f))))
(render-json
`((status . ,(if database-status
"ok"
"not ok")))
#:code (if (eq? database-status
#t)
200
500))))
Extract the database connection handling from the server Previously, one of the first things that happened when responding to a request was a database connection was made, even when serving the CSS. This is unnecessary, so move the database connection handling in to the controller. Also, to allow for separating it out from the assets, separate the assets out from the parts of the controller that require a database connection.
2019-06-06 21:39:06 +02:00
(_
(with-postgresql-connection
Associate a name with database connections This helps when working out which connection to the database is doing what.
2019-07-12 20:45:41 +02:00
"web"
Extract the database connection handling from the server Previously, one of the first things that happened when responding to a request was a database connection was made, even when serving the CSS. This is unnecessary, so move the database connection handling in to the controller. Also, to allow for separating it out from the assets, separate the assets out from the parts of the controller that require a database connection.
2019-06-06 21:39:06 +02:00
(lambda (conn)
(controller-with-database-connection request
method-and-path-components
mime-types
body
conn))))))
(define (controller-with-database-connection request
method-and-path-components
mime-types
body
conn)
Improve the compare derivations page Add support for filtering the results, and add the system and target to the output.
2019-03-17 23:44:09 +01:00
(define query-parameters
(-> request
request-uri
uri-query
parse-query-string))
Make some pages around revisions more generic So that they can also be used for the /branch/foo/latest-processed-revision pages. The content is the same, but the title, link, and some of the links on the page are different.
2019-06-16 11:27:14 +02:00
(define path
(uri-path (request-uri request)))
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(match method-and-path-components
Fix match expression Specify 'GET rather than GET, to actually match the method, rather than binding it to GET.
2019-07-19 21:20:52 +02:00
(('GET)
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (index
(map
(lambda (git-repository-details)
(cons
git-repository-details
Switch the index page to show branches for a repository As that's probably more useful than recent revisions and jobs.
2019-07-22 21:51:54 +02:00
(all-branches-with-most-recent-commit
conn (first git-repository-details))))
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(all-git-repositories conn)))))
Fix match expression Specify 'GET rather than GET, to actually match the method, rather than binding it to GET.
2019-07-19 21:20:52 +02:00
(('GET "builds")
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (view-builds (select-build-stats conn)
(select-builds-with-context conn))))
Fix match expression Specify 'GET rather than GET, to actually match the method, rather than binding it to GET.
2019-07-19 21:20:52 +02:00
(('GET "statistics")
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (view-statistics (count-guix-revisions conn)
(count-derivations conn))))
Fix match expression Specify 'GET rather than GET, to actually match the method, rather than binding it to GET.
2019-07-19 21:20:52 +02:00
(('GET "revision" commit-hash) (if (guix-commit-exists? conn commit-hash)
Add missing revision handling As previously the pages would render, but with no data.
2019-05-19 23:31:07 +02:00
(render-view-revision mime-types
conn
Make some pages around revisions more generic So that they can also be used for the /branch/foo/latest-processed-revision pages. The content is the same, but the title, link, and some of the links on the page are different.
2019-06-16 11:27:14 +02:00
commit-hash
#:path-base path)
Add missing revision handling As previously the pages would render, but with no data.
2019-05-19 23:31:07 +02:00
(render-unknown-revision mime-types
conn
commit-hash)))
Fix match expression Specify 'GET rather than GET, to actually match the method, rather than binding it to GET.
2019-07-19 21:20:52 +02:00
(('GET "revision" commit-hash "packages")
Add missing revision handling As previously the pages would render, but with no data.
2019-05-19 23:31:07 +02:00
(if (guix-commit-exists? conn commit-hash)
(let ((parsed-query-parameters
(guard-against-mutually-exclusive-query-parameters
(parse-query-parameters
request
`((after_name ,identity)
(field ,identity #:multi-value
#:default ("version" "synopsis"))
(search_query ,identity)
(limit_results ,parse-result-limit
#:no-default-when (all_results)
#:default 100)
(all_results ,parse-checkbox-value)))
;; You can't specify a search query, but then also limit the
;; results by filtering for after a particular package name
'((after_name search_query)
(limit_results all_results)))))
Add some basic pagination to the packages page Lower powered devices will have problems displaying all ~9000+ packages, so return a smaller number by default.
2019-05-12 20:51:44 +02:00
Add missing revision handling As previously the pages would render, but with no data.
2019-05-19 23:31:07 +02:00
(render-revision-packages mime-types
conn
commit-hash
Make some pages around revisions more generic So that they can also be used for the /branch/foo/latest-processed-revision pages. The content is the same, but the title, link, and some of the links on the page are different.
2019-06-16 11:27:14 +02:00
parsed-query-parameters
#:path-base path))
Add missing revision handling As previously the pages would render, but with no data.
2019-05-19 23:31:07 +02:00
(render-unknown-revision mime-types
conn
commit-hash)))
Fix match expression Specify 'GET rather than GET, to actually match the method, rather than binding it to GET.
2019-07-19 21:20:52 +02:00
(('GET "revision" commit-hash "package" name version)
Add missing revision handling As previously the pages would render, but with no data.
2019-05-19 23:31:07 +02:00
(if (guix-commit-exists? conn commit-hash)
(render-revision-package mime-types
conn
commit-hash
name
version)
(render-unknown-revision mime-types
conn
commit-hash)))
Add a page to show the lint warnings for a revision
2019-09-01 13:59:45 +02:00
(('GET "revision" commit-hash "lint-warnings")
(if (guix-commit-exists? conn commit-hash)
(let ((parsed-query-parameters
(parse-query-parameters
request
`((package_query ,identity)
(linter ,identity #:multi-value)
(message_query ,identity)
(field ,identity #:multi-value
#:default ("linter"
"message"
"location"))))))
(render-revision-lint-warnings mime-types
conn
commit-hash
parsed-query-parameters
#:path-base path))
(render-unknown-revision mime-types
conn
commit-hash)))
Improve how repositories and branches are handled Make the link between repositories and branches clearer, replacing the /branches and /branch pages by /repository/ and /repository/*/branch/* pages.
2019-07-19 22:22:15 +02:00
(('GET "repository" id)
(match (select-git-repository conn id)
((label url cgit-url-base)
(render-html
#:sxml
(view-git-repository
id
label url cgit-url-base
Use numeric ids for git repositories
2019-09-05 16:40:39 +02:00
(all-branches-with-most-recent-commit conn
(string->number id)))))
Improve how repositories and branches are handled Make the link between repositories and branches clearer, replacing the /branches and /branch pages by /repository/ and /repository/*/branch/* pages.
2019-07-19 22:22:15 +02:00
(#f
(render-html
#:sxml (general-not-found
"Repository not found"
"")
#:code 404))))
(('GET "repository" repository-id "branch" branch-name)
Add some options to the branch page Add handling for some query parameters to the branch page. This takes advantage of the improvements for building forms and query parameter validation.
2019-05-11 17:49:18 +02:00
(let ((parsed-query-parameters
(parse-query-parameters
request
`((after_date ,parse-datetime)
(before_date ,parse-datetime)
(limit_results ,parse-result-limit #:default 100)))))
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (if (any-invalid-query-parameters? parsed-query-parameters)
Improve how repositories and branches are handled Make the link between repositories and branches clearer, replacing the /branches and /branch pages by /repository/ and /repository/*/branch/* pages.
2019-07-19 22:22:15 +02:00
(view-branch repository-id
branch-name parsed-query-parameters '())
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(view-branch
Improve how repositories and branches are handled Make the link between repositories and branches clearer, replacing the /branches and /branch pages by /repository/ and /repository/*/branch/* pages.
2019-07-19 22:22:15 +02:00
repository-id
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
branch-name
parsed-query-parameters
(most-recent-commits-for-branch
conn
Use numeric ids for git repositories
2019-09-05 16:40:39 +02:00
(string->number repository-id)
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
branch-name
#:limit (assq-ref parsed-query-parameters 'limit_results)
#:after-date (assq-ref parsed-query-parameters
'after_date)
#:before-date (assq-ref parsed-query-parameters
'before_date)))))))
Fix the latest-processed-revision pages
2019-07-27 10:12:25 +02:00
(('GET "repository" repository-id "branch" branch-name "latest-processed-revision")
Support accessing the latest processed revision for a branch This makes is easier to get the latest data for a branch in a single request, rather than making one request to find the latest revision, then another to get the data.
2019-06-14 00:14:04 +02:00
(let ((commit-hash
Fix the latest-processed-revision pages
2019-07-27 10:12:25 +02:00
(latest-processed-commit-for-branch conn repository-id branch-name)))
Support accessing the latest processed revision for a branch This makes is easier to get the latest data for a branch in a single request, rather than making one request to find the latest revision, then another to get the data.
2019-06-14 00:14:04 +02:00
(if commit-hash
(render-view-revision mime-types
conn
Make some pages around revisions more generic So that they can also be used for the /branch/foo/latest-processed-revision pages. The content is the same, but the title, link, and some of the links on the page are different.
2019-06-16 11:27:14 +02:00
commit-hash
#:path-base path
#:header-text
`("Latest processed revision for branch "
(samp ,branch-name)))
Support accessing the latest processed revision for a branch This makes is easier to get the latest data for a branch in a single request, rather than making one request to find the latest revision, then another to get the data.
2019-06-14 00:14:04 +02:00
(render-unknown-revision mime-types
conn
commit-hash))))
Fix the latest-processed-revision pages
2019-07-27 10:12:25 +02:00
(('GET "repository" repository-id "branch" branch-name "latest-processed-revision" "packages")
Support accessing the latest processed revision for a branch This makes is easier to get the latest data for a branch in a single request, rather than making one request to find the latest revision, then another to get the data.
2019-06-14 00:14:04 +02:00
(let ((commit-hash
Fix the latest-processed-revision pages
2019-07-27 10:12:25 +02:00
(latest-processed-commit-for-branch conn repository-id branch-name)))
Support accessing the latest processed revision for a branch This makes is easier to get the latest data for a branch in a single request, rather than making one request to find the latest revision, then another to get the data.
2019-06-14 00:14:04 +02:00
(if commit-hash
(let ((parsed-query-parameters
(guard-against-mutually-exclusive-query-parameters
(parse-query-parameters
request
`((after_name ,identity)
(field ,identity #:multi-value
#:default ("version" "synopsis"))
(search_query ,identity)
(limit_results ,parse-result-limit
#:no-default-when (all_results)
#:default 100)
(all_results ,parse-checkbox-value)))
;; You can't specify a search query, but then also limit the
;; results by filtering for after a particular package name
'((after_name search_query)
(limit_results all_results)))))
(render-revision-packages mime-types
conn
commit-hash
Make some pages around revisions more generic So that they can also be used for the /branch/foo/latest-processed-revision pages. The content is the same, but the title, link, and some of the links on the page are different.
2019-06-16 11:27:14 +02:00
parsed-query-parameters
#:path-base path
#:header-text
`("Latest processed revision for branch "
(samp ,branch-name))
#:header-link
(string-append
Fix the latest-processed-revision pages
2019-07-27 10:12:25 +02:00
"/repository/" repository-id
Make some pages around revisions more generic So that they can also be used for the /branch/foo/latest-processed-revision pages. The content is the same, but the title, link, and some of the links on the page are different.
2019-06-16 11:27:14 +02:00
"/branch/" branch-name
"/latest-processed-revision")))
(render-unknown-revision mime-types
conn
commit-hash))))
Add a page to show the lint warnings for a revision
2019-09-01 13:59:45 +02:00
(('GET "repository" repository-id "branch" branch-name "latest-processed-revision"
"lint-warnings")
(let ((commit-hash
(latest-processed-commit-for-branch conn repository-id branch-name)))
(if commit-hash
(let ((parsed-query-parameters
(parse-query-parameters
request
`((package_query ,identity)
(linter ,identity #:multi-value)
(message_query ,identity)
(field ,identity #:multi-value
#:default ("linter"
"message"
"location"))))))
(render-revision-lint-warnings mime-types
conn
commit-hash
parsed-query-parameters
#:path-base path
#:header-text
`("Latest processed revision for branch "
(samp ,branch-name))
#:header-link
(string-append
"/repository/" repository-id
"/branch/" branch-name
"/latest-processed-revision")))
(render-unknown-revision mime-types
conn
commit-hash))))
Fix the latest-processed-revision pages
2019-07-27 10:12:25 +02:00
(('GET "repository" repository-id "branch" branch-name "latest-processed-revision" "package" name version)
Make some pages around revisions more generic So that they can also be used for the /branch/foo/latest-processed-revision pages. The content is the same, but the title, link, and some of the links on the page are different.
2019-06-16 11:27:14 +02:00
(let ((commit-hash
Fix the latest-processed-revision pages
2019-07-27 10:12:25 +02:00
(latest-processed-commit-for-branch conn repository-id branch-name)))
Make some pages around revisions more generic So that they can also be used for the /branch/foo/latest-processed-revision pages. The content is the same, but the title, link, and some of the links on the page are different.
2019-06-16 11:27:14 +02:00
(if commit-hash
(render-revision-package mime-types
conn
commit-hash
name
version
#:header-text
`("Latest processed revision for branch "
(samp ,branch-name))
#:header-link
(string-append
Fix the latest-processed-revision pages
2019-07-27 10:12:25 +02:00
"/repository/" repository-id
Make some pages around revisions more generic So that they can also be used for the /branch/foo/latest-processed-revision pages. The content is the same, but the title, link, and some of the links on the page are different.
2019-06-16 11:27:14 +02:00
"/branch/" branch-name
"/latest-processed-revision"))
Support accessing the latest processed revision for a branch This makes is easier to get the latest data for a branch in a single request, rather than making one request to find the latest revision, then another to get the data.
2019-06-14 00:14:04 +02:00
(render-unknown-revision mime-types
conn
commit-hash))))
Fix match expression Specify 'GET rather than GET, to actually match the method, rather than binding it to GET.
2019-07-19 21:20:52 +02:00
(('GET "gnu" "store" filename)
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
;; These routes are a little special, as the extensions aren't used for
;; content negotiation, so just use the path from the request
(let ((path (uri-path (request-uri request))))
(if (string-suffix? ".drv" path)
(render-derivation conn path)
(render-store-item conn path))))
Fix match expression Specify 'GET rather than GET, to actually match the method, rather than binding it to GET.
2019-07-19 21:20:52 +02:00
(('GET "compare")
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
(with-base-and-target-commits
Improve the compare derivations page Add support for filtering the results, and add the system and target to the output.
2019-03-17 23:44:09 +01:00
query-parameters conn
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
(lambda (base-commit base-revision-id target-commit target-revision-id)
(if (not (and base-revision-id target-revision-id))
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(render-compare-unknown-commit mime-types
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
conn
base-commit
base-revision-id
target-commit
target-revision-id)
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(render-compare mime-types
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
conn
base-commit
base-revision-id
target-commit
target-revision-id)))))
Fix match expression Specify 'GET rather than GET, to actually match the method, rather than binding it to GET.
2019-07-19 21:20:52 +02:00
(('GET "compare" "derivations")
Update the derivation comparison implementation This adds more query parameter validation, and uses form-horizontal-control to neaten up the view code.
2019-05-11 21:38:16 +02:00
(let* ((parsed-query-parameters
(parse-query-parameters
request
`((base_commit ,(parse-commit conn) #:required)
(target_commit ,(parse-commit conn) #:required)
(system ,parse-system #:multi-value)
(target ,parse-system #:multi-value)
(build_status ,parse-build-status #:multi-value)))))
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(render-compare/derivations mime-types
Update the derivation comparison implementation This adds more query parameter validation, and uses form-horizontal-control to neaten up the view code.
2019-05-11 21:38:16 +02:00
conn
parsed-query-parameters)))
Fix match expression Specify 'GET rather than GET, to actually match the method, rather than binding it to GET.
2019-07-19 21:20:52 +02:00
(('GET "compare" "packages")
Add a packages comparison page The primary use I have in mind for this is producing a list of strings suitable for building a limited Cuirass job with.
2019-02-26 00:44:32 +01:00
(with-base-and-target-commits
Improve the compare derivations page Add support for filtering the results, and add the system and target to the output.
2019-03-17 23:44:09 +01:00
query-parameters conn
Add a packages comparison page The primary use I have in mind for this is producing a list of strings suitable for building a limited Cuirass job with.
2019-02-26 00:44:32 +01:00
(lambda (base-commit base-revision-id target-commit target-revision-id)
(if (not (and base-revision-id target-revision-id))
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(render-compare-unknown-commit mime-types
Add a packages comparison page The primary use I have in mind for this is producing a list of strings suitable for building a limited Cuirass job with.
2019-02-26 00:44:32 +01:00
conn
base-commit
base-revision-id
target-commit
target-revision-id)
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(render-compare/packages mime-types
Add a packages comparison page The primary use I have in mind for this is producing a list of strings suitable for building a limited Cuirass job with.
2019-02-26 00:44:32 +01:00
conn
base-commit
base-revision-id
target-commit
target-revision-id)))))
Fix match expression Specify 'GET rather than GET, to actually match the method, rather than binding it to GET.
2019-07-19 21:20:52 +02:00
(('GET "jobs")
Add a new page listing jobs
2019-06-17 12:21:58 +02:00
(render-jobs mime-types
conn))
Fix match expression Specify 'GET rather than GET, to actually match the method, rather than binding it to GET.
2019-07-19 21:20:52 +02:00
(('GET "job" job-id)
Iterate the log viewing Replace the Guile-side HTML escaping with a less complete, but hopefully faster PostgreSQL side HTML escaping approach. Also, allow reading part of the log, by default, the last 1,000,000 characters, as this should render quickly.
2019-06-24 21:30:47 +02:00
(let ((parsed-query-parameters
(parse-query-parameters
request
`((start_character ,parse-number)
(characters ,parse-number #:default 1000000)))))
(render-job mime-types
conn
job-id
parsed-query-parameters)))
Fix match expression Specify 'GET rather than GET, to actually match the method, rather than binding it to GET.
2019-07-19 21:20:52 +02:00
(('GET path ...)
Extract the database connection handling from the server Previously, one of the first things that happened when responding to a request was a database connection was made, even when serving the CSS. This is unnecessary, so move the database connection handling in to the controller. Also, to allow for separating it out from the assets, separate the assets out from the parts of the controller that require a database connection.
2019-06-06 21:39:06 +02:00
(not-found (request-uri request)))))
Copy permalink