guix/data-service
2
0
Fork 0
mirror of git://git.savannah.gnu.org/guix/data-service.git synced 2023-12-14 03:23:03 +01:00
Code Issues Releases Wiki Activity
data-service/guix-data-service/web/controller.scm

638 lines
26 KiB
Scheme
Raw Normal View History

Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
;;; Guix Data Service -- Information about Guix over time
;;; Copyright © 2016, 2017, 2018, 2019 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2019 Christopher Baines <mail@cbaines.net>
;;;
;;; This program is free software: you can redistribute it and/or
;;; modify it under the terms of the GNU Affero General Public License
;;; as published by the Free Software Foundation, either version 3 of
;;; the License, or (at your option) any later version.
;;;
;;; This program is distributed in the hope that it will be useful,
;;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
;;; Affero General Public License for more details.
;;;
;;; You should have received a copy of the GNU Affero General Public
;;; License along with this program. If not, see
;;; <http://www.gnu.org/licenses/>.
(define-module (guix-data-service web controller)
#:use-module (ice-9 match)
Add a packages comparison page The primary use I have in mind for this is producing a list of strings suitable for building a limited Cuirass job with.
2019-02-26 00:44:32 +01:00
#:use-module (ice-9 vlist)
Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
#:use-module (ice-9 pretty-print)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-11)
#:use-module (srfi srfi-26)
#:use-module (web request)
#:use-module (web uri)
Allow specifying the fields on the packages page This is mostly for the JSON output, as it allows much more information to be included.
2019-05-16 23:28:16 +02:00
#:use-module (texinfo)
#:use-module (texinfo html)
#:use-module (texinfo plain-text)
Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
#:use-module (squee)
Allow specifying the fields on the packages page This is mostly for the JSON output, as it allows much more information to be included.
2019-05-16 23:28:16 +02:00
#:use-module (json)
Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
#:use-module (guix-data-service comparison)
Start to handle information about Git branches Add some new pages /branches and /branch/... as well as a new git_branches table. Also extend the email processing to enter the branch information in to the database.
2019-05-05 21:06:28 +02:00
#:use-module (guix-data-service model git-branch)
Switch to storing Git repositories in a table Rather than just storing the URL in the guix_revisions and load_new_guix_revision_jobs tables. This will help when storing more information like tags and branches in the future.
2019-05-05 14:35:48 +02:00
#:use-module (guix-data-service model git-repository)
Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
#:use-module (guix-data-service model guix-revision)
Add a few new pages For showing more information about builds, revisions and derivations.
2019-03-06 23:59:27 +01:00
#:use-module (guix-data-service model package)
Many changes A large proportion of these changes relate to changing the way packages relate to derivations. Previously, a package at a given revision had a single derivation. This was OK, but didn't account for multiple architectures. Therefore, these changes mean that a package has multiple derivations, depending on the system of the derivation, and the target system. There are multiple changes, small and large to the web interface as well. More pages link to each other, and the visual display has been improved somewhat.
2019-03-11 23:11:14 +01:00
#:use-module (guix-data-service model package-derivation)
#:use-module (guix-data-service model package-metadata)
Start to visualise derivations
2019-03-07 09:43:16 +01:00
#:use-module (guix-data-service model derivation)
Improve the compare derivations page Add support for filtering the results, and add the system and target to the output.
2019-03-17 23:44:09 +01:00
#:use-module (guix-data-service model build-status)
Add a few new pages For showing more information about builds, revisions and derivations.
2019-03-06 23:59:27 +01:00
#:use-module (guix-data-service model build)
Show when revisions are missing, but queued for processing
2019-02-24 17:47:29 +01:00
#:use-module (guix-data-service jobs load-new-guix-revision)
Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
#:use-module (guix-data-service web render)
Allow specifying the fields on the packages page This is mostly for the JSON output, as it allows much more information to be included.
2019-05-16 23:28:16 +02:00
#:use-module (guix-data-service web sxml)
Add a new query-parameters module The query parameters feed in to the results shown, but also forms on pages. Validation is important to avoid errors and security issues, but it's also important to provide appropriate feedback to the user. This module provides some utilities and structure around handling query parameters.
2019-05-11 17:44:17 +02:00
#:use-module (guix-data-service web query-parameters)
Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
#:use-module (guix-data-service web util)
#:use-module (guix-data-service web view html)
#:export (controller))
(define-syntax-rule (-> target functions ...)
(fold (lambda (f val) (and=> val f))
target
(list functions ...)))
(define (render-with-error-handling page message)
(apply render-html (page))
;; (catch #t
;; (lambda ()
;; (receive (sxml headers)
;; (pretty-print (page))
;; (render-html sxml headers)))
;; (lambda (key . args)
;; (format #t "ERROR: ~a ~a\n"
;; key args)
;; (render-html (error-page message))))
)
Improve the compare derivations page Add support for filtering the results, and add the system and target to the output.
2019-03-17 23:44:09 +01:00
(define (assoc-ref-multiple alist key)
(filter-map
(match-lambda
((k . value)
(and (string=? k key)
value)))
alist))
(define (with-base-and-target-commits query-parameters conn f)
(let* ((base-commit (assoc-ref query-parameters "base_commit"))
(target-commit (assoc-ref query-parameters "target_commit")))
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
(f base-commit
(commit->revision-id conn base-commit)
target-commit
(commit->revision-id conn target-commit))))
Add a JSON variant of the revision page
2019-05-12 10:51:00 +02:00
(define (render-view-revision mime-types
conn
commit-hash)
(let ((packages-count
(count-packages-in-revision conn commit-hash))
Show the branches for a revision
2019-05-18 13:36:03 +02:00
(git-repositories-and-branches
(git-branches-with-repository-details-for-commit conn commit-hash))
Add a JSON variant of the revision page
2019-05-12 10:51:00 +02:00
(derivations-counts
(count-packages-derivations-in-revision conn commit-hash)))
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
`((packages_count . ,(caar packages-count))
(derivations_counts . ,(list->vector
(map (match-lambda
((system target derivation_count)
`((system . ,system)
(target . ,target)
(derivation_count . ,derivation_count))))
derivations-counts))))))
(else
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (view-revision
commit-hash
packages-count
git-repositories-and-branches
derivations-counts))))))
Add a JSON variant of the revision page
2019-05-12 10:51:00 +02:00
Allow specifying the fields on the packages page This is mostly for the JSON output, as it allows much more information to be included.
2019-05-16 23:28:16 +02:00
(define (texinfo->variants-alist s)
(let ((stexi (texi-fragment->stexi s)))
`((source . ,s)
(html . ,(with-output-to-string
(lambda ()
(sxml->html (stexi->shtml stexi)))))
(plain . ,(stexi->plain-text stexi)))))
Add a JSON variant of the packages page
2019-05-12 10:56:24 +02:00
(define (render-revision-packages mime-types
conn
Add some basic pagination to the packages page Lower powered devices will have problems displaying all ~9000+ packages, so return a smaller number by default.
2019-05-12 20:51:44 +02:00
commit-hash
query-parameters)
Add a textual search to the packages page
2019-05-12 22:17:08 +02:00
(if (any-invalid-query-parameters? query-parameters)
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
`((error . "invalid query"))))
(else
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (view-revision-packages commit-hash
query-parameters
'()
'()
#f))))
Add a textual search to the packages page
2019-05-12 22:17:08 +02:00
(let* ((search-query (assq-ref query-parameters 'search_query))
(limit-results (assq-ref query-parameters 'limit_results))
Allow specifying the fields on the packages page This is mostly for the JSON output, as it allows much more information to be included.
2019-05-16 23:28:16 +02:00
(fields (assq-ref query-parameters 'field))
Add a textual search to the packages page
2019-05-12 22:17:08 +02:00
(packages
(if search-query
(search-packages-in-revision
Add some basic pagination to the packages page Lower powered devices will have problems displaying all ~9000+ packages, so return a smaller number by default.
2019-05-12 20:51:44 +02:00
conn
commit-hash
Add a textual search to the packages page
2019-05-12 22:17:08 +02:00
search-query
#:limit-results limit-results)
(select-packages-in-revision
conn
commit-hash
#:limit-results limit-results
#:after-name (assq-ref query-parameters 'after_name))))
Allow specifying the fields on the packages page This is mostly for the JSON output, as it allows much more information to be included.
2019-05-16 23:28:16 +02:00
(git-repositories
(git-repositories-containing-commit conn
commit-hash))
Add a textual search to the packages page
2019-05-12 22:17:08 +02:00
(show-next-page?
(and (not search-query)
(>= (length packages)
limit-results))))
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
Allow specifying the fields on the packages page This is mostly for the JSON output, as it allows much more information to be included.
2019-05-16 23:28:16 +02:00
`((revision
. ((commit . ,commit-hash)))
(packages
. ,(list->vector
(map (match-lambda
((name version synopsis description home-page
location-file location-line
location-column-number licenses)
`((name . ,name)
,@(if (member "version" fields)
`((version . ,version))
'())
,@(if (member "synopsis" fields)
`((synopsis
. ,(texinfo->variants-alist synopsis)))
'())
,@(if (member "description" fields)
`((description
. ,(texinfo->variants-alist description)))
'())
,@(if (member "home-page" fields)
`((home-page . ,home-page))
'())
,@(if (member "location" fields)
`((location
. ((file . ,location-file)
(line . ,location-line)
(column . ,location-column-number))))
'())
,@(if (member "licenses" fields)
`((licenses
. ,(if (string-null? licenses)
#()
(json-string->scm licenses))))
'()))))
packages))))))
Add a textual search to the packages page
2019-05-12 22:17:08 +02:00
(else
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (view-revision-packages commit-hash
query-parameters
packages
git-repositories
show-next-page?)))))))
Add a JSON variant of the packages page
2019-05-12 10:56:24 +02:00
Add initial JSON output for the package page
2019-05-12 11:11:14 +02:00
(define (render-revision-package mime-types
conn
commit-hash
name
version)
(let ((metadata
(select-package-metadata-by-revision-name-and-version
conn
commit-hash
name
version))
(derivations
(select-derivations-by-revision-name-and-version
conn
commit-hash
name
Store and display the location of packages Store the location a package can be found at, and display this on the package page. If available, link off to the git repository containing the package.
2019-05-13 22:02:53 +02:00
version))
(git-repositories
(git-repositories-containing-commit conn
commit-hash)))
Add initial JSON output for the package page
2019-05-12 11:11:14 +02:00
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
`((name . ,name)
(version . ,version)
,@(match metadata
(((synopsis description home-page))
`((synopsis . ,synopsis)
(description . ,description)
(home-page . ,home-page))))
(derivations . ,(list->vector
(map (match-lambda
((system target file-name status)
`((system . ,system)
(target . ,target)
(derivation . ,file-name))))
derivations))))))
(else
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (view-revision-package-and-version commit-hash
name
version
metadata
derivations
git-repositories))))))
Add initial JSON output for the package page
2019-05-12 11:11:14 +02:00
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(define (render-compare-unknown-commit mime-types
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
conn
base-commit
base-revision-id
target-commit
target-revision-id)
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
'((unknown_commit . #t))))
(else
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (compare-unknown-commit base-commit
target-commit
(if base-revision-id #t #f)
(if target-revision-id #t #f)
(select-job-for-commit conn
base-commit)
(select-job-for-commit conn
target-commit))))))
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(define (render-compare mime-types
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
conn
base-commit
base-revision-id
target-commit
target-revision-id)
(let-values
(((base-packages-vhash target-packages-vhash)
(package-data->package-data-vhashes
(package-differences-data conn
base-revision-id
target-revision-id))))
(let* ((new-packages
(package-data-vhashes->new-packages base-packages-vhash
target-packages-vhash))
(removed-packages
(package-data-vhashes->removed-packages base-packages-vhash
target-packages-vhash))
(version-changes
(package-data-version-changes base-packages-vhash
target-packages-vhash))
Many changes A large proportion of these changes relate to changing the way packages relate to derivations. Previously, a package at a given revision had a single derivation. This was OK, but didn't account for multiple architectures. Therefore, these changes mean that a package has multiple derivations, depending on the system of the derivation, and the target system. There are multiple changes, small and large to the web interface as well. More pages link to each other, and the visual display has been improved somewhat.
2019-03-11 23:11:14 +01:00
(derivation-changes
(package-data-derivation-changes base-packages-vhash
target-packages-vhash)))
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
`((new-packages . ,(list->vector new-packages))
(removed-packages . ,(list->vector removed-packages))
(version-changes . ,version-changes)
(derivation-changes . ,derivation-changes))))
(else
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (compare base-commit
target-commit
new-packages
removed-packages
version-changes
derivation-changes)))))))
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(define (render-compare/derivations mime-types
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
conn
Update the derivation comparison implementation This adds more query parameter validation, and uses form-horizontal-control to neaten up the view code.
2019-05-11 21:38:16 +02:00
query-parameters)
Many changes A large proportion of these changes relate to changing the way packages relate to derivations. Previously, a package at a given revision had a single derivation. This was OK, but didn't account for multiple architectures. Therefore, these changes mean that a package has multiple derivations, depending on the system of the derivation, and the target system. There are multiple changes, small and large to the web interface as well. More pages link to each other, and the visual display has been improved somewhat.
2019-03-11 23:11:14 +01:00
(define (derivations->alist derivations)
(map (match-lambda
Improve the compare derivations page Add support for filtering the results, and add the system and target to the output.
2019-03-17 23:44:09 +01:00
((file-name system target buildstatus)
Many changes A large proportion of these changes relate to changing the way packages relate to derivations. Previously, a package at a given revision had a single derivation. This was OK, but didn't account for multiple architectures. Therefore, these changes mean that a package has multiple derivations, depending on the system of the derivation, and the target system. There are multiple changes, small and large to the web interface as well. More pages link to each other, and the visual display has been improved somewhat.
2019-03-11 23:11:14 +01:00
`((file_name . ,file-name)
Improve the compare derivations page Add support for filtering the results, and add the system and target to the output.
2019-03-17 23:44:09 +01:00
(system . ,system)
(target . ,target)
Fix comparison in render-compare/derivations
2019-03-18 00:13:57 +01:00
(build_status . ,(if (string=? buildstatus "")
Many changes A large proportion of these changes relate to changing the way packages relate to derivations. Previously, a package at a given revision had a single derivation. This was OK, but didn't account for multiple architectures. Therefore, these changes mean that a package has multiple derivations, depending on the system of the derivation, and the target system. There are multiple changes, small and large to the web interface as well. More pages link to each other, and the visual display has been improved somewhat.
2019-03-11 23:11:14 +01:00
"unknown"
buildstatus)))))
derivations))
Update the derivation comparison implementation This adds more query parameter validation, and uses form-horizontal-control to neaten up the view code.
2019-05-11 21:38:16 +02:00
(if (any-invalid-query-parameters? query-parameters)
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
'((error . "invalid query"))))
(else
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (compare/derivations
query-parameters
(valid-systems conn)
build-status-strings
'()
'()))))
Update the derivation comparison implementation This adds more query parameter validation, and uses form-horizontal-control to neaten up the view code.
2019-05-11 21:38:16 +02:00
(let ((base-commit (assq-ref query-parameters 'base_commit))
(target-commit (assq-ref query-parameters 'target_commit))
(systems (assq-ref query-parameters 'system))
(targets (assq-ref query-parameters 'target))
(build-statuses (assq-ref query-parameters 'build_status)))
(let-values
(((base-packages-vhash target-packages-vhash)
(package-data->package-data-vhashes
(package-differences-data conn
(commit->revision-id conn base-commit)
(commit->revision-id conn target-commit)))))
(let ((base-derivations
(package-data-vhash->derivations-and-build-status
conn
base-packages-vhash
systems
targets
build-statuses))
(target-derivations
(package-data-vhash->derivations-and-build-status
conn
target-packages-vhash
systems
targets
build-statuses)))
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
`((base . ((commit . ,base-commit)
(derivations . ,(list->vector
(derivations->alist
base-derivations)))))
(target . ((commit . ,target-commit)
(derivations . ,(list->vector
(derivations->alist
target-derivations))))))))
(else
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (compare/derivations
query-parameters
(valid-systems conn)
build-status-strings
base-derivations
target-derivations)))))))))
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(define (render-compare/packages mime-types
Add a packages comparison page The primary use I have in mind for this is producing a list of strings suitable for building a limited Cuirass job with.
2019-02-26 00:44:32 +01:00
conn
base-commit
base-revision-id
target-commit
target-revision-id)
(define (package-data-vhash->json vh)
Many changes A large proportion of these changes relate to changing the way packages relate to derivations. Previously, a package at a given revision had a single derivation. This was OK, but didn't account for multiple architectures. Therefore, these changes mean that a package has multiple derivations, depending on the system of the derivation, and the target system. There are multiple changes, small and large to the web interface as well. More pages link to each other, and the visual display has been improved somewhat.
2019-03-11 23:11:14 +01:00
(delete-duplicates
(vhash-fold (lambda (name data result)
(cons `((name . ,name)
(version . ,(car data)))
result))
'()
vh)))
Add a packages comparison page The primary use I have in mind for this is producing a list of strings suitable for building a limited Cuirass job with.
2019-02-26 00:44:32 +01:00
(let-values
(((base-packages-vhash target-packages-vhash)
(package-data->package-data-vhashes
(package-differences-data conn
base-revision-id
target-revision-id))))
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(case (most-appropriate-mime-type
'(application/json text/html)
mime-types)
((application/json)
(render-json
`((base
. ((commit . ,base-commit)
(packages . ,(list->vector
(package-data-vhash->json base-packages-vhash)))))
(target
. ((commit . ,target-commit)
(packages . ,(list->vector
(package-data-vhash->json target-packages-vhash))))))))
(else
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (compare/packages
base-commit
target-commit
base-packages-vhash
target-packages-vhash))))))
Add a packages comparison page The primary use I have in mind for this is producing a list of strings suitable for building a limited Cuirass job with.
2019-02-26 00:44:32 +01:00
Start to visualise derivations
2019-03-07 09:43:16 +01:00
(define (render-derivation conn derivation-file-name)
(let ((derivation (select-derivation-by-file-name conn
derivation-file-name)))
(if derivation
(let ((derivation-inputs (select-derivation-inputs-by-derivation-id
conn
(first derivation)))
(derivation-outputs (select-derivation-outputs-by-derivation-id
conn
Continue improving pages and linking things together
2019-03-08 00:50:51 +01:00
(first derivation)))
(builds (select-builds-with-context-by-derivation-id
conn
(first derivation))))
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (view-derivation derivation
derivation-inputs
derivation-outputs
builds)))
Start to visualise derivations
2019-03-07 09:43:16 +01:00
#f ;; TODO
)))
(define (render-store-item conn filename)
Continue improving pages and linking things together
2019-03-08 00:50:51 +01:00
(let ((derivation (select-derivation-by-output-filename conn filename)))
(match derivation
(()
#f)
Many changes A large proportion of these changes relate to changing the way packages relate to derivations. Previously, a package at a given revision had a single derivation. This was OK, but didn't account for multiple architectures. Therefore, these changes mean that a package has multiple derivations, depending on the system of the derivation, and the target system. There are multiple changes, small and large to the web interface as well. More pages link to each other, and the visual display has been improved somewhat.
2019-03-11 23:11:14 +01:00
(derivations
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (view-store-item filename
derivations
(map (lambda (derivation)
(match derivation
((file-name output-id rest ...)
(select-derivations-using-output
conn output-id))))
derivations)))))))
Start to visualise derivations
2019-03-07 09:43:16 +01:00
Update the derivation comparison implementation This adds more query parameter validation, and uses form-horizontal-control to neaten up the view code.
2019-05-11 21:38:16 +02:00
(define (parse-commit conn)
(lambda (s)
(if (guix-commit-exists? conn s)
s
(make-invalid-query-parameter
s "unknown commit"))))
(define (parse-system s)
s)
(define (parse-build-status s)
s)
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(define (controller request method-and-path-components mime-types body conn)
Improve the compare derivations page Add support for filtering the results, and add the system and target to the output.
2019-03-17 23:44:09 +01:00
(define query-parameters
(-> request
request-uri
uri-query
parse-query-string))
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(match method-and-path-components
Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
((GET)
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (index
(map
(lambda (git-repository-details)
(cons
git-repository-details
(map
(match-lambda
((id job-id commit source)
(list id
job-id
commit
source
(git-branches-for-commit conn commit))))
(guix-revisions-and-jobs-for-git-repository
conn
(car git-repository-details)))))
(all-git-repositories conn)))))
Add a few new pages For showing more information about builds, revisions and derivations.
2019-03-06 23:59:27 +01:00
((GET "builds")
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (view-builds (select-build-stats conn)
(select-builds-with-context conn))))
Add a statistics page To show the total number of derivations, and guix revisions.
2019-04-08 22:04:12 +02:00
((GET "statistics")
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (view-statistics (count-guix-revisions conn)
(count-derivations conn))))
Add a JSON variant of the revision page
2019-05-12 10:51:00 +02:00
((GET "revision" commit-hash) (render-view-revision mime-types
conn
commit-hash))
Add some basic pagination to the packages page Lower powered devices will have problems displaying all ~9000+ packages, so return a smaller number by default.
2019-05-12 20:51:44 +02:00
((GET "revision" commit-hash "packages")
(let ((parsed-query-parameters
Add a textual search to the packages page
2019-05-12 22:17:08 +02:00
(guard-against-mutually-exclusive-query-parameters
(parse-query-parameters
request
`((after_name ,identity)
Allow specifying the fields on the packages page This is mostly for the JSON output, as it allows much more information to be included.
2019-05-16 23:28:16 +02:00
(field ,identity #:multi-value
#:default ("version" "synopsis"))
Add a textual search to the packages page
2019-05-12 22:17:08 +02:00
(search_query ,identity)
(limit_results ,parse-result-limit #:default 100)))
;; You can't specify a search query, but then also limit the
;; results by filtering for after a particular package name
'((after_name search_query)))))
Add some basic pagination to the packages page Lower powered devices will have problems displaying all ~9000+ packages, so return a smaller number by default.
2019-05-12 20:51:44 +02:00
(render-revision-packages mime-types
conn
commit-hash
parsed-query-parameters)))
Add initial JSON output for the package page
2019-05-12 11:11:14 +02:00
((GET "revision" commit-hash "package" name version) (render-revision-package
mime-types
conn
commit-hash
name
version))
Start to handle information about Git branches Add some new pages /branches and /branch/... as well as a new git_branches table. Also extend the email processing to enter the branch information in to the database.
2019-05-05 21:06:28 +02:00
((GET "branches")
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (view-branches
(all-branches-with-most-recent-commit conn))))
Start to handle information about Git branches Add some new pages /branches and /branch/... as well as a new git_branches table. Also extend the email processing to enter the branch information in to the database.
2019-05-05 21:06:28 +02:00
((GET "branch" branch-name)
Add some options to the branch page Add handling for some query parameters to the branch page. This takes advantage of the improvements for building forms and query parameter validation.
2019-05-11 17:49:18 +02:00
(let ((parsed-query-parameters
(parse-query-parameters
request
`((after_date ,parse-datetime)
(before_date ,parse-datetime)
(limit_results ,parse-result-limit #:default 100)))))
Remove the HTTP headers from the html module Given that the headers may be the same, regardless whether it's HTML or JSON being sent in the body of the response, I think it makes more sense to handle the headers in the controller.
2019-05-18 21:08:34 +02:00
(render-html
#:sxml (if (any-invalid-query-parameters? parsed-query-parameters)
(view-branch branch-name parsed-query-parameters '())
(view-branch
branch-name
parsed-query-parameters
(most-recent-commits-for-branch
conn
branch-name
#:limit (assq-ref parsed-query-parameters 'limit_results)
#:after-date (assq-ref parsed-query-parameters
'after_date)
#:before-date (assq-ref parsed-query-parameters
'before_date)))))))
Start to visualise derivations
2019-03-07 09:43:16 +01:00
((GET "gnu" "store" filename)
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
;; These routes are a little special, as the extensions aren't used for
;; content negotiation, so just use the path from the request
(let ((path (uri-path (request-uri request))))
(if (string-suffix? ".drv" path)
(render-derivation conn path)
(render-store-item conn path))))
Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
((GET "compare")
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
(with-base-and-target-commits
Improve the compare derivations page Add support for filtering the results, and add the system and target to the output.
2019-03-17 23:44:09 +01:00
query-parameters conn
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
(lambda (base-commit base-revision-id target-commit target-revision-id)
(if (not (and base-revision-id target-revision-id))
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(render-compare-unknown-commit mime-types
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
conn
base-commit
base-revision-id
target-commit
target-revision-id)
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(render-compare mime-types
Add some super crude JSON pages Provide JSON versions of the existing HTML compare and compare/derivations pages. Refactor the code and extract some functions to make this a little less painful.
2019-02-25 23:07:26 +01:00
conn
base-commit
base-revision-id
target-commit
target-revision-id)))))
Add a new page comparing the derivations of two revisions
2019-02-24 16:38:08 +01:00
((GET "compare" "derivations")
Update the derivation comparison implementation This adds more query parameter validation, and uses form-horizontal-control to neaten up the view code.
2019-05-11 21:38:16 +02:00
(let* ((parsed-query-parameters
(parse-query-parameters
request
`((base_commit ,(parse-commit conn) #:required)
(target_commit ,(parse-commit conn) #:required)
(system ,parse-system #:multi-value)
(target ,parse-system #:multi-value)
(build_status ,parse-build-status #:multi-value)))))
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(render-compare/derivations mime-types
Update the derivation comparison implementation This adds more query parameter validation, and uses form-horizontal-control to neaten up the view code.
2019-05-11 21:38:16 +02:00
conn
parsed-query-parameters)))
Add a packages comparison page The primary use I have in mind for this is producing a list of strings suitable for building a limited Cuirass job with.
2019-02-26 00:44:32 +01:00
((GET "compare" "packages")
(with-base-and-target-commits
Improve the compare derivations page Add support for filtering the results, and add the system and target to the output.
2019-03-17 23:44:09 +01:00
query-parameters conn
Add a packages comparison page The primary use I have in mind for this is producing a list of strings suitable for building a limited Cuirass job with.
2019-02-26 00:44:32 +01:00
(lambda (base-commit base-revision-id target-commit target-revision-id)
(if (not (and base-revision-id target-revision-id))
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(render-compare-unknown-commit mime-types
Add a packages comparison page The primary use I have in mind for this is producing a list of strings suitable for building a limited Cuirass job with.
2019-02-26 00:44:32 +01:00
conn
base-commit
base-revision-id
target-commit
target-revision-id)
Improve the content negotiation handling in general Previously, the routing layer handled the content negotiation, and the Accept header was ignored. Now, the extension if one is provided in the URL is still used, and more widely than before, but the Accept header is also taken in to account. This all now happens before the routing decisions are made, so the routing is now pretty much extension independant (with the exception of the /gnu/store/... routes).
2019-05-11 23:56:25 +02:00
(render-compare/packages mime-types
Add a packages comparison page The primary use I have in mind for this is producing a list of strings suitable for building a limited Cuirass job with.
2019-02-26 00:44:32 +01:00
conn
base-commit
base-revision-id
target-commit
target-revision-id)))))
Initial commit This is a service designed to provide information about Guix. At the moment, this initial prototype gathers up information about packages, the associated metadata and derivations. The initial primary use case is to compare two different revisions of Guix, detecting which packages are new, no longer present, updated or otherwise different. It's based on the Mumi project. [1]: https://git.elephly.net/software/mumi.git
2019-02-06 17:14:44 +01:00
((GET path ...)
(render-static-asset request))))
Copy permalink