Service to store and provide data about Guix
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Christopher Baines 68850065d7 Extend the invalid parameter handling for more types 1 week ago
assets Bring back the old table header style 3 years ago
build-aux Add missing build-aux files 4 years ago
guix-data-service Extend the invalid parameter handling for more types 1 week ago
scripts Set request timeouts for the thread pools 1 month ago
sqitch Store the distribution of derivations related to packages 3 months ago
tests Set current-guix-package when computing system test derivations 3 months ago
.dir-locals.el Rework the shortlived PostgreSQL specific connection channel 3 years ago
.envrc Improve handling of errors 3 years ago
.gitattributes Add a .gitattributes file 4 years ago
.gitignore Split out querying of build servers and substitute servers 3 years ago
COPYING Initial commit 4 years ago Add support for incrementally tracking blocked builds 7 months ago
README State where to send patches in the README 2 years ago
VERSION Initial commit 4 years ago Initial commit 4 years ago Add dependency on Guile-lzlib. 3 years ago Generate .go files for the tests 3 years ago
guix-dev.scm Switch to guile-fibers@1.1 1 year ago Fix some comments 3 years ago
sqitch.conf Remove localhost from the Sqitch config 4 years ago


-*- mode: org -*-

The Guix Data Service processes, stores and provides data about Guix over

It's written in Guile, stores data in a PostgreSQL database, and provides a
web interface and API to browse and access the data.

* Overview

The aim of the Guix Data Service is to provide a complementary interface to
Guix itself, providing a different way of interacting with Guix data, which
hopefully enables tools and services that otherwise wouldn't be feasible to

For example, with the command line tooling or Guile API for Guix, you can
quickly and easily access information about packages, and manipulate
derivations. However, if you wish to compare two revisions of Guix, or look at
the history of the derivations for a package over recent Guix revisions, it
could take some time until you have the information you're looking for.

However, with the Guix Data Service, this information can be gathered
continuously, and stored in a way that makes these kind of queries much
quicker to answer. By storing data about multiple Guix revisions in a SQL
database (PostgreSQL), it becomes possible to quickly compare the data for
different revisions. By configuring the Guix Data Service to listen for
changes in the Guix Git repository, the database can be kept up to date.

* Local Development

There's no one right way of doing local development, but outlined here is a
recommended approach.

You will need to have:
 - The Guix package manager, either installed on any GNU/Linux distribution,
   or be using Guix as a system.
 - A running PostgreSQL service, version 10 or above is recommended
 - Git, so you can clone the repository
 - direnv, to manage environment variables

If you have any problems with the instructions here, you can ask for help on
the #guix IRC channel on Freenode. You can also email, please
include the step you got to, and the problem you encountered (with the error
message if applicable).

** Cloning the repository

To clone the Git repository, run the following command:

#+BEGIN_SRC shell
git clone guix-data-service

** Getting direnv working

direnv is used to update the environment within the shell to contain the
dependencies of the Guix Data Service. If you've just installed direnv, check
you've hooked it in to your shell .

Change in to the =guix-data-service= directory. You should see direnv
prompting to trust the =.envrc= file contained within the repository.

Check the contents of the =.envrc= file, and if you trust it, run =direnv

** Setup the database

Dumps of the database behind are available at, use the following URL to download the latest
small dump.

There are multiple ways you can setup the database, the instructions here
involve creating a =guix_data_service= user with a password, then connecting
to the database over the local network interface.

To create the user and database use the scripts from PostgreSQL. The
=createuser= command will prompt for a password for the database user, you'll
need this at multiple points so it might be useful to keep it simple.

#+BEGIN_SRC shell
createuser guix_data_service --pwprompt
createdb guix_data_service --owner=guix_data_service

You may need to run these commands as the postgres user.

To restore the downloaded dump, run =pg_restore=. This command should prompt
for a password, at which point enter the password for the database user that
you set earlier.

#+BEGIN_SRC shell
pg_restore --jobs=4 --user=guix_data_service --no-tablespaces --host= --dbname=guix_data_service guix_data_service_small.dump

** Set the database paramstring

To setup the configuration for your local environment, create a file called
.local.envrc. To provide the password you've set for the =guix_data_service=
user, you'll need to set the =GUIX_DATA_SERVICE_DATABASE_PARAMSTRING=
environment variable in this file. Add the following line, replacing
THEPASSWORD with the password you set earlier.

#+BEGIN_SRC shell
export GUIX_DATA_SERVICE_DATABASE_PARAMSTRING="dbname=guix_data_service user=guix_data_service password=THEPASSWORD host="

After, run =direnv reload= to update the environment in your shell.

** Building the source files

The following three commands will compile the =.go= files and generate the
scripts used as entry points to the Guix Data Service.

#+BEGIN_SRC shell

** Run the web server

The =guix-data-service= script is the main entry point in to the service, run
it from the shell.

#+BEGIN_SRC shell

You should now be able to access the Guix Data Service web interface through a
web browser using the URL http://localhost:8765/

To automatically restart the Guix Data Service when a change to the code is
made, you can use the rerun command:

#+BEGIN_SRC shell
rerun -d guix-data-service -p "**/*.scm" guix-data-service

** Checking the setup

You now should be ready to do local development. To double check that you can
change the code and see those changes, try changing the "Guix Data Service"
text within the index function, in the =(guix-data-service web view html)=
module (contained in the =guix-data-service/web/view/html.scm= file). That
text appears towards the top of the index page (the page with the path =/=).

* Contributing

Please send patches to

* Roadmap

Ideas for future enhancements to the Guix Data Service, in no particular

** WebSub support for subscribing to branches and other changable state
** Link to other sites (Free Software Directory, Wikidata, Debian, ...)
** Track package replacements (grafts)
** Provide data in RDF formats (TTL, XML, ...)
** Provide statistics about Guix over time (with graphs)