Commit graph

52 commits

Author SHA1 Message Date
gerblesh
aee351fcbf
feat: add support for nested modules (#152)
* feat: add support for nested modules

* chore: test out module nesting

* fix: made build function

* revert: move back to old recipe.yml configuration

* fix: formatting in recipe.yml

* style: add newline between modules
2023-09-19 06:32:16 +00:00
gerblesh
cc90a91733
feat: added systemd and files module (#142)
* feat: add files and systemd module

* fix: yaml formatting

* fix: yaml formatting

* fix: remove comment completely

* fix: yaml formatting

* docs: add back inline comment

* reformat: rename variables

* fix: fix systemd escaped string

* fix: fix systemd service formatting with printf

* fix: attempting to fix systemd module problems

* chore: remove debug config and code from systemd module

* docs: added WIP docs for systemd, reworked files README

* docs: added more detail for systemd module

* docs: update READMEs to be more consistent

* docs: remove unneeded sentence

* docs: remove unneeded sentence

* chore: fix issues described in PR review

* docs: fix markdown formatting

* docs: fix markdown formatting

* docs: better markdown
2023-09-05 15:56:42 +00:00
ER
2740320a51 fix: don't output yaml array as json 2023-08-26 17:20:45 +03:00
ER
bca8b81e16 docs: a ginormous amount of docs for the modules 2023-08-26 14:05:42 +03:00
ER
14d78ee2a7 fix: can't set readonly var after declaration
(duh)
2023-08-13 18:14:06 +03:00
ER
38dda90a23 refactor: rename fedora version to os version
this change makes sense for when in the future it's possible to
build other OS's with this same system
it was done now, because i'm bundling a bunch of breaking changes
into a single update
2023-08-13 18:00:58 +03:00
ER
8f0aa9cec9 chore: export more vars as readonly 2023-08-13 17:29:11 +03:00
ER
8e80937db0 docs: better inline docs and prints 2023-08-13 17:20:25 +03:00
ER
b2034c7a1c feat: rpm-ostree module 2023-08-13 17:05:06 +03:00
ER
9ad0048fc3 refactor: global get_yaml_array function 2023-08-13 16:59:23 +03:00
ER
18d1b3d4cd Revert "refactor: export get_yaml_array"
This reverts commit c27685fae2.

(get_yaml_array can't be used in modules since it reads from
the $RECIPE_FILE)
2023-08-13 16:15:22 +03:00
ER
c27685fae2 refactor: export get_yaml_array 2023-08-13 16:12:57 +03:00
ER
cb0631831c chore: export config directory 2023-08-13 16:06:44 +03:00
ER
59e16045a8 fix: remove trailing slashes from directory vars 2023-08-13 15:24:11 +03:00
ER
d0c6deeb9b feat: running modules from files 2023-08-13 15:20:34 +03:00
ER
057b45a45f refactor: remove get yml string, add configdir var 2023-08-13 15:19:55 +03:00
ER
245fc69b13 chore: launch modules, detect if module inline 2023-08-13 15:09:40 +03:00
ER
2b633ff5c8 fix: make get_yaml_array work
apparently it's not possible to return an array from a function
(eventhough doing so would be nice)
2023-08-13 15:00:21 +03:00
ER
e537d657e5 chore: investigate reading module array 2023-08-09 21:02:06 +03:00
ER
7b9faf690c fix: bad yq command syntax
* I'm never trusting AI again!
2023-08-09 20:57:03 +03:00
ER
4325e65c10 chore: refactor code to support new structure
* this doesn't do much yet and a lot of code was deleted,
functionality will be gradually added back
2023-08-09 20:27:42 +03:00
ER
9950409702 chore: outline rewrite without code changes
* this does not work yet
* comments & documentation will be added once the rewrite is complete
2023-08-09 19:59:04 +03:00
Arcitec
e8b5be6e83 fix!: optimize container layers and reduce image size
Every individual RUN, COPY and ADD action creates an extra container layer, so there was plenty of room for improvement in our Containerfile.

This optimization gets rid of 4 useless layers from our final container image, and shrinks the final OCI download size as follows:

- Removing the "mkdir /tmp/scripts" layer. It's not necessary to manually create the target directory for the container copy action.

- Removing the manual "chmod +x" for the scripts, and putting that step inside "build.sh" instead.

- Removing the manual copying of "build.sh", by instead placing it at "scripts/build.sh" so that it's automatically copied together with all the other scripts in one layer instead.

- Removing the separate "chmod +x build.sh && run build script" step by merging it with the "cleanup temp files and then finalize the container" step, so that we don't create a pointless extra filesystem layer just for the build.sh script execution.

These changes also reduce the size of the final image, because we're cleaning up the image in the exact same step that we run the "build.sh". If we didn't combine these steps, we'd still be keeping a useless extra layer with all the /tmp/ and /var/ junk files that were left over after the build.

Most seriously, the "/var/cache" folder contained copies of ALL RPM FILES that build.sh installed via "rpm-ostree install". This meant that we were generating a very big layer with a lot of junk data that shipped in the final image.

Our build now only generates 7 layers (instead of 11), and users will have a much smaller OCI download since we aren't shipping the cached RPM "build leftovers" or temp files via useless extra layers anymore.
2023-05-20 08:07:46 +00:00
Arcitec
63a5e3583b feat: refactor build.sh to use the new yafti launcher
This implements the new "yafti" launcher and ties everything together:

- The "/etc/profile.d" script is now a symlink to an immutable location, instead of an actual plaintext file, to prevent risk of user systems breaking due to ostree thinking that the script had been modified. We can now be certain that systems always work properly when switching between distros or uBlue flavors.

- Yafti is only installed if the "yafti" flag is enabled.

- If the user disables the "yafti" flag, we completely wipe out the entire yafti.yml configuration, all of our new "yafti autostart" components, and the "profile.d" symlink, so that nothing remains on the system image. If an upstream image installed yafti itself in `/usr/bin`, then we leave that component alone though (because "pip uninstall" and the potential errors it might throw seems like a pointless hassle just for a tiny, unused binary).
2023-05-14 18:18:18 +03:00
Arcitec
8ee996722f fix!: move yafti.yml to standardized location 2023-05-10 21:44:01 +03:00
Arcitec
077e7243f2 fix!: move recipe.yml to standardized location 2023-05-10 21:44:01 +03:00
Arcitec
55ff6363be feat: implement a very flexible runner for "pre" and "post" scripts
This new functionality now makes it possible to execute scripts at the start or end of the build process, while also being super simple to expand to add further script stages in the future.

It also supports effortless reuse of scripts for multiple stages, since the scripts are now executed with the "current stage" as their 1st argument, to allow them to easily determine which stage they're running in.
2023-05-10 18:44:25 +03:00
Arcitec
946f3d82ee fix: automatically determine current fedora version during build
This fixes the issue where someone specifies `fedora-version: latest`, which won't be known until build-time.

I also added a small "welcome" banner to the build log. It's really just there to retain a somewhat contrived use-case example for how to use `get_yaml_string()`, for other programmers who want to extend this in the future.
2023-05-10 18:44:25 +03:00
Arcitec
4ca33a3fe3 feat: streamlined YAML configuration for RPM-related settings
All RPM-related settings are now within their own YAML sub-category, to improve readability.
2023-05-10 18:44:25 +03:00
Arcitec
baa8c47ffa feat: automatic repo version selection via YAML configuration
You can now define your custom repos with the `%FEDORA_VERSION%` variable, to automatically use the correct repo version, so that you never have to maintain their custom URLs again in the future.
2023-05-10 18:44:25 +03:00
Arcitec
33ab88f794 feat: add function for reading individual YAML strings
The recipe file is now also moved to a separate variable, and we use `--` to ensure that yq never misinterprets any custom arguments as `-flags`.
2023-05-10 18:44:25 +03:00
Arcitec
bf19fa5eca feat: implement effortless RPM removal via YAML configuration
You can now easily remove RPMs from your custom image, without having to edit the build.sh script.

This also changes the old "rpms" config key, to "rpm-install", for consistency with the new setting.
2023-05-10 18:44:25 +03:00
Arcitec
5f072b8b67 fix: faster installation of custom RPMs
Thanks to leveraging Bash arrays, we are now able to use parameter expansion to perform every package installation as one rpm-ostree operation instead of individual ones. This will give us a huge build speedup.

Bash properly expands every array element as safely quoted arguments, complete with support for spaces and special characters.
2023-05-10 18:44:25 +03:00
Arcitec
07cbe2cc08 fix: extremely robust handling for YAML data fetching
This new technique is way easier to use, more maintainable, and safely handles special characters and spaces in the YAML data. It will also enable other features, such as the rpm-ostree improvement coming in the next commit.

The function name is `get_yaml_array()` because it literally reads it into a Bash array, by the way. Don't change the function name in the future. We may need other functions to read values into strings, etc.
2023-05-10 18:44:25 +03:00
Arcitec
20e0a06588 fix: remove extremely verbose "x" flag
The debug "x" flag that prints every command is really excessive.
2023-05-10 18:44:25 +03:00
Arcitec
d09dd76243 feat: install repos first, so that they're available for scripting
This ensures that custom scripts have the flexibility to work with the repos that have been added.
2023-05-10 18:44:25 +03:00
Arcitec
f24c4072fd fix: use proper shebang line
This is the most portable shebang line and should always be used.
2023-05-10 18:44:25 +03:00
Arcitec
44660085eb chore: clean up sentences to make them easier to expand
Lowercase, run-on sentences are a nightmare to expand later, because the lack of sentence structure means you can't just add more lines and have a nice flow anymore.

Let's add some punctuation and grammar.
2023-05-10 18:44:25 +03:00
Arcitec
4b9fe0b05b fix: add flags to ensure that build aborts if there are errors 2023-05-10 18:44:25 +03:00
Eino Rauhala
c537104ba2
feat: add extra repos field to recipe (#52)
* feat: add extra repos field to recipe

* fix: semicolon before "done" keyword in shell

* fix: don't put repos in /usr/etc/
2023-04-28 13:55:37 +00:00
Eino Rauhala
b3f7f07d6c
fix: only create yafti grp when flatpaks defined (#46) 2023-04-26 16:49:26 +00:00
Eino Rauhala
95ff830b49
fix: moving recipe back to a directory available on the host (#47)
* fix: mv recipe to a dir available on the host

* docs: add notice for recipe editing on host

* fix: replace /tmp/ublue-recipe.yml with /usr/etc/.
2023-04-26 16:19:31 +00:00
Eino Rauhala
085d26aa1a
fix: prevent /etc/ merge issues (#43)
* fix: prevent /etc/ merge issues
discussed at length in #28
also including tons of comments about the issue

* fix: change references of /etc/ to /usr/etc
2023-04-26 15:23:43 +00:00
Emmett Wasik (Viv)
37143b0e23
feat: add per-recipe shell script support (#38) 2023-04-24 18:40:51 +00:00
ER
e9c31aba83 chore: put ublue-recipe in /tmp
-> there's no reason to send the recipe to the client
as it is only used in the build stages
2023-04-02 15:51:18 +03:00
ER
773fc23804 fix: loop over lines, functional var substitution 2023-04-02 15:31:01 +03:00
ER
97ee7169f4 fix: add shebang 2023-04-02 12:48:14 +03:00
ER
616c59c90b fix: remove unneeded escape 2023-04-02 12:48:03 +03:00
ER
5a98ef6532 fix: remove -y flag from yq; not in go version 2023-04-02 12:38:06 +03:00
ER
0cf6b13b6c feat: yafti package group for recipe.yml
read .flatpaks[] in recipe.yml
and add declared packages to their own
package group in yafti
2023-04-02 12:24:32 +03:00