Support default kernel arguments #479

vtolstov · 2016-08-31T11:11:16Z

How can i add some things to kernel cmdline when compose tree?
Or how i control cmdline when boot ostree system?

cgwalters · 2016-08-31T11:24:16Z

ostree admin deploy --karg=. Which is used by e.g. the Fedora/CentOS Anaconda installer

vtolstov · 2016-08-31T11:31:40Z

Thanks, i miss it with using rpm-ostree =(.
This is really hard use rpm-ostree without ostree admin =).
But not ostree admin upgrade works fine, because i'm deploy ref via ostree admin deploy and it not complain about unknown refspec

vtolstov · 2016-09-07T08:49:42Z

Last qeustion - i'm add custom cmdline with treefile (dracut additional cmdline and specify kernel cmdline to dracut). Why after composing tree i don't have this options in dracut config and grub entries does not contains this line?

cgwalters · 2016-09-07T12:36:18Z

We don't currently support that, but we could.

cgwalters · 2016-09-07T12:36:47Z

Basically for now though, use the kickstart https://github.com/rhinstaller/pykickstart/blob/master/docs/kickstart-docs.rst#bootloader verb.

paulvt · 2018-02-26T19:48:58Z

Another thing to note is that --karg allows one to update or set a kernel boot argument, but not to remove one.
For know, I use --karg=foo= to at least clear it, but the relevant part of the kernel does complain it cannot handle the value ''.

dustymabe · 2018-11-07T13:50:18Z

Marking this as medium difficulty, although I suspect it might lean towards hard. @cgwalters WDYT?

cgwalters · 2018-11-13T14:36:05Z

Marking this as medium difficulty, although I suspect it might lean towards hard. @cgwalters WDYT?

As a general rule I think for ostree/rpm-ostree, the difficulty level is weighted to a much greater degree by the person's knowledge (e.g. bootloaders, C or Rust, whether it requires learning the test framework etc.) compared to other projects. I understand the goal of labeling the difficulty level of issues, but I'm not sure it's worth trying to figure out the exact levels here. If someone wants to dive in they should feel free to ask questions on an issue I'd say.

dustymabe · 2018-11-13T16:25:12Z

Yep. It's hard to get a standard metric. I'm just trying to get a ballpark estimate so that we can get an idea and group issues into buckets. Maybe there will be some easier medium tasks and some harder medium tasks, but that's probably ok.

Since we have ~150 open issues, if we have buckets of good-first-issue and difficulty/medium tasks then we have a starting point and also a ramp up for new contributors.

If you'd like me to stop this endeavor I will.

rfairley · 2018-11-13T16:52:31Z

Just to give a little input, I do like the good-first-issue and difficulty/medium split to roughly differentiate the size of the change. good-first-issue usually means a 1-10 line change, mostly involving rearranging existing logic and possibly adding a test, and difficulty/medium might be adding something new but still only touches one or two components. But I can see how the difficulty/medium would get easier once one is familiar with the codebase, so the size of the change doesn't necessarily indicate difficulty.

I find the difficulty/medium label is still handy for quickly looking up issues that should be straightforward but more involved than the good-first-issue type - as a "next step" once one has done a good-first-issue and is more familiar with the codebase.

I agree that asking is the best way to get an idea of the difficulty and if one has the specific prereq. knowledge for it. difficulty/medium might just be a way to group those tasks that one might consider doing after completing a good-first-issue.

cgwalters · 2018-11-13T18:51:56Z

I have absolutely no problem with you or anyone trying to classify the issues! I just feel like for me personally the benefit/time ratio is too low to worry too much about whether an issue is medium/high or whatever.

dustymabe · 2018-11-13T19:15:59Z

+1

I'll stop asking for input, but will reserve the right to be slightly or significantly wrong in my estimates :)

rfairley · 2019-03-14T15:31:48Z

Some thoughts after reading over the code around kargs:

We have mentioned having OSTree commits contain the default kargs. Could this take the form of a file containing the default kargs which gets committed to the ostree, e.g. /usr/lib/ostree/kargs? OSTree would read the default kargs from that file during a deployment, and append the kargs to the state file at "/run/ostree/staged-deployment", and the BLS fragment on reboot.

For rpm-ostree treefiles, I'm thinking we could add a field e.g. default-kargs or just kargs, where the user specifies a list of key-value pairs for each karg (arg, value). The key-value pairs would be written to /usr/lib/ostree/kargs at compose time, and that file would be committed to the ostree.

This still means mangling the defaults with any kargs specified with ostree admin deploy --karg= or rpm-ostree kargs - I'm not sure of how to keep those separate other than storing them as suggested previously. There are other ways we could make the mangling more visible, e.g. a switch to print out the default, user-specified, and mangled kargs during ostree admin --karg= during the deploy, or have additional fields containing the default and user kargs to the state file in /run.

If the user wanted to completely override the default kargs, we could also support /etc/ostree/kargs which would shadow /usr/lib/ostree/kargs.

How does the file idea sound?

cgwalters · 2019-03-14T16:15:19Z

Maybe /usr/lib/ostree-boot/kargs? Today that gets copied into /boot but we could teach the library to skip that one file?

Another bikeshed issue I see here is: what is the file format of /usr/lib/ostree/kargs? Newline-separated plain text? JSON? Do we need to care about kargs with non-UTF-8 data?

If the user wanted to completely override the default kargs, we could also support /etc/ostree/kargs which would shadow /usr/lib/ostree/kargs.

Well this gets tricky since today one can configure kernel args, they live the deployment's BLS fragments in /boot. But that's the "merged" state, so it mixes user-supplied and system default which we want to get away from. So probably indeed /etc/ostree/kargs would make sense.

There's a very special hazard with this though which is trying to do an upgrade from an existing commit to one which uses this - if the "source of truth" of kargs switches to /{/usr,/etc}/ostree/kargs then we would drop any kargs which the user explicitly configured in their previous deployment.

Maybe what we need to do is add something like:

# ostree: generated from /usr/lib/ostree/kargs and /etc/ostree/kargs
options rd.lvm.lv=sb/root root=/dev/mapper/sb-root ostree=/ostree/boot.0/fedora-silverblue/78f3e4c34baaa9af28885bb3b6d92db7e59b73a9261d68a566de1c66230534dc/0 coreos.oem.id=qemu

And that comment acts as a marker that says "use deployment for source of truth on upgrades, not the merge deployment's kargs"?

rfairley · 2019-03-14T19:52:18Z

Maybe /usr/lib/ostree-boot/kargs? Today that gets copied into /boot but we could teach the library to skip that one file?

Sounds good to me! Makes sense to keep the files relating to boot in ostree-boot where it'll be seen close to the other boot files.

what is the file format of /usr/lib/ostree/kargs? Newline-separated plain text? JSON? Do we need to care about kargs with non-UTF-8 data?

Had been wondering between those formats, was also thinking the kargs file could be a keyfile (to the Desktop Entry Specification). I think during parsing to a GKeyFile this would help enforce a definite "key-value pairs only, one per line" format. Though validating JSON could accomplish that too, and it'd be clean to work with translating from the treefile.

Is the the encoding of the kernel parameters dependent on the locale of the system, or is there software that might specially need non-UTF-8 in the kargs? If we did support non-UTF-8, would plain text be a better option (with some switch to specify the encoding of the kargs)?

# ostree: generated from /usr/lib/ostree/kargs and /etc/ostree/kargs

If we had this comment be a marker, would libostree delete that comment and apply the new kargs if the user did rpm-ostree kargs or ostree admin deploy --kargs=? Then with the comment not present, the merge deployment becomes the source of truth (like it is today). If the comment is present, then OSTree would regenerate from the deployment (i.e. /usr/lib/ostree/kargs and /etc/ostree/kargs).

To reset the source of truth, we could also have a command like rpm-ostree kargs --reset or rpm-ostree kargs --regenerate which would rewrite the BLS config from /usr/lib/ostree/kargs and /etc/ostree/kargs.

rfairley · 2019-04-05T13:32:19Z

Just another question, which isn't blocking the initial patch I'm working on, but came up while thinking about the design:

Should the host config (/etc/ostree/kargs) have an equivalent of rpm-ostree kargs --delete when merging/overriding with the base config (/usr/lib/ostree-boot/kargs)? I'm not sure of a specific need for this, but I wonder if a user might want full control of the kargs through their host config to override and delete kargs in the base config, while using the etc/usr configs as a source of truth. That way the host would still be "subscribed" to updates to the /usr/lib/ostree-boot/kargs file in newer ostrees. We could also support the --append behavior easily as well if we did this.

I imagine /etc/ostree/kargs would need a format like the following to support that:

d root=FOO # d <=> --delete
d KEY=VAL
r rd.lvm.lv=sb/root # r <=> --replace
a coreos.oem.id=qemu # a <=> --append
...

or

[kargs]
delete="root=FOO KEY=VAL ..."
replace="rd.lvm.lv=sb/root ..."
append="coreos.oem.id=qemu ..."

If the user did use rpm-ostree kargs --{append,delete,replace} or ostree admin --karg*, then the merge deployment would become the source of truth.

Instead of merging kargs from the previous deployment, by default regenerate kargs from the following config files: /etc/ostree/kargs (host config) /usr/lib/ostree-boot/kargs (base config) The base config is sourced from the repo at the commit (revision) being deployed, so that newly committed kargs will take effect immediately after upgrading. The host config is sourced from the merged /etc configuration of the new deployment, so that host edits as well as edits to /usr/etc/ostree/kargs are reflected. Kernel arguments present in the base config are appended to kernel arguments in the host config. Using the commands that modify kargs (of the form `ostree admin deploy --karg*`) will cause kargs in later deployments to be copied from the merge deployment, rather than regenerate from the config files. Resolves: ostreedev#479

rfairley · 2019-04-11T22:47:20Z

Thinking about rpm-ostree integration (Re: #479 (comment)):

For rpm-ostree treefiles, I'm thinking we could add a field e.g. default-kargs or just kargs, where the user specifies a list of key-value pairs for each karg (arg, value). The key-value pairs would be written to /usr/lib/ostree/kargs at compose time, and that file would be committed to the ostree.

It looks like an easier way to add /usr/lib/ostree-boot/kargs at compose time is to use the add-files field (https://github.com/projectatomic/rpm-ostree/blob/master/docs/manual/treefile.md#treefile). Will test this out.

Instead of merging kargs from the previous deployment, by default regenerate kargs from the following config files: /etc/ostree/kargs (host config) /usr/lib/ostree-boot/kargs (base config) The base config is sourced from the repo at the commit (revision) being deployed, so that newly committed kargs will take effect immediately after upgrading. The host config is sourced from the merged /etc configuration of the new deployment, so that host edits as well as edits to /usr/etc/ostree/kargs are reflected. Kernel arguments present in the base config are appended to kernel arguments in the host config. Using the commands that modify kargs (of the form `ostree admin deploy --karg*`) will cause kargs in later deployments to be copied from the merge deployment, rather than regenerate from the config files. Resolves: ostreedev#479

Add the following config directories for setting default kernel arguments: /etc/ostree/kargs.d (host config) /usr/lib/ostree-boot/kargs.d (base config) These directories contain files whose contents consist of a karg snippet, which is a collection kernel parameter keys and values. Example of a snippet: ``` KEY=VALUE ANOTHERKEY SAMEKEY=VALUE2 FIRSTKEY=1 SAMEKEY=VALUE3 SECONDKEY=2 KEY= KEY.KEY-KEY_KEY="some space-separated values" ``` The bootconfig key `ostree-kargs-generated-from-config` indicates whether the kargs were generated by ostree from the kargs.d directories (`true`), or copied from the previous deployment (`false`). Editing the kargs through the command line (e.g. `ostree admin deploy --karg=`) will set the flag to `false`, and kargs will be copied from the previous deployment for subsequent deployments. Closes: ostreedev#479

Add the following config directories for setting default kernel arguments: /etc/ostree/kargs.d (host config) /usr/lib/ostree-boot/kargs.d (base config) These directories contain files whose contents consist of a karg snippet, which is a collection kernel parameter keys and values. Example of a snippet: ``` KEY=VALUE ANOTHERKEY SAMEKEY=VALUE2 FIRSTKEY=1 SAMEKEY=VALUE3 SECONDKEY=2 KEY= KEY.KEY-KEY_KEY="some space-separated values" ``` The bootconfig key `ostree-kargs-generated-from-config` indicates whether the kargs were generated by ostree from the kargs.d directories (`true`), or copied from the previous deployment (`false`). Editing the kargs through the command line (e.g. `ostree admin deploy --karg=`) will set the flag to `false`, and kargs will be copied from the previous deployment for subsequent deployments. Also add support for `ostree admin instutil set-kargs` so that installer programs using this command (such as Anaconda) remain managed by the kargs.d directories from the first deployment. Closes: ostreedev#479

Add the following config directories for setting default kernel arguments: /etc/ostree/kargs.d (host config) /usr/lib/ostree-boot/kargs.d (base config) These directories contain files whose contents consist of a karg snippet, which is a collection kernel parameter keys and values. Example of a snippet: ``` KEY=FOO KEYFLAG ANOTHERKEY=VALUE1 ANOTHERKEY=VALUE2 ``` Snippets are read in alphanumeric order of the karg snippet filename when generating the final kernel options from the host and base config directories. Ordering may be specified by prefixing a number, e.g. `/etc/ostree/kargs.d/4000_localhost`. The bootconfig key `ostree-kargs-generated-from-config` indicates whether the kargs were generated by ostree from the kargs.d directories (`true`), or copied from the previous deployment (`false`). Editing the kargs through the command line (e.g. `ostree admin deploy --karg=`) will set the flag to `false`, and kargs will be copied from the previous deployment for subsequent deployments. Also add support for `ostree admin instutil set-kargs` so that installer programs using this command (such as Anaconda) remain managed by the kargs.d directories from the first deployment. Closes: ostreedev#479

rfairley · 2019-06-25T18:07:48Z

In #1836 I was initially planning on optimizing ostree admin deploy --karg=, --karg-append=, etc. (which COSA uses since removing Anaconda https://github.com/coreos/coreos-assembler/pull/556/files#diff-2d426a046eb99541d19814f8fac698bbR54) to write snippets to /etc/ostree/kargs.d in order to modify kargs. However in the case of checking existence of this directory before doing a reboot (coreos/ignition-dracut#86), this would trigger an unnecessary reboot even if Ignition did not write snippets there.

(I didn't realize this when suggesting it at https://github.com/coreos/ignition-dracut/issues/81#issuecomment-494888494).

One way to get around this could be to have COSA bake the kargs snippets in a new commit, i.e. under /usr/lib/ostree-boot/kargs.d (or use add-files in the FCOS treefile) to avoid the /etc/ostree/kargs.d path. However this is restrictive - there should really be a way to specify snippets in FCOS at install time without needing a new commit (i.e. write in /etc/ostree/kargs.d), and without needing to specify kargs snippets through an Ignition config.

Ignition could write a magic file e.g. /etc/ostree/kargs.d/0000_ignition-reboot - however we want to avoid special handling in Ignition. Users also shouldn't be required to write to a special directory in their Ignition config e.g. /etc/ostree/kargs.d/ignition/<user's kargs snippets here> for the needed reboot to happen (if the reboot was triggered on /etc/ostree/kargs.d/ignition instead).

cgwalters · 2020-04-21T20:26:44Z

Was thinking about this more since doing factory reset support.

One model might be to suggest people to add default kernel arguments into the kernel binary; this is already supported with CONFIG_CMDLINE. For distributions like Fedora that want to build a generic kernel image that might be used multiple ways (e.g. FCOS vs other systems), it probably wouldn't be hard to patch the kernel to reserve some fixed space for arguments that one could write into the binary. Or, maybe instead we support "re-linking" the binary with some default args?

This would break Secure Boot signatures, but on the other hand, not validating kernel arguments is often an easy way to execute arbitrary code too....

cgwalters · 2020-04-21T20:28:41Z

When FCOS ships kargs, FCOS should still ship 1 karg per file (or possibly group dependent ones) in /usr/lib/ostree-boot/kargs.d

The current model is now /usr/lib/modules/<kver> - if we defined a file format there, it could actually also apply to traditional systems.

nullr0ute · 2020-04-22T10:53:28Z

This would break Secure Boot signatures, but on the other hand, not validating kernel arguments is often an easy way to execute arbitrary code too....

You can deal with validation of kernel args to ensure they don't change with IMA/tpm2.

Add the following config directories for setting default kernel arguments: /etc/ostree/kargs.d (host config) /usr/lib/ostree-boot/kargs.d (base config) These directories contain files whose contents consist of a karg snippet, which is a collection kernel parameter keys and values. Example of a snippet: ``` KEY=FOO KEYFLAG ANOTHERKEY=VALUE1 ANOTHERKEY=VALUE2 ``` Snippets are read in alphanumeric order of the karg snippet filename when generating the final kernel options from the host and base config directories. Ordering may be specified by prefixing a number, e.g. `/etc/ostree/kargs.d/4000_localhost`. The bootconfig key `ostree-kargs-generated-from-config` indicates whether the kargs were generated by ostree from the kargs.d directories (`true`), or copied from the previous deployment (`false`). Editing the kargs through the command line (e.g. `ostree admin deploy --karg=`) will set the flag to `false`, and kargs will be copied from the previous deployment for subsequent deployments. Also add support for `ostree admin instutil set-kargs` so that installer programs using this command (such as Anaconda) remain managed by the kargs.d directories from the first deployment. Closes: ostreedev#479 [rebased to master] Signed-off-by: Stefan Agner <stefan.agner@toradex.com>

cgwalters added the question label Aug 31, 2016

cgwalters added enhancement and removed question labels Sep 7, 2016

cgwalters changed the title ~~specify additional cmdline to kernel when compose tree~~ Support default kernel arguments Sep 7, 2016

dustymabe added the jira label Oct 12, 2018

dustymabe added the difficulty/medium medium complexity/difficutly issue label Nov 7, 2018

cgwalters mentioned this issue Jan 18, 2019

Determine how to handle automatic rollback coreos/fedora-coreos-tracker#47

Open

This was referenced Feb 1, 2019

kickstart: enable prjquota for quota enablement coreos/coreos-assembler#303

Merged

Add metal-bios and metal-uefi targets coreos/coreos-assembler#305

Merged

cgwalters mentioned this issue Mar 25, 2019

pkg/daemon: allow reboots from external componenents openshift/machine-config-operator#576

Closed

rfairley linked a pull request Apr 8, 2019 that will close this issue

Support kargs.d directories for default kargs #1836

Open

2 tasks

jlebon mentioned this issue Jun 25, 2019

dracut/30ignition: Reboot the system after ignition if kargs.d is present coreos/ignition-dracut#86

Closed

dustymabe removed the jira label Sep 5, 2019

cgwalters mentioned this issue Oct 18, 2019

Move to Fedora 31! coreos/fedora-coreos-config#200

Merged

cgwalters mentioned this issue Nov 21, 2019

kargs: Support --append and --delete simultaneously coreos/rpm-ostree#1940

Merged

jlebon mentioned this issue Dec 16, 2019

cgroups v2 strategy coreos/fedora-coreos-tracker#292

Closed

clime mentioned this issue Apr 2, 2020

ways to update grub.cfg with an OSTree update #2052

Closed

cgwalters mentioned this issue Apr 2, 2020

kernel command line arguments do not get updated #2050

Closed

pattivacek mentioned this issue Apr 3, 2020

kernel command line parameters are not being updated advancedtelematic/meta-updater#706

Closed

cgwalters mentioned this issue May 15, 2020

Consider an rpm-ostree history command coreos/rpm-ostree#1489

Closed

cgwalters mentioned this issue Jun 24, 2020

overlay/fcos: pin legacy ifnames on existing machines coreos/fedora-coreos-config#490

Merged

bgilbert mentioned this issue Jul 27, 2020

Allow rebooting after Ignition if needed coreos/ignition#1051

Open

agners mentioned this issue Oct 14, 2020

Support kargs.d directories for default kargs (rebased) #2217

Closed

jlebon mentioned this issue Jun 11, 2021

Add support for new crashkernel.conf file in kernel package coreos/rpm-ostree#2894

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support default kernel arguments #479

Support default kernel arguments #479

vtolstov commented Aug 31, 2016

cgwalters commented Aug 31, 2016

vtolstov commented Aug 31, 2016

vtolstov commented Sep 7, 2016

cgwalters commented Sep 7, 2016

cgwalters commented Sep 7, 2016

paulvt commented Feb 26, 2018

dustymabe commented Nov 7, 2018

cgwalters commented Nov 13, 2018

dustymabe commented Nov 13, 2018

rfairley commented Nov 13, 2018

cgwalters commented Nov 13, 2018

dustymabe commented Nov 13, 2018

rfairley commented Mar 14, 2019 •

edited

cgwalters commented Mar 14, 2019

rfairley commented Mar 14, 2019

rfairley commented Apr 5, 2019

rfairley commented Apr 11, 2019

rfairley commented Jun 25, 2019

cgwalters commented Apr 21, 2020

cgwalters commented Apr 21, 2020

nullr0ute commented Apr 22, 2020

Support default kernel arguments #479

Support default kernel arguments #479

Comments

vtolstov commented Aug 31, 2016

cgwalters commented Aug 31, 2016

vtolstov commented Aug 31, 2016

vtolstov commented Sep 7, 2016

cgwalters commented Sep 7, 2016

cgwalters commented Sep 7, 2016

paulvt commented Feb 26, 2018

dustymabe commented Nov 7, 2018

cgwalters commented Nov 13, 2018

dustymabe commented Nov 13, 2018

rfairley commented Nov 13, 2018

cgwalters commented Nov 13, 2018

dustymabe commented Nov 13, 2018

rfairley commented Mar 14, 2019 • edited

cgwalters commented Mar 14, 2019

rfairley commented Mar 14, 2019

rfairley commented Apr 5, 2019

rfairley commented Apr 11, 2019

rfairley commented Jun 25, 2019

cgwalters commented Apr 21, 2020

cgwalters commented Apr 21, 2020

nullr0ute commented Apr 22, 2020

rfairley commented Mar 14, 2019 •

edited