Define a nightly macro with a nightly githash as input

[ehelms]

Requires theforeman/obal#320 to work, as this allows us to define and control the nightly macro here in the project itself and allow Obal to be more generic.

Currently we run obal nightly on actual nightly jobs but obal release for packaging PRs, these have disjointed workflows and obal nightly (https://github.com/theforeman/obal/blob/master/obal/data/playbooks/nightly/nightly.yaml) is trying to do a lot that is not required anymore; it was working around tito whereas with tito out of the picture we have more direct control and methods.

This will allow us in our CI to simply set -e nightly_githash=whateverweant and it will get incorporated into the SRPM. Example: http://koji.katello.org/koji/taskinfo?taskID=530916

[ehelms]

Now the tricky bit is how in our generic PR jobs to determine and then copy over the commit artifact to set the nightly hash, in our nightly we do:

                                    artifact_path = "${pwd()}/artifacts"
                                    copyArtifacts(projectName: source_project_name, target: artifact_path)
                                    commit_hash = readFile("${artifact_path}/commit")

Given this data is known to the configuration within package manifest, I am thinking I may have to build more knowledge into Obal via either:

1. Teach the srpm module how to fetch a git hash from some location and set the nightly macro itself
2. An obal command that can fetch a hash from some location that is then used as input to the scratch/release commands

The downsides of this being it teaches Obal more things that are specific to the nightly workflow, the upside being it keeps CI and local use of commands congruent based on configuration.

The CI approach I can think of would be to keep a hash of package to project mappings and if a package to be built is in the hash, then it copies over and reads the commit hash just like the nightly jobs do today and aligns those workflows. We just have to keep that hash (not that it changes much) within CI.

[ekohl]

I was thinking along the same lines with #8412: getting the actual commit hash is tricky.

[evgeni]

this makes the macros var always available and I am not 100% you had intended this? (if you did, cool, just wanted to explicitly point it out)

[evgeni]

I had to try it out to realize the else omit actually wipes the whole thing when nightly_githash is not set.

Suggested change

	nightly: "{{ '.' + lookup('pipe', 'date +%Y%m%d%H%M%S') + 'git' + nightly_githash[:7] if nightly_githash is defined else omit }}"
	nightly: "{{ ('.' + lookup('pipe', 'date +%Y%m%d%H%M%S') + 'git' + nightly_githash[:7]) if nightly_githash is defined else omit }}"

IMHO this makes it more readable?

[evgeni]

The more I look at it, the less I understand how today obal scratch rubygem-katello (or any other nightly pkg) is even working, as I don't see where it is getting the source from?!

(It's obvious for the foreman-x-develop-release jobs, but PR scratch/release doesn't use that logic)

[ehelms]

The more I look at it, the less I understand how today obal scratch rubygem-katello (or any other nightly pkg) is even working, as I don't see where it is getting the source from?!

The source location is defined in the package_manifest, which then is passed to build_srpm (https://github.com/theforeman/obal/blob/master/obal/data/roles/build_package/tasks/koji_build.yml#L42-L48) which under the hood does (https://github.com/theforeman/obal/blob/master/obal/data/modules/srpm.py#L115-L121) to fetch the remote source and then use it during the SRPM build process. That is why, on our release branches we clean out those variables so prevent that triggering. I would do the same for this nightly: entry in the macros.

[evgeni]

Well, it would have certainly helped, if I didn't look at the manifest in rpm/3.3 🙈