Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reproducible release tarballs #1550

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Reproducible release tarballs #1550

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
17b1a28
build: Sort distributekernel METALOG when using -DNO_ROOT
patmaddox Dec 13, 2024
a6fbdd7
build: Set METALOG time when using -DWITH_REPRODUCIBLE_BUILD
patmaddox Dec 13, 2024
a5eb00d
build: Wait to build mandoc.db when making distributeworld
patmaddox Dec 13, 2024
6bc903d
release: Document reproducible release tarballs
patmaddox Dec 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 9 additions & 11 deletions Makefile.inc1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1903,25 +1903,22 @@ distributekernel distributekernel.debug: .PHONY
false
.endif
mkdir -p ${DESTDIR}/${DISTDIR}
.if defined(NO_ROOT)
@echo "#${MTREE_MAGIC}" > ${DESTDIR}/${DISTDIR}/kernel.premeta
.endif
rm -f ${DESTDIR}/${DISTDIR}/kernel.premeta
${_+_}cd ${KRNLOBJDIR}/${INSTALLKERNEL}; \
${IMAKEENV} ${IMAKE_INSTALL:S/METALOG/kernel.premeta/} \
${IMAKE_MTREE} PATH=${TMPPATH:Q} ${MAKE} KERNEL=${INSTKERNNAME} \
DISTBASE=/kernel DESTDIR=${INSTALL_DDIR}/kernel \
METALOG=${METALOG:S/METALOG/kernel.premeta/} \
${.TARGET:S/distributekernel/install/}
.if defined(NO_ROOT)
@sed -e 's|^./kernel|.|' ${DESTDIR}/${DISTDIR}/kernel.premeta > \
${DESTDIR}/${DISTDIR}/kernel.meta
echo "#${MTREE_MAGIC}" > ${DESTDIR}/${DISTDIR}/kernel.meta
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to keep the @s presumably

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO the @s are bugs. These aren't steps with should be hiding from the caller.

sed -e 's|^./kernel|.|' ${DESTDIR}/${DISTDIR}/kernel.premeta | \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does the unsortedness happen only with -j? I'd be a bit surprised by nondeterminism without -j.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure - I've never had the patience to run without -j.

One potential source of non-determinism is when file lists are generated rather than declared. For example, some of the items returned by grep -R '=.*find' --include 'Makefile*' sys could potentially be non-determinstic. But I can't say for sure.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like those are all in sys/contrib (and so not actually used in our build), but indeed anything that returns files in filesystem order could be an issue.

I expect -j will result in nondeterminism and so the sorting is necessary, mainly just wondering if this is the only source of nondeterminism here.

${METALOG_SORT_CMD} >> ${DESTDIR}/${DISTDIR}/kernel.meta
.endif
.endif
.if ${BUILDKERNELS:[#]} > 1 && ${NO_INSTALLEXTRAKERNELS} != "yes"
.for _kernel in ${BUILDKERNELS:[2..-1]}
.if defined(NO_ROOT)
@echo "#${MTREE_MAGIC}" > ${DESTDIR}/${DISTDIR}/kernel.${_kernel}.premeta
.endif
rm -f ${DESTDIR}/${DISTDIR}/kernel.${_kernel}.premeta
${_+_}cd ${KRNLOBJDIR}/${_kernel}; \
${IMAKEENV} ${IMAKE_INSTALL:S/METALOG/kernel.${_kernel}.premeta/} \
${IMAKE_MTREE} PATH=${TMPPATH:Q} ${MAKE} \
Expand All @@ -1930,9 +1927,10 @@ distributekernel distributekernel.debug: .PHONY
METALOG=${METALOG:S/METALOG/kernel.${_kernel}.premeta/} \
${.TARGET:S/distributekernel/install/}
.if defined(NO_ROOT)
@sed -e "s|^./kernel.${_kernel}|.|" \
${DESTDIR}/${DISTDIR}/kernel.${_kernel}.premeta > \
${DESTDIR}/${DISTDIR}/kernel.${_kernel}.meta
echo "#${MTREE_MAGIC}" > ${DESTDIR}/${DISTDIR}/kernel.${_kernel}.meta
sed -e "s|^./kernel.${_kernel}|.|" \
${DESTDIR}/${DISTDIR}/kernel.${_kernel}.premeta | \
${METALOG_SORT_CMD} >> ${DESTDIR}/${DISTDIR}/kernel.${_kernel}.meta
.endif
.endfor
.endif
Expand Down