Eliminate --pkgid and --hdrid commandline switches

[pmatilai]

As pointed out by @dralley in #2374 (comment): we support indexed db queries for "pkgid" and "hdrid", which are actually backed by SIGMD5 and SHA1HEADER tags. These aren't really security sensitive because they're just an identifier in this context, but both will be gone in the v6 format, so we need to eliminate these indexes based on obsolete crypto, and if pkgid/hdrid concepts are to be preserved (I don't know anybody using them so dunno), figure out something that doesn't have an expiration date like the crypto functionality nowadays.

AC:

• --pkgid and --hdrid cli switches and corresponding documentation is removed
• RPMDBI_SIGMD5 and RPMDBI_SHA1HEADER are documented as deprecated

[dralley]

(I don't know anybody using them so dunno)

@Conan-Kudo I believe you may have mentioned in the past that the MD5 headers (pkgid) are used in some build systems?

[Conan-Kudo]

Yes. I think Koji did at one point, and OBS does last I checked.

[pmatilai]

Yes various places show it because ... it's there. And course, for the longest time that was the only checksum kind of thing that we had on the payload.

[pmatilai]

This is basically three different but related parts:

• the db indexes themselves
• the --hdrid and --pkgid query flags
• RPMTAG_PKGID and RPMTAG_HDRID tag aliases

Then there's also RPMTAG_SOURCEPKGID that's still md5 but that is a separate topic (#3329 )
And, tools like koji will need to be updated, so there are a whole bunch of external dependencies. Koji actually does a low-level read of the rpm package by itself, instead of using bindings. I guess it needs this to handle stuff that system rpm doesn't, but ouch. Filed https://pagure.io/koji/issue/4200

[pmatilai]

Split the pkgid/hdrid tag aliases to a separate ticket too, they're not actually used for the database operation: #3330

[pmatilai]

And, from there we get to something resembling an AC:

• RPMDBI_SIGMD5 and RPMDBI_SHA1HEADER indexes are not generated for new databases. In existing databases, stale references must not be left behind.
• --hdrid and --pkgid cli switches are removed - in my >= 20 years of rpm, I've yet to see anybody actually use these

Thoughts?

[dmnks]

• --hdrid and --pkgid cli switches are removed - in my >= 20 years of rpm, I've yet to see anybody actually use these

Is the removal of these two switches settled then? Also, isn't the new #3330 related to this in a way? I.e. if we end up keeping the aliases, shouldn't we also keep the CLI switches?

[dmnks]

Well, OK, that's why you were asking for feedback in the first place (while you also justified the removal) 😄

[dmnks]

I also think that removing obscure functionality makes sense unless we know of some users (and we don't here). It can always be added back in the future if there's enough interest...

[pmatilai]

The PKGID and HDRID tags are not used for the index generation, or query. The db index uses RPMDBI_SIGMD5 and RPMDBI_SHA1HEADER, which "happen to be" equivalent to RPMTAG_SIGMD5 and RPMTAG_SHA1HEADER.
so the PKGID and HDRID tags could be removed and I think rpm would still compile and work, something external may be using those though.

So while they appear related, they're much less so than you'd initially think - I too thought it was more closely bound.

[dmnks]

Oh, looking at the code here too, you're right.

[pmatilai]

So in other words, its two practically unused tags, practically unused cli-switches, backed by two practically unused db indexes 🤦 😆

We could keep --pkgid and --hdrid without the indexes and just do a brute-force search, but there seems little point in preserving this functionalty. Who wants to query the db based on an header hash they likely don't know even exists in the first place?

[dmnks]

Yup, makes sense now, thanks 😄 As for the first AC point:

In existing databases, stale references must not be left behind.

This part still sounds quite vague (to me)...

[pmatilai]

In existing databases, stale references must not be left behind.

This part still sounds quite vague (to me)...

Basically that means in existing databases, we need to keep updating the indexes. Minimally, we need to update the index on package removal. If we just drop the code, we could leave orphans in the index, and that cannot be allowed, so it's a hard requirement. If we actually cared about this functionality we'd require maintining the index for new items on existing dbs too, but since we dont, just taking care of orphans seems enough to me. Whether it actually makes sense to implement it that way or preserve both, I don't know, I haven't really looked (trying to play brave 😄 )

[pmatilai]

Hmm. On a second thought: we could simply just remove the query switches for now, and leave the db indexes to die of old age, to be buried in rpm 7.0. That preserves them for existing databases which in theory matters because we'll be dealing with v4 databases for years to come. In new ones they'll eventually just grow empty because the data is not there. These indexes are not expensive to maintain in any way, and actually trying to remove them is far more tricky than just letting them be.

[pmatilai]

So... that would make an AC something like:

• --pkgid and --hdrid cli switches and corresponding documentation is removed
• RPMDBI_SIGMD5 and RPMDBI_SHA1HEADER are documented as deprecated

That would let us off the hook with minimal effort, this isn't a very interesting item from v6 perspective.

[dmnks]

Ack, this looks like the right amount of effort for such a tiny thing, indeed.