Practical steps to splitting out opentelemetry-ruby-contrib?

[fbogsany]

We want to split out non-core gems to an open-telemetry/opentelemetry-ruby-contrib repo. What are the steps we need to take to accomplish this split and what are some important considerations?

Which gems should be moved to the ...-contrib repo?

Should we have the same maintainers and approvers?

How do we preserve commit history?

Should we migrate issues and discussions?

We've discussed this split several times over the history of this project, but I'm not sure we've written down the expected benefits and anticipated drawbacks of splitting the gems across two repos. Let's talk about that here too, and make sure we're happy that this is the right time for this split (before 1.0).

[robertlaurin]

Which gems should be moved to the ...-contrib repo?

• Instrumentation gems -> contrib
• Instrumentation base stays with the API/SDK

I see the instrumentation gems as consumers of the api, and the instrumentation base gem (which imo is just an unofficial extension of the api). It makes sense to me for these things to stay adjacent to one another, as they are foundational pieces.

Common is less obvious to me, as discussed yesterday it's at risk of becoming a catch all dumping ground. What is it common to? Consumers of opentelemetry, instrumentation authors, api/sdk helpers? It's not totally clear what should and should not be in there yet.

[ahayworth]

I think -common should be a dumping ground only for things in the API/SDK. An -instrumentation-helpers library was mentioned yesterday and I thought that was a lovely idea.

[robertlaurin]

Should we have the same maintainers and approvers?

I'm in favour of keeping parity between both repos until a reason not to arises.

[ahayworth]

We've discussed this split several times over the history of this project, but I'm not sure we've written down the expected benefits and anticipated drawbacks of splitting the gems across two repos. Let's talk about that here too, and make sure we're happy that this is the right time for this split (before 1.0).

An attempt at listing out some benefits and drawbacks:

Expected Benefits

• If (and only if) we create separate/additional maintainers for the -contrib repo, then we can reduce maintainership burden.
• The decoupling of 1.0.0 for the api/sdk and 0.NN for instrumentation (and misc. things) will make more sense, in some ways.
• Community members who only wish to track the progress of the api/sdk can filter their attention a bit better.
• (from @robertlaurin) Separating the repos at very least makes us consistent with the other language implementations.

Anticipated Drawbacks

• Multiple repositories can be a confusing experience for newcomers to the project. For example, I have a very hard time navigating the opentelemetry-js and opentelemetry-go projects, both of which have a -contrib repo.
• If we split discussions and/or issues, we may find ourselves transferring issues back and forth as people may will open issues in the wrong place. It may also cause difficulty if issues arise that are a legitimate cross-repo concern, since it won't be clear where to actually discuss that.
• We will need to maintain multiple sets of "infrastructure" - such as rubocop rules, documentation, and CI settings.

I think there are additional possible drawbacks in addition to these, but these are the ones I feel confident that will arise. I can't be so certain about additional drawbacks - maybe they'll happen, maybe they won't.

---

If there are additional benefits and drawbacks, I'll update the comment here with them as they're mentioned.

I should say that listing out the benefits, for me, was helpful: I'm not sure that this is a great idea. For at least the benefits I outlined, we could achieve those in ways that do not trigger the probable downsides. If there are additional benefits, of course, I would probably feel differently.

[robertlaurin]

If we split discussions and/or issues, we may find ourselves transferring issues back and forth as people may will open issues in the wrong place. It may also cause difficulty if issues arise that are a legitimate cross-repo concern, since it won't be clear where to actually discuss that.

This is the point I'm not looking forward to, I doubt everyone coming to report issues will immediately know where the issue should be reported and we'll have added toil to shuffling issues to the correct places.

We could probably mitigate this by only allowing issues on one repo and having the other point to it. That is until the two repos diverge a bit, and we end up with a different set of maintainers/approvers for each repo. At which time it probably makes sense to have separate issue trackers.

[robertlaurin]

Another benefit worth stating explicitly is that separating the repos at very least makes us consistent with the other language implementations.

[arielvalentin]

Reviving this discussion:

Which gems should be moved to the ...-contrib repo?

Anything that is not considered part of the core requirements for the API, SDK, or website_docs

• Resource Detectors
• Automatic Instrumentation Libraries
• Examples related to Automatic Instrumentation
• OTTrace and X-Ray Propagators

Should we have the same maintainers and approvers?

I think at a minimum you all should add existing active maintainers to the repo. I think you should reach-out to inactive maintainers to see if they have any capacity to commit to maintaining the contrib repo.

I would like to be added as a contrib maintainer.

How do we preserve commit history?

There is guidance here but I think we should consider this process:

• Create a new subdirectory for contrib
• Copy all source into the contrib subdirectory
• Use git filter-repo to restructure the repository using the contrib directory as the target
• Push changes to the upstream contrib repo
• Delete the contents of the contrib directory in the opentelemetry-ruby repo and leave a README tombstone with a link to contrib

Some challenges I have run into with git filter-repo is one person is will receive credit for a single commit making git blame a bit unhelpful. I have also had issues with PGP signatures but I am sure that was a PEBCAK problem.

Should we migrate issues and discussions?

I have mixed feelings about this. I find tracking issues in multiple repos to be unpleasant but most folks I work with are of the opposite opinion. That being said, I am willing to go with the majority in the group prefer.

[robbkidd]

In the SIG meeting today, we talked about and formed a proposal for first steps:

1. Create a new contrib repo (Matt Wear)
2. Attempt copying only the not-OTel propagators to the new home.
3. Confirm the not-OTel propagators test and release from the contrib repo.
4. Remove the not-OTel propagators from original repo.

We expect the propagators don't have as many potential changes during a repo transition period and there are no pull requests currently open modifying them. (Someone might double-check me on that.) This would provide multiple gems—2 (ottrace, xray)? or 4 (+ b3, jaeger)?—to act as the foundation upon which to confirm and tweak test and release automation in the new home.

I believe it is possible to still preserve pertinent git history, even if we piecemeal the migration, so long as we're frankengrafting top-level directories together at the root.

[mwear]

This step is done:

1. Create a new contrib repo (Matt Wear)

The new repo is here: https://github.com/open-telemetry/opentelemetry-ruby-contrib. We have the same set of maintainers from this repo + @arielvalentin and @ericmustin. Let me know if we need any adjustments and we can refine as we go.