Structure for expansion of "Ultimate Geography"

[aplaice]

There are several closely-related issues. We're interested in expansions both in terms of additional notes (more "topics" — more autonomous regions, islands, lakes etc.) and additional cards per note (more info, such as pronunciations of names, maybe currencies etc.).

Some slightly chaotic questions (this is a bit of an infodump so that our discussions with @axelboc and @ohare93 aren't forgotten — I haven't had time to make this more organised :/) to be considered (at some point — IMO none of them are critical in the short or medium term):

1. How should the material be split in terms of repos?

   We're currently going with having all additional notes in "hardcore geography", but we might (possibly) later want to switch to having everything in the main deck or split more fine-grainedly.

   Also, what about extra cards per note? BrainBrew currently AFAIU doesn't support having deck material split between repos, so it'd seem that all additional info for AUG would have to stay in the ultimate-geography, at least for now. However, in the long run, we can change BrainBrew to suit our needs.

2. How should the material be split in terms of decks?

   This is related to 1 but slightly different — we can combine material from multiple repos into one deck or produce multiple decks from one repo (actually, we already do — the "standard" and the "extended" decks — though they differ only in the cards per note rather than the content of notes).

   We could have the same notes in multiple decks (e.g. both in a "Hardcore Geography" and in a "Lakes" deck).

3. How could we make the system of allowing the user choose what material to study?

   This combines 1 and 2, in a way.

   Ideally, the users would be able to pick exactly what they want. Quoting @ohare93 (I hope this is OK — I can't formulate it better):

   I keep landing on the conclusion that in order to have a system that is user friendly (not requiring pulling git repos and building a deck with Brain Brew) it has to be a selection system in an Anki add-on (CrowdAnki or otherwise). I imagine the main UG deck having a link to the repo, and a file in the repo linking to other add-on deck(s), so that this add-on could have its data populated. Then from the user selection it would use Brain Brew to mash together the deck + add-ons into a new CrowdAnki deck, and initiate a CrowdAnki import for that new deck file. This is the only user friendly/quick way I can imagine that would not require a big rewrite of CrowdAnki's structure, yet still allow for the combination of not only more notes but also fields as well.

   Anyways the too much detail above is just to say: should each independent piece of this "extended" deck be in one big repo (Supreme Geography) or many smaller repos? (UG - Bodies of Water; UG - Country Audio Field; etc). The above Deck Lego idea can also function on tags as well as repos, so it does not matter for that in the end. I guess the most important distinction is the concept of ownership/responsibility of maintaining, and that lends itself to another deck for each, with an "owner" for whoever wants to actually implement and maintain that information.

4. Should we use the same note models for all "same-level" (having the same cards) decks (i.e. with the exact same styling, same name and same UUID) or separate ones (i.e. similar, possibly even identical in terms of templates/styling, but with different names and UUIDs)?

   The note templates likely will be very similar anyway and having the exact same one has the advantage of making things easier for end-users in case they wish to adjust the notes. It also makes it easier for the "mix-and-match" described in 3.

   OTOH having the same note model copied among several repos means that updating the decks might have to be synchronised if/when the note models are updated.

5. Overlapping notes/topics

   How do we deal with entities that are split between more than one deck?

   For instance, for some of our dependent territories, we include them, but only with country<->map. If we include them in "Hardcore Geography" (or some other deck), do we just include them with capital and flag (i.e. excluding country<->map) or do we duplicate the country<->map card from AUG?

6. How do we deal with cases where the flag similarities or capital hint is only needed when a user has both (or multiple decks) decks?

---

In terms of medium-term plans, I don't think that we risk much wasted effort if, as a first stage, we just add all the non-AUG notes to a single, separate repository ("Hardcore Geography") and package it as a single deck. BrainBrew gives us enough flexibility that I don't think that we'll have any issues migrating notes, if necessary, as long as we do it before the first non-core-AUG release. Even after a release there shouldn't be any major user-facing problems provided we have a clean split of one deck into many (e.g. "hardcore geo" -> "lakes", "autonomous territories", "mountains") as opposed to migrating notes between decks in a many-to-many fashion. Autonomous territories are good as a starting point, since we already have a clear demarcation line of what will and won't be in core-AUG (so there's no risk that we'll need to move notes back into the core).

[axelboc]

1. Repos

IMO, it's important that the maintainers of a repo be willing to maintain the whole content of that repo and not just a part of it; otherwise, we risk ending up with dead parts. This is already a bit of a problem with translations: we, as core maintainers of UG, end up having to translate content ourselves any time there's a change; otherwise, translations would lag behind. Everything is coupled together, which means everything has to be updated at the same time.

With multiple repos and proper versioning, each repo can evolve at its own pace. Of course, we could imagine a monorepo set-up of sorts, with "packages" versioned independently from one another. However, I do believe that supporting multiple repos would be more scalable and really help create an anki-geo community. People could build their extension repos at their own pace, choose their own maintainers and contribution guidelines ... completely independently from us.

2. Decks

The sky's the limit here. We can easily provide any number of Brain Brew configs, or configs for the selection system that @ohare93 talks about, in order for people to generate the various decks they're interested in learning.

3. Selection system

This is definitely the key to the whole thing. People need to be able to generate their own decks, with the notes, fields/models, templates, and translations they want. Each part of those "franken-decks" should be versioned so that they can be updated deterministically. People should be able to add/remove parts and back-up their "franken-deck" configurations.

The way I see it, each extension repo could contribute various things to the ecosystem: notes, note models, templates, translations, decks, etc. So there should be a common way to declare these "contribution points".

Suffice to say, this will be very complicated to implement.

4. Note models

Multiple notes in a deck can use different note models, right? If so, then I kinda feel like their should be different note models for different types of content. I find that our own note model is already a bit weird to use with some entities like water bodies. Why restrict ourselves and try to fit a square peg into a round hole? UG could very well have different note models, templates and data files for states, territories, water bodies, etc.

Then it's a matter of making note models extensible so extension repos can add fields to notes from other repos.

5. Overlapping notes

To me this is a problem for the selection system, not for the HG repo (cf. below).

6. Replacing existing content (capital hints, flag similarities)

I guess extension decks could contribute "content overrides", for instance:

- noteId: <abcd>
  fieldName: Capital hint
  value: <New hint>

I could imagine this being useful also to fix outdated or missing content in case a repo stops being maintained, or even to replace content people disagree with (for instance if someone wants to make a repo that follows another source than Wikipedia.)

Then users could ask the selection system to process a configuration such as this (EDIT this is more of a user-facing mental model than a realistic implementation):

1. Take all the notes from UG.
2. Apply all the content overrides from HG.
3. Add all the notes from HG.
4. Generate cards from all the notes based on UG's main note model (i.e. with templates Country - Capital, Capital - Country, etc.)

---

Thinking out loud, I think Brain Brew is really not far off from being that "selection system" that we're talking about. Obviously, we'd need a UI on top of it, ideally in the form of an Anki add-on, but Brain Brew's concept of recipes is exactly what we need.

One thing that's definitely missing is a standardised way to declare "contribution points" in a repo. This could be a file at the root named "contributions.yml" or simply a standardised file structure similar to UG's. This is so the selection system knows where to look for notes, fields, templates, note models, etc.

---

I recommend attacking the problem from existing needs, of course, rather than to imagine a complete solution with all the bells and whistles: what deck do we want to generate and how to we get there:

• the extended deck = need a way of contributing templates and selecting them when defining the note model in the "selection system";
• the translations = need a way of contributing notes and generating a deck from those notes only.
• the hardcore/supreme deck = need a way of contributing notes and generating a deck from those notes combined with notes from another deck.

Other popular user needs that could be good starting points too:

• Replacing/modifying a template with one that shows auto-generated content (e.g. link to Wikipedia, text-to-speech, etc.)
• Adding a new field and replacing/modifying a template to show the content of this new field (e.g. IPA or audio pronunciation in a given accent)
• Adding a new field and a new template to generate cards based on this new field (e.g. currency, spoken language, geographical coordinates of capital so it can be shown automatically on a map in the template, etc.)

[ohare93]

(AAAHHH I wrote a reply and new hit Comment 🙈 Most parts have been addressed by @axelboc but here's what I wrote anyways)

Quote away, good sir! 👍

Seems to me like we need to consider there being 4 types of additions that an extended deck can provide:

1. Completely new notes using existing note models
   • Examples
     • New territories
     • American states
   • Must use the same fields that currently exist
2. New notes with new Note Model
   • Examples
     • Perhaps the American states should be formatted different and/or have more fields
   • No issue here with adding in these extra fields, as they exist in another note model
3. Change existing fields for existing notes
   • Add extra info that does not exist yet
     • Capital of Hong Kong / Java / Melilla / Ceuta
     • Capital info/hint where you believe there should be one
   • Change an existing field
     • Add a state flag alongside the existing flag (combine)
     • Capital name
   • This can be handled the same way we do currently for translations: another csv with this extra data
     • Though the "add extra info" flow does not exist, but it could easily be added into Brain Brew to join together two fields into one 😄
4. Completely new fields for existing cards 😨 🤩
   • As useful as it is daunting! The hail mary long term plan!
   • Not something I believe Adding autonomous territories #602 is asking for, but just something to take into account
   • Data format is not an issue, it is the same as the current setup: a separate csv for each new field, with only the filled in countries mapped