GloBI publication -Arctos coauthors? #2539
Comments
campmlc
assigned dbloom, ccicero, mkoo, dustymc, jrdemboski, DerekSikes, acdoll, campmlc and jldunnum
|
|
|
But are we listing individual people or just citing the collections themselves? If individuals, then should we be looking at the collectors as well as those who manage the data in Arctos? Personally, I think citing the collections themselves should be sufficient unless you want to have hundreds of authors on the pub. |
|
This dataset isn't citing all our association records, just those that meet
TPT criteria. Dusty, can you figure out what sources are in their actual
dataset on the pub? Or is that what you've pulled?
…On Fri, Mar 6, 2020, 8:00 AM dustymc ***@***.***> wrote:
* UNM-IT Warning:* This message was sent from outside of the LoboMail
system. Do not click on links or open attachments unless you are sure the
content is safe. (2.3)
select guid_prefix,count(*) from collection,cataloged_item,coll_obj_other_id_num 2 where collection.collection_id=cataloged_item.collection_id and cataloged_item.collection_object_id=coll_obj_other_id_num.collection_object_id and ID_REFERENCES in
('eaten by','ate','host of','parasite of') group by guid_prefix order by guid_prefix; GUID_PREFIX COUNT(*) ------------------------------------------------------------ ---------- CHAS:Bird 6 CHAS:Ento 6 CHAS:Herb 1 CHAS:Herp 1 CHAS:Mamm 2 CHAS:Teach 1 DMNS:Bird
297 DMNS:Herp 1 DMNS:Inv 4 DMNS:Mamm 569 DMNS:Para 825 HWML:Para 861 KNWR:Ento 47 KNWR:Herb 12 KNWR:Inv 1 KNWRObs:Fish 7 MSB:Bird 213 MSB:Fish 4 MSB:Herp 2 MSB:Host 15756 MSB:Mamm 11902 MSB:Para 29055 MVZ:Bird 7 MVZ:Herp 5 MVZ:Mamm 40 NMU:Mamm 19 NMU:Para
12 UAM:Ento 6658 UAM:Inv 3 UAM:Mamm 7966 UAMObs:Mamm 1 UAMb:Herb 1 UCM:Mamm 2 UCM:Obs 2 UMNH:Mamm 9 USNPC:Para 99 UTEP:Herp 16 UTEP:HerpOS 1 UTEP:Inv 1 UTEPObs:Ento 1 UTEPObs:Herp 2
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
<#2539?email_source=notifications&email_token=ADQ7JBHTYTT6GC34UMSAQJTRGEFZHA5CNFSM4LCXAKR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOBUSVY#issuecomment-595806551>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBBKYPVIP7FBWFAB7N3RGEFZHANCNFSM4LCXAKRQ>
.
|
|
I pulled this:
|
|
I'm thinking these aren't representative numbers, because the MSB Para
collection only has 4719 arthropod "parasite of" relationships, not 29,055,
which is most of the collection. How do we access what is in the
actualdataset to be published? Do we need to ask Jorrit, or is there some
way I don't understand to access the data from this site?
https://zenodo.org/record/3685365#.XmKSjKhKjIV
…On Fri, Mar 6, 2020 at 9:03 AM dustymc ***@***.***> wrote:
* UNM-IT Warning:* This message was sent from outside of the LoboMail
system. Do not click on links or open attachments unless you are sure the
content is safe. (2.3)
I pulled this:
has relationship links for host, and ectoparasite, or associated taxa, and
your specimen records have GloBI links
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
<#2539?email_source=notifications&email_token=ADQ7JBFYGXMZ4OY3XGKLTFTRGENFVA5CNFSM4LCXAKR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOB337Y#issuecomment-595836415>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBEGY7QXYDZHTU5Z5TDRGENFVANCNFSM4LCXAKRQ>
.
|
I'm not sure what you mean by 'arthropod' in that, but MSB:Para has 28823 distinct catalog records with a 'parasite of' relationship. (Some of them probably have multiple relationships, which inflated the previous count a bit.) Perhaps globi isn't using them all, or they don't include resolvable identifiers (which probably means globi isn't using them) or similar? I can't know what they're doing, just what we're making available. There's some download links at the bottom of the page you linked, but I don't see any super-obvious summary. I might get one downloaded eventually, I'll let you know if I find anything interesting if it completes.... |
|
Globi is using them all, but the TPT data are specifically arthropods.
…On Fri, Mar 6, 2020, 11:25 AM dustymc ***@***.***> wrote:
* UNM-IT Warning:* This message was sent from outside of the LoboMail
system. Do not click on links or open attachments unless you are sure the
content is safe. (2.3)
MSB Para collection only has....
I'm not sure what you mean by 'arthropod' in that, but MSB:Para has 28823
distinct catalog records with a 'parasite of' relationship. (Some of them
probably have multiple relationships, which inflated the previous count a
bit.) Perhaps globi isn't using them all, or they don't include resolvable
identifiers (which probably means globi isn't using them) or similar? I
can't know what they're doing, just what we're making available.
There's some download links at the bottom of the page you linked, but I
don't see any super-obvious summary. I might get one downloaded eventually,
I'll let you know if I find anything interesting if it completes....
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
<#2539?email_source=notifications&email_token=ADQ7JBARUYJUK2AOYXXERVDRGE53DA5CNFSM4LCXAKR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOCK74Y#issuecomment-595898355>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBE3N6PM737TNTLR3KLRGE53DANCNFSM4LCXAKRQ>
.
|
|
Chat from Happy Hour: Dusty M · Current Room · 5:21 PM |
|
Better formatting: By collection: |
|
just a quick comment-- what a cool way to see diversity and richness of Arctos connectivity! we need to encourage more in our collections (I know I'm greedy but I get the feeling it's really just the tip of the relationsberg!) |
|
Interesting that just yesterday I began modeling biotic relations as part
of a "grand unified model" plus a simplified data publishing model for
GBIF. It forms part of a study of use cases that challenge the limitations
of Darwin Core for data publishing. I am curious if you think you have any
particularly interesting use cases that I might investigate or prove
against the model.
…On Fri, Nov 12, 2021 at 1:36 AM Michelle Koo ***@***.***> wrote:
just a quick comment-- what a cool way to see diversity and richness of
Arctos connectivity!
we need to encourage more in our collections (I know I'm greedy but I get
the feeling it's really just the tip of the relationsberg!)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#2539 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AADQ7235DSWFCJY2BTTBMRDULSKTVANCNFSM4LCXAKRQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
|
@tucotuco not sure what you need, but here are links to some records with relationships. https://arctos.database.museum/guid/CHAS:Herp:18616 (ate) https://arctos.database.museum/guid/DMNS:Bird:44431 (ate, host/parasite) https://arctos.database.museum/guid/UAM:Ento:378955 (iNaturalist relationship) https://arctos.database.museum/guid/MSB:Mamm:129858 (parent/offspring, siblings) |
|
@dustymc could I take a look at those UWYMV:Herp records? That seems like a lot of relationships...
I love this relationship! And I want to figure out how to do more of these...habitat data relationships! |
|
|
Thanks @dustymc! |
|
Definitely a record of a historic versus modern curation practice in our collection. |
Thank you @Jegelewicz . Those all look like straightforward relationships. One of the things I am looking for and hoping not to find is an example of a relationship that is complex, in that there are multiple simultaneous interactions that can NOT be modeled as multiple pairwise interactions. The only thing so far that might qualify is the desire/need to model sequentially dependent relationships. So far I have not see demand for that. |
You won't find that in Arctos, they're all simple, a record can just have lots of them.
That we can do. https://arctos.database.museum/guid/MSB:Bird:40556 was collected with something that can be identified only by some sorta-not-really-typed-string that presumably resembles something that got scribbled on some label or something somewhere, maybe. A knowledgeable and dedicated person can probably figure it out and make a pair, but until that happens it's just a lonely pointer off into the aether.
I'm not sure what that means, but it makes me suspect @campmlc should be asking for one or more less-metadata-ey date-like values for #4101. |
|
@dusty ***@***.***> OK, That IS interesting. It seems like you need
the pointer into the aether because the thingy pointed to doesn't exist in
a way that can share a collecting event that would otherwise suffice. And
based on its aethereal nature it requires a special string to represent it.
By sequentially dependent I mean a relationship C only happened because
relationship B happened, and that in turn relied on relationship A.
I don't know good use cases for it, but in the interim I am modeling it
just in case my intuition is better than my knowledge.
…On Wed, Nov 17, 2021 at 4:13 PM dustymc ***@***.***> wrote:
relationship that is complex,
You won't find that in Arctos, they're all simple, a record can just have
lots of them.
can NOT be modeled as multiple pairwise interactions
That we can do. https://arctos.database.museum/guid/MSB:Bird:40556 was
collected with something that can be identified only by some
sorta-not-really-typed-string that presumably resembles something that got
scribbled on some label or something somewhere, maybe. A knowledgeable and
dedicated person can probably figure it out and make a pair, but until that
happens it's just a lonely pointer off into the aether.
sequentially dependent
I'm not sure what that means, but it makes me suspect @campmlc
<https://github.com/campmlc> should be asking for one or more
less-metadata-ey date-like values for #4101
<#4101>.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#2539 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AADQ724EWRNFLL6QN57ULOTUMP5HFANCNFSM4LCXAKRQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
Like a person ended up with Zika because a mosquito bit them? |
|
This could be used for experimental infections, A is host B which
experimentally infects C which experimentally infects D
…On Wed, Nov 17, 2021, 1:04 PM Teresa Mayfield-Meyer < ***@***.***> wrote:
* [EXTERNAL]*
By sequentially dependent I mean a relationship C only happened because
relationship B happened, and that in turn relied on relationship A.
Like a person ended up with Zika because a mosquito bit them?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#2539 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBB5YUCTUREGXSKTMVDUMQDETANCNFSM4LCXAKRQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
Yep, "can" is the key. Lots of things can lead to the same dead end: it's not a resolvable identifier, it will be but hasn't been entered yet, it's there but your permissions don't let you see it (maybe its encumbered), it's there but something is refusing to talk to you at the moment, etc., etc.
To be clear: Someone could create an "ain't got nuthin at all to do with" relationship tomorrow. There should be no expectations at all, but hopefully the definitions will clear things up (eg so you can just ignore whatever you don't care about). And "collecting event" (in the sense of the thing with a pkey in Arctos) probably isn't a realistic expectation either - two geese are "collected with" each other because they're doing whatever bird-people say that is, but they're both also wearing GPS collars and come from 1 cubic meter nonoverlapping events occurring 38 seconds apart.
That level of resolution would probably demand something beyond identifiers, but (especially if you add a few more cycles in) I can maybe see how identifiers would need something beyond "assigned_on_date" as well. |
It would enable that kind of assertion if one were inclined to make it, yes. There are lots of examples. Here is my favorite so far: https://onlinelibrary.wiley.com/doi/10.1111/mec.16065?af=R |
Thinking about it more, I don't think a model actually needs that. If it is important enough, why not make a sparse record for it and link to that? Especially if the idea @Jegelewicz put forth here takes traction. |
|
I'd be curious to see these ideas/models go beyond thought experiments, and be put in practice. I believe the context in which ideas thrive and develop matters. Also, dealing with real data in real (resource constrained) settings in existing infrastructures puts to the viability of an approach to the test. Much appreciate the experimental approach of Arctos and their ability to adapt to take on new ideas and concepts. btw - is this issue still about a GloBI - Arctos publication? |
The usual, resources. "Says ABC123 right here" takes a second to enter, "resolving" that to a distinct entity of some sort can be a lot more complicated. The MSB:Host collection has taken that additional step - there's a need for data objects so they've created them (instead of making unresolvable relationships to "RLR1"). (And the step beyond that is finding the curated "real" record of the host and relinking to it - many Arctos records have done so, maybe eventually that entire catalog will have served its purpose and can go away - but probably not...) In any case, there's a very large functional difference between things that do have resolvable identifiers and things that do not, and I don't think there could be any defensible objections to some model having some minimal data requirements/standards and refusing to accept not-so-paired pairs. |
|
We developed the MSB:Host model to capture host "observations" as recorded
in the original collection ledger so that we could link out to actual
vouchers scattered across many different institutions as we discovered
them, many of which were cataloged from tags or other incomplete data.
These vouchers have all kinds of different versions of the data and of
"original IDs" - e.g "collector number = RLR1", "preparator number = RR1",
"original identifier = RLR 1", etc etc, which makes discovery slow but
eventually possible. The MSB Host record acts as a central repository for
the original data and also the platform to link to all these other various
records and identifiers via "same individual as", as well as to the
parasite record as "host of". This is where having relationship metadata
could be very helpful - e.g. I "think" on this date for this reason that
these two records in two different places are the same, and this is why or
how I determined this.
Having a separate catalog for the host observations also makes
host/parasite or host/pathogen or other relationship searches possible
across higher taxonomic categories, e.g. "find all sciurids that are hosts
of cestodes" by integrating the central observation record within taxonomy.
…On Thu, Nov 18, 2021 at 12:44 PM dustymc ***@***.***> wrote:
* [EXTERNAL]*
why not make a sparse record for it and link to that
The usual, resources. "Says ABC123 right here" takes a second to enter,
"resolving" that to a distinct entity of some sort can be a lot more
complicated.
The MSB:Host collection has taken that additional step - there's a need
for data objects so they've created them (instead of making unresolvable
relationships to "RLR1"). (And the step beyond that is finding the curated
"real" record of the host and relinking to it - many Arctos records have
done so, maybe eventually that entire catalog will have served its purpose
and can go away - but probably not...)
In any case, there's a very large functional difference between things
that do have resolvable identifiers and things that do not, and I don't
think there could be any defensible objections to some model having some
minimal data requirements/standards and refusing to accept not-so-paired
pairs.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#2539 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBC2QM7ULGTLHYARGO3UMVJQVANCNFSM4LCXAKRQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
HAHAHAHA!!! YES! Our issues tend to bleed into each other....but I think the concept of an Organism ID (same individual as relationships) is tangentially related to the interactions you are interested in. |
Here's one from the other issue:
Assuming no less-obvious problems exist, I'd expect half of those things to work should GAN start resolving. That sort of thing is easy to detect in linked data (the link works or it doesn't), in unlinked data undetectable messes just pile up. |
|
to document the links (and dependencies) of some of the Arctos collections over time, I've published two Zenodo data "papers" - Arctos/Vernet/Museum of Southwestern Biology. (2021). Global Biotic Interactions: Elton Dataset Cache Museum of Southwestern Biology and dependencies (0.1) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5711446 and Vertnet/Arctos Community. (2021). Global Biotic Interactions: Elton Dataset Cache Arctos/Vertnet [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5711415 The first publication contains tracked resources of MSB Para and their dependencies needed to resolve linked records. The second contains a (complete?) set of Arctos resources published via Vertnet as discovered via their RSS feed. Thanks for the Arctos linking strategy, I was able to determine the dependencies between Arctos collections and beyond. Sometimes the links can be established after applying some (implicit) naming conventions. The neat thing about these publications is that they contain a versioned timeline (~weekly snapshots or more frequently) of Arctos related resources starting from 2019-07-04T23:07:52.994Z . These versioned snapshots contain valuable clues on how the record linking developed over time, as well as giving an impression of the resources needed to keep track of these archives. Anyways, I've added some more info to the abstract that @Jegelewicz put together with some numbers. By the way, I was impressed by the number of non-Arctos links originating from MSB Para. Can't wait to put some more numbers on it. And, I am a little stressed out that my servers are filling up with evidence needed to enable full data provenance. And, I know that without this provenance, our collection records as permanent as words written on a public whiteboard with erasable markers. With this provenance, I can answer questions like: What is the origin of a claim that X interacted with Y as indexed by GloBI around May 2020? Current solution is a bit clunky, but hey . . . work in progress. |
|
Here's an (incomplete) list of the msb-para dependencies (note the non-vertnet) dependencies - @campmlc did I miss any? |
|
@dustymc about those unresolved links - I am hoping to include the "same individual as" relations as additional interaction claims (for those that also have biotic associations), so that you can search GloBI by identifiers known across different collection management system. I've done a similar exercise to resolve links for msb-para. Fun stuff! |
|
In the review report for msb-para, you can find some of these unresolved links: https://depot.globalbioticinteractions.org/reviews/globalbioticinteractions/msb-para/README.txt so, there's a bunch of work left for me . . . |
|
Happy to do a call to go over these individually to see which we can
resolve. We should be able to resolve the references within Arctos
collections. Perhaps you can help resolve the USNM references - since we
can't link to them directly in Emu, but perhaps Globi or via GBIF?
…On Thu, Nov 18, 2021 at 1:55 PM Jorrit Poelen ***@***.***> wrote:
* [EXTERNAL]*
In the review report for msb-para, you can find some of these unresolved
links:
https://depot.globalbioticinteractions.org/reviews/globalbioticinteractions/msb-para/README.txt
so, there's a bunch of work left for me . . .
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#2539 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBFX65WMPS63B4TPKJ3UMVR5NANCNFSM4LCXAKRQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
|
Looks pretty comprehensive - I apologize for not being able to follow and
respond in real time to this entire discussion while in my day job - a
dedicated discussion time would be great.
…On Thu, Nov 18, 2021 at 1:49 PM Jorrit Poelen ***@***.***> wrote:
* [EXTERNAL]*
Here's an (incomplete) list of the msb-para dependencies (note the
non-vertnet) dependencies -
$ cat /var/cache/elton/datasets/globalbioticinteractions/msb-para/access.tsv | cut -f2 | grep -v "globalbioticinteractions" | sort | uniqhttp://digir.mcz.harvard.edu/ipt/archive.do?r=mczbasehttp://ipt.vertnet.org:8080/ipt/archive.do?r=cumv_mammhttp://ipt.vertnet.org:8080/ipt/archive.do?r=msb_birdhttp://ipt.vertnet.org:8080/ipt/archive.do?r=msb_fishhttp://ipt.vertnet.org:8080/ipt/archive.do?r=msb_hosthttp://ipt.vertnet.org:8080/ipt/archive.do?r=msb_mammhttp://ipt.vertnet.org:8080/ipt/archive.do?r=msb_parahttp://ipt.vertnet.org:8080/ipt/archive.do?r=mvz_birdhttp://ipt.vertnet.org:8080/ipt/archive.do?r=mvz_mammalhttp://ipt.vertnet.org:8080/ipt/archive.do?r=nmu_mammhttp://ipt.vertnet.org:8080/ipt/archive.do?r=uam_entohttp://ipt.vertnet.org:8080/ipt/archive.do?r=uam_mammhttp://ipt.vertnet.org:8080/ipt/archive.do?r=ucm_mammalshttp://ipt.vertnet.org:8080/ipt/archive.do?r=umnh_mammalshttps://collections.nmnh.si.edu/ipt/archive.do?r=nmnh_extant_dwc-a
@campmlc <https://github.com/campmlc> did I miss any?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#2539 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADQ7JBHEOULD4SFREKLKOD3UMVRFPANCNFSM4LCXAKRQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
…endencies; related to ArctosDB/arctos#2539
|
@dustymc this method above to check links seems a bit resource intensive. Anyway you can publish all identifiers known to Arctos in a single file? |
Yeah - that's the kind of thing we could present on! |
I think that's just Arctos, but I can probably get you public catalog record IDs (including non-relationship-forming??) - what columns do you need? |
|
A public catalog of Arctos records ids sounds like a great start! This way, I can quickly check which Arctos Ids are known to exist, and which ones likely result from typos or parsing/translation errors. In a way, I see that list of identifiers as the most basic of APIs - a list of identifiers that Arctos knows stuff about. |
|
@dustymc seems like that is already in our API? |
From Jorrit Poelen of GloBI - looking for Arctos coauthors of the following dataset for the Parasite Tracker TCN:
https://zenodo.org/record/3685365#.XmGtXqhKjIU
https://www.globalbioticinteractions.org/?interactionType=interactsWith&sourceTaxon=NCBI%3A9
http://parasitetracker.org/
Jorrit is publishing this dataset as a baseline for the Parasite Tracker TCN to show GloBI records which include Arctos ectoparasite biotic relationship data from multiple institutions. If your collection has relationship links for host, and ectoparasite, or associated taxa, and your specimen records have GloBI links, you may want to be included as an author, or we need to figure out a way to have Arctos itself cited. It is difficult to isolate just Arctos collections from all the records published through the VertNet IPT - or perhaps these are all our records? @dbloom
@dustymc @Jegelewicz @mkoo @DerekSikes can you figure out what institutions are represented here? Who should we include on the author line, if anyone? Just TCN participants, or all those who have made these relationships discoverable? I put my name as a placeholder, but should include others.
The text was updated successfully, but these errors were encountered: