Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade Bubastis ontology diff tool to version 1.3.0 #109

Open
Tracked by #216
jvendetti opened this issue Aug 3, 2020 · 5 comments
Open
Tracked by #216

Upgrade Bubastis ontology diff tool to version 1.3.0 #109

jvendetti opened this issue Aug 3, 2020 · 5 comments

Comments

@jvendetti
Copy link
Member

There's been some recent activity again on Bubastis. We should consider upgrading to their newest release.

One hurdle is their use of Java 8 - we're running Java 11 on the production parsing box.

@syphax-bouazzouni
Copy link

Hi,

I have worked recently on Bubastis and I think we at Agroportal use the 1.3.0 version (updated by my predecessor ontoportal-lirmm@4e237b1), we hadn't had any issue using it on java 11 (till now).

And about this issue (and the owlapi wrapper) I had some propositions and I wanted to have your opinions before doing it :

  1. What do you think of extracting Bubastis and owlapi wrapper into two independent gems with their own tests and repository, and think also this will make updating easier.
  2. Bubastis and owlapi wrapper both use owlapi 4 (4.5.1 for Bubastis and 4.5.18 for owlapi wrapper), what do you think of passing the two to owlapi 5 (I'm not an expert on owlapi, do you think that the upgrading is worthing it?)
  3. Is migrating Bubastis to java 11, something that interests you and do you have any plans about it, enhance it or change it with another tool that you had found out?
  4. Finally about missing import stragey, in owlapi wrapper we use the SILENT strategy wherein Bubastis it's the defaultTHROW_EXCEPTION that is implemented, At Agroportal we thought of making the two silent but to add MISSING IMPORT submission error status

Thanks,

@jvendetti
Copy link
Member Author

Hi Syphax.

  1. Honestly speaking, we do releases of the owlapi_wrapper so infrequently that changing the way we make updates hasn't been high on the priority list. I don't know how to advise you here. There's no one else actively working on BioPortal at the moment for me to discuss this with. I don't think this should stop you from experimenting with an independent gem if you think it improves processes. (For Bubastis, see below).
  2. From a debugging perspective, we've found it very valuable to be able to compare how BioPortal loads and displays ontologies with another independent ontology tool, i.e., Protege Desktop. We've stayed on the 4.x series of the OWL API so that we could remain in sync with Protege, which also uses the 4.x series.
  3. The last time I looked at Bubastis, it seemed like it wasn't under active development anymore. We've talked about replacing it with ROBOT. It's described as an OBO tool, but it uses the OWL API internally, and seems capable of performing diffs on any ontology format that the OWL API handles.
  4. You're referring to adding a value here? I'm assuming you want to do this so that you can sort ontologies on the Admin page by a missing import error status?

@alexskr
Copy link
Member

alexskr commented Dec 15, 2022

Moving the owlapi wrapper and bubastis into independent gem would be beneficial.
Every time the owlapi_wrapper file is updated, the installed ontologies_linked_data gem only increases in its size on the disk because git holds all previous version of .jar files. The current version is ~205Mb and the previous takes ~165MB. We could use git LFS for storing .jar files which would help with storage but moving them out into an independent versioned gem is even better.

@syphax-bouazzouni
Copy link

A problem that I have also is when it comes to merging with our local code, I can't know if the jar files changed or not (and in what they changed)

@ncbo-deployer
Copy link

ncbo-deployer commented Dec 19, 2022 via email

galviset referenced this issue in EarthPortal/ontologies_linked_data Mar 14, 2024
galviset referenced this issue in EarthPortal/ontologies_linked_data Mar 14, 2024
* add request language to @context

* add subscribed attribute to the user model (#68)

Co-authored-by: Syphax bouazzouni <[email protected]>

* change name ( language -> lang )

* add @language to @context

* in tests no more use the reload argument in the exist? method

* rename the INRAETHES test file name

* fix mappings test test_external_mapping_with_no_source_ids validity

* handle users with no username  to_string

* add requested language tests

* use for Goo the language filter feature branch

* add request_store gem

* update requested language tests to use RequestStore gem

* migrate submission attributes metadata to a yaml scheme file

* migrate submission types enforce to the type section

* update ontology submission types and validators

* add metadata information to some attributes of ontology submission 

Updated metadata information for version, URI,  hasOntologyLanguage, status, hasFormalityLevel, hasOntologySyntax, deprecated, versionIRI attributes

* fix a typo in ontology submission curatedOn attribute validator

* reading language value from request query parameter called 'language'

* update the metadata extractor to handle the new goo validators

* fix tests to handle the new ontology submission validators

* force averageChildCount  metrics to be a float

* remove the ontology_submission  creationDate attribute default value

* disable extractedMeta  the keyClasses and exampleIdentifier attributes

* remove unique validator fro the URI attribute

* make exampleIdentifier default the first class of the ontology

* add enforce_symmetric_ontologies proc validator test

* implment enforce_symmetric_ontologies proc validator

* use enforce_symmetric_ontologies proc validator for som submission attrs

* move ontology_submission_validators tests to a seperate file

* add status_align_callbacks tests

* implement submission status align callbacks

* add the new two status onUpdate callback to the submission attribute

* remove the old enforce_symmetric_ontologies from the submission tests

* implement submission retired validators and  callback

* use submission retired validators and callbacks

* add submission retired status validators and callbacks tests

* add submission modification_date_previous_align validator test

* implement modification_date_previous_align validator

* refactor validators module to extract the callback module

* add submission test_has_prior_version_callback

* implement and use include_previous_submission onUpdate callback

* add ontology update_submissions_has_part test

* implement and use ontology update_submissions_has_part callback

* implement and use submission include_ontology_views validator

* Update ontology_submission.yml
Update ontology_submission.yml
Update ontology_submission.yml
Update ontology_submission.yml
Update ontology_submission.yml
Update ontology_submission.yml
Update ontology_submission.yml
Update ontology_submission.yml

* Update Gemfile.luck to get the new version of Goo

* fix some syntax issue the ontology_submission.yml file

* add submission inverse ontology properties callbacks tests

* implement and use ontology_inverse_of callback for useImports and usedBy

* Update ontology_submission.yml

labels : uri,homepage, natural language,langauge, publication
Update ontology_submission.yml

Labels => Documentation, version, Description, Status, Creation date, contact
Update ontology_submission.yml

adding labels : Number of classes, Number of individuals, Number of properties, Modification date, Number of individuals, Number of axioms or triples, Keywords, Key classes, Known usage, Notes, Knowledge representation paradigm
Update ontology_submission.yml

adding labels : Designed for task
Update ontology_submission.yml

Adding labels : Was generated by, curator, was invalidated by, Endorsed by, Funded or sponsored by, Translator, Subject, Formality level, License, Engineering methodology
Update ontology_submission.yml

Adding labels : Created With, Prior version, Imports, Created With, Incompatible, Backward compatible, Deprecated, Generally related to, Has equivalences with, Hidden or past name, Coverage, Publisher, Other identifier, Source
Update ontology_submission.yml

adding labels : Abstract, Has part (has views), Has format, Audience, SPARQL endpoint, Accrual policy, Accrual periodicity, Accrual method, Download URL, Free-text search endpoint, Competency question, logo, depiction, Disparate modelling with, To do list, Preferred namespace URI, Preferred namespace prefix,More permissions, Use guidelines, Curation date, Repository, Bug database, Mailing list, Example of resource, Rights holder, Associated media, Indexed or Included in catalog or repository

* refactor and reorder submission attributes  and group them by category

* disable metadata extraction for hasOntologyLanguage attribute

* update Gemfile.lock to get the latest version of Goo

* fix syntax issues in the ontology_submission.yml file

* enable submission creationDate automatic extraction

* implement and use lexvo_language validator

* add submission lexvo_language_validator test

* remove no more needed concatenate, textarea and isOntology attributes enforce

* remove submission metrics attributes  and move them to the metrics model

* set the default values callbacks to submissions attributes

* don't show request language if @context is not enabled

* Accrual periodicity values

* IncludedInDataCatalog : enforcedValues

* apply for some submission attributes ontologies_inverse_of  callback

* Add metadataMappings to properties 

Synchronize and consolidate the file

* Add metadataMappings values 

Synchronize and consolidate the file

* Add missing properties (Object description properties) & pullLocation

* Add obsoleteParent property

* Add Missing Metrics properties

* fix  json serializer nil exception (#79)

if hash was missing @context key the json serializer throwed a nil exception

* add some metrics attributes as embed values for Submissions

* prevent a bug when an embed object is null  (e.g submission with no metrics)

* Feature: return requested languages in @language - part 2 - multiple langugaes (#80)

* remove platform languages

* set all to default in language params

* upcase the requested language

* support select multilanguage

* return only existing language

* extract get_languages

* fix json serializer nil exception

* access submission safely

* refactor

---------

Co-authored-by: Syphax bouazzouni <[email protected]>

* add get_languages and  get_submission_languages helpers

* use get_languages helper to get the current context language

* extract get_language helper to get the requested language

* get serialized object submission only if include 'submission' attribute (#85)

* call to read_restricted?  only if instance_to_base_on not nil

* remove duplicated gzip? function

* add zip_file function to the file utilities module

* add archiving submission zipping old uploadedFile file test

* add removing archived submission unzipped folder test

* implement and extract the new archive action in the submission process

* prevent interconnected tests  of zipped_submission process tests

* add archiving file size threshold for zipping uploaded file

* fix ontology submission yaml file syntax issues

* add AgentIdentifier model

* add FOAF Agent model

* update submission person and organization attributes to use Agent model

* update Agent model to enforce name to be unique and an auto generated ID

* disable extracting metadata for attributes of type

* rename Agent 'type' to 'agentType'  to not conflicts with JSON-LD @type

* remove property: :pageHome form Agent model because it does not exist

* make identifier model embded true

* add read write access rules for agent and identifier models

* remove firstName and lastName attribute from Agent model

* enforce Agent affiliations to be all of type Organization

* enforce Identifier notation attribute to not be a URL

* add agent test for creation validity

* enforce to show all attributes of the embed affiliations of Agents

* add show_with_language param

* update get_submission_languages

* change identifier id to make be  schemaAgency +  notation

* make agent identifiers list unique

* make agent email attribute unique

* make agent name no more unique

* change agent affiliations attribute namespace to org:memberOf

* fix identifier id to be nil if notation or agentType empty

* make email agent attribute optional

* add is_organizaion validator to agent and submission affiliation attributes

* fix the class portal language test for the new multilanagual support 

Now when we ask a language we ether the values with the matching language, or nil or empty if not.
Were before we had fallback to show the values in the others languages to not have empty fields

* optimize request and portal lang tests by parsing the test ontology once

* add  test request all languages unit test

* use the new getters argument to show languages

* fix skos xl tests after the new multilangual support 

SKOS XL labels are tagged with a language so literal value is empty if it does not match the requested language  but the object is still fetched and showed

* add test for multilingual terms indexation and search

* update SOLR terms schema to add dynamicFields for values in all langs

* make the indexation use map_attributes with include_languages enabled

* override the Ressource to_hash method in Class model to

* update the indexed document  of a class model to include the dynamic fields for values in other languages

* rename the getters to show_all_languages argument to include_languages

* add is_person validator to hasCreator and hasContributor attributes

* embed all agents attributes for the submission model

* fix submission deprecate_previous_submissions callback when no valid

* refactor submission agent validators to extract common logic

* update the include_ontology_views validator to test prefixed url id

* add Agent type to submission endorsedBy attribute

* remove URI type of submission usedOntologyEngineeringTool attribute

* remove class type for exampleIdentifier and keyClasses attributes

* check if object.class respond to attributes before calling it

* check if the attribute method is a getter and not serialized method

* check if the attribute method is a getter and not serialized method

* use the same populate method as the attribute to show languages in properties also

* use the same populate method as the attribute to show languages in properties also

* Fix bug in authorization.rb

* Improved support for OAuth 2.0 and SSO

* bundle add oauth2

* remove duplicate find_apikey in the authorization call

* add find_access_token method oauth2 authorization

* add the unauthorized message the alternative text if oath2 enabled

* add user oauth2 tests

* enforce user email attribute to be unique

* on user save (creation) send a notification if enabled in the arguments

* add oauth2 providers configs

* implement oauth2 providers token authentification

* remove access token authorization to use only apikeys methods

* fix tests after enforcing user email uniqueness

* override security when sync user with an external provider

* fix update_submissions_has_part callback to check if valid before saving

* add uri type enforcement to submission URI attribute

* disable attributes onUpdate callbacks feature

* fix URI extraction process to enforce RDF::URI instead of string

* fix test after URI enforced to be URI

* disable complex attriubtes validators and callbacks

* Update Object description properties

Changed enforced values and definitons.

* Remove enforced values for pullLocation

* fix ontology submission attributes categories

* disable complex validators tests

* embed for submission model all the attributes of the object metrics

* Fix: ontology submission metadata file rework (#108)

* Template added at top of file

* Consolidation of the "General" properties

* Consolidation of "Licensing", "Description" and "Dates"

* Consolidation of the "Person and orga" properties

* Update ontology_submission.yml

* Consolidation of the "Usage" and "Methodology" properties

* Consolidation of the "object description properties"

* Consolidated the Metrics properties

* Consolidation of Links and Relations properties

* Consolidation of Content and Media properties
* Fix syntax error

---------

Co-authored-by: Syphax bouazzouni <[email protected]>

* add usage method to the agent models (#109)

* Fix multiple errors and syntax bugs in ontology_submission.ym file (#110)

* Fix multiple errors and syntax bugs.

* Update ontology_submission.yml

* Fix: optimize fetch all agents usages query  (#111)

* add usage method to the agent models

* optimize the agents usages query, to do batch loading when fetching all agents

* Fix: the lexvo language validator to no check the iso version (#112)

* update the lexvo language validator to no check the iso version

* Fix: login auth type mail options (#113)

* added mail options test to catch auth type case

* fixed auth type mail options not being saved

* Fix: ontology submisison "includedInDataCatalog" and "accrualMethod" metadata  (#114)

* fix includedInCatalog enforced values to be urls

* update accrualMethod to no more be limited to URIs

---------

Co-authored-by: HADDAD Zineddine <[email protected]>
Co-authored-by: DJEBABRIA Hiba <[email protected]>
Co-authored-by: Raimi Solorzano Niederhausen <[email protected]>
Co-authored-by: Clement Jonquet <[email protected]>
Co-authored-by: Guillaume Alviset <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants