Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Semantic Graph Synthesis #331

Open
wants to merge 82 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
14de56c
Template finder
haneslinger Sep 12, 2023
f29fb92
Merge remote-tracking branch 'origin/develop' into template-finder
gtfierro Sep 16, 2023
53da375
Test with spruce
haneslinger Sep 18, 2023
2fbd6d6
Merge branch 'template-finder' of github.com:NREL/BuildingMOTIF into …
gtfierro Sep 19, 2023
2911504
making it easier to recover the actual param/token mapping from the o…
gtfierro Sep 20, 2023
025d042
format tokens
gtfierro Sep 20, 2023
e0fcd47
running notebook with new code
gtfierro Sep 20, 2023
056b4a3
add a more specific sa temp sensor entity
gtfierro Sep 22, 2023
38f3743
WIP
haneslinger Sep 28, 2023
9685913
updates from conversation 10/4
gtfierro Oct 4, 2023
8d65d72
Clean up
haneslinger Oct 9, 2023
98cacc4
Merge remote-tracking branch 'origin/develop' into template-finder
gtfierro Dec 7, 2023
c809ab9
aider: Add comments to the 'unify_bindings' function to explain what …
gtfierro Dec 7, 2023
e129d88
add comments to graph synthesizer components
gtfierro Dec 7, 2023
878a37d
update deps
gtfierro Dec 7, 2023
d4a2f68
Merge remote-tracking branch 'origin/develop' into template-finder
gtfierro Jul 3, 2024
3b533db
update brick
gtfierro Jul 4, 2024
a1ada86
Merge remote-tracking branch 'origin/develop' into template-finder
gtfierro Sep 1, 2024
5979465
move templates inside the graphsynethesizer object
gtfierro Sep 5, 2024
f9859f4
add example of semantic graph synthesis along with a new ingress and …
gtfierro Sep 18, 2024
8888c2b
remove dev folder
gtfierro Sep 18, 2024
82940bc
move some dev notebooks where they will not be tested
gtfierro Sep 18, 2024
a07d53c
Merge branch 'develop' into template-finder
gtfierro Sep 18, 2024
736ada9
Merge branch 'template-finder' of github.com:NREL/BuildingMOTIF into …
gtfierro Sep 18, 2024
9c07a75
poetry.lock
gtfierro Sep 18, 2024
ef11826
drop support for 3.9 for numpy 2.0?
gtfierro Sep 18, 2024
65842be
add SGS ingress
gtfierro Sep 18, 2024
4852762
fix kernel
gtfierro Sep 18, 2024
1988898
Merge branch 'develop' into template-finder
gtfierro Sep 18, 2024
a0bb134
remove dev notebooks
gtfierro Oct 29, 2024
f06b32a
moving files around
gtfierro Oct 29, 2024
a5dfc83
fix imports
gtfierro Oct 29, 2024
e7498a3
Merge branch 'develop' into template-finder
gtfierro Oct 30, 2024
6b8a019
Improve validation report output (#320)
gtfierro Nov 19, 2024
1a5cff9
Remove hardcoded namespaces and make them parameterized
gtfierro Nov 19, 2024
d4b45d7
Import Graph to resolve undefined name errors in synthesizer.py
gtfierro Nov 19, 2024
9f02958
Accept ontology as input argument in semantic graph synthesis methods
gtfierro Nov 19, 2024
5b1b738
factoring out the hardcoded Brick/NS values
gtfierro Nov 19, 2024
7c7e089
reformatting
gtfierro Nov 19, 2024
17b15c2
Merge branch 'develop' into template-finder
gtfierro Nov 19, 2024
5b41514
try to handle rolling back on library dep error?
gtfierro Nov 19, 2024
644827e
update poetry.lock
gtfierro Nov 19, 2024
4be13ca
try custom errors
gtfierro Nov 20, 2024
7b923db
add errors
gtfierro Nov 20, 2024
5cf0c78
add debug statement
gtfierro Nov 20, 2024
958a2f5
more debug statements
gtfierro Nov 20, 2024
89de0a6
add more debug
gtfierro Nov 20, 2024
ce1d680
default values for template error
gtfierro Nov 20, 2024
1bbb607
expand library errors
gtfierro Nov 22, 2024
3bf4a14
using my new errors
gtfierro Nov 22, 2024
72e91b8
fix exception name
gtfierro Nov 22, 2024
5b9a95b
try a new cascae delete on deps?
gtfierro Nov 22, 2024
e444e82
commit delete library on overwrite
gtfierro Nov 22, 2024
7ec1062
delete dependencies
gtfierro Nov 22, 2024
2024132
delete library directly
gtfierro Nov 22, 2024
8ddb49b
enable foreign keys in sqlite
gtfierro Nov 22, 2024
db1e695
CASCADE on delete
gtfierro Nov 22, 2024
a9c9f28
removing passive deletes
gtfierro Nov 22, 2024
c3fbed4
remove bad param
gtfierro Nov 22, 2024
70b010d
adjusting cascades and passives
gtfierro Nov 22, 2024
d638016
switch back to template clearing
gtfierro Nov 22, 2024
2990b71
add debug on sgs
gtfierro Nov 22, 2024
0c8265d
more log
gtfierro Nov 22, 2024
972f789
debug -> info
gtfierro Nov 22, 2024
5c32a81
parser needs a Constant for the first
gtfierro Nov 24, 2024
bdf578c
add comment
gtfierro Nov 24, 2024
a39dd28
print graph CBDs in the output
gtfierro Dec 2, 2024
da3418f
allow missing imports for validation
gtfierro Dec 2, 2024
007238f
add api endpoint to get template body
gtfierro Dec 2, 2024
5daeb7a
make it a GET request
gtfierro Dec 2, 2024
253d05d
typos
gtfierro Dec 2, 2024
5ce3233
add wrap combinator
gtfierro Dec 3, 2024
5de8c11
SGS does not do fill()
gtfierro Dec 3, 2024
2b44be7
clean up PARAM triples
gtfierro Dec 3, 2024
c1f88dd
fix use of shacl path
gtfierro Dec 3, 2024
6e7299e
remove context
gtfierro Dec 3, 2024
073293f
use pyshacl for manifest
gtfierro Dec 3, 2024
6f59c54
try to inline
gtfierro Dec 4, 2024
1ae16e0
fix query
gtfierro Dec 4, 2024
f525260
refactor: Move template inference logic from Library to ShapeCollection
gtfierro Dec 17, 2024
b40079c
fix: Resolve undefined names in shape_collection.py for template infe…
gtfierro Dec 17, 2024
25bab24
add docs on converting shapes to templates
gtfierro Dec 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.8', '3.9', '3.10', '3.11']
python-version: ['3.9', '3.10', '3.11']
steps:
- name: checkout
uses: actions/checkout@v4
Expand Down
4 changes: 2 additions & 2 deletions buildingmotif/api/views/library.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
from flask import Blueprint, current_app, jsonify
from flask_api import status
from rdflib import URIRef
from sqlalchemy.orm.exc import NoResultFound

from buildingmotif.api.serializers.library import serialize
from buildingmotif.database.errors import LibraryNotFound
from buildingmotif.dataclasses.shape_collection import ShapeCollection

blueprint = Blueprint("libraries", __name__)
Expand Down Expand Up @@ -81,7 +81,7 @@ def get_library(library_id: int) -> flask.Response:
"""
try:
db_lib = current_app.building_motif.table_connection.get_db_library(library_id)
except NoResultFound:
except LibraryNotFound:
return {
"message": f"No library with id {library_id}"
}, status.HTTP_404_NOT_FOUND
Expand Down
11 changes: 10 additions & 1 deletion buildingmotif/api/views/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

from buildingmotif.api.serializers.model import serialize
from buildingmotif.dataclasses import Library, Model, ShapeCollection
from buildingmotif import get_building_motif

blueprint = Blueprint("models", __name__)

Expand Down Expand Up @@ -164,10 +165,13 @@ def validate_model(models_id: int) -> flask.Response:
return {"message": f"No model with id {models_id}"}, status.HTTP_404_NOT_FOUND

shape_collections = []
shacl_engine = None

# no body provided -- default to model manifest and default SHACL engine
# TODO: take the shacl engine as a parameter
if request.content_length is None:
shape_collections = [model.get_manifest()]
shacl_engine = "pyshacl"
else:
# get body
if request.content_type != "application/json":
Expand Down Expand Up @@ -198,7 +202,12 @@ def validate_model(models_id: int) -> flask.Response:

# if shape_collections is empty, model.validate will default
# to the model's manifest
vaildation_context = model.validate(shape_collections)
bm = get_building_motif()
old_shacl_engine = bm.shacl_engine
bm.shacl_engine = shacl_engine
vaildation_context = model.validate(shape_collections, error_on_missing_imports=False)
bm.shacl_engine = old_shacl_engine


return {
"message": vaildation_context.report_string,
Expand Down
26 changes: 23 additions & 3 deletions buildingmotif/api/views/template.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
from rdflib import Literal, URIRef
from rdflib.term import Node
from sqlalchemy.orm.exc import NoResultFound
from buildingmotif.database.errors import TemplateNotFound

from buildingmotif.api.serializers.template import serialize
from buildingmotif.dataclasses import Model, Template
Expand Down Expand Up @@ -42,7 +43,7 @@ def get_template(templates_id: int) -> flask.Response:
template = current_app.building_motif.table_connection.get_db_template(
templates_id
)
except NoResultFound:
except TemplateNotFound:
return {
"message": f"No template with id {templates_id}"
}, status.HTTP_404_NOT_FOUND
Expand All @@ -55,7 +56,7 @@ def evaluate_ingress(template_id: int) -> flask.Response:
# get template
try:
template = Template.load(template_id)
except NoResultFound:
except TemplateNotFound:
return {
"message": f"No template with id {template_id}"
}, status.HTTP_404_NOT_FOUND
Expand Down Expand Up @@ -107,7 +108,7 @@ def evaluate_bindings(template_id: int) -> flask.Response:
"""
try:
template = Template.load(template_id)
except NoResultFound:
except TemplateNotFound:
return {
"message": f"No template with id {template_id}"
}, status.HTTP_404_NOT_FOUND
Expand Down Expand Up @@ -141,6 +142,25 @@ def evaluate_bindings(template_id: int) -> flask.Response:
return graph.serialize(format="ttl"), status.HTTP_200_OK


@blueprint.route("/<template_id>/body", methods=(["GET"]))
def get_template_body(template_id: int) -> flask.Response:
"""Get template body.

:param template_id: template id
:type template_id: int
:return: template body
:rtype: flask.Response
"""
try:
template: Template = Template.load(template_id)
except TemplateNotFound:
return {
"message": f"No template with id {template_id}"
}, status.HTTP_404_NOT_FOUND

return template.body.serialize(format="ttl"), status.HTTP_200_OK


def get_bindings(binding_dict) -> Dict[str, Node]:
"""type binding_dict values to nodes

Expand Down
22 changes: 22 additions & 0 deletions buildingmotif/database/errors.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
from typing import Optional


class LibraryNotFound(Exception):
def __init__(self, name: Optional[str] = None, idnum: Optional[int] = None):
self.lib_name = name
self.lib_id = idnum

def __str__(self):
if self.lib_name:
return f"Library with name '{self.lib_name}' not found"
return f"Library with id '{self.lib_id}' not found"

class TemplateNotFound(Exception):
def __init__(self, name: Optional[str] = None, idnum: Optional[int] = None):
self.template_name = name
self.template_id = idnum

def __str__(self):
if self.template_name:
return f"Template with name '{self.template_name}' not found"
return f"Template with id '{self.template_id}' not found"
22 changes: 16 additions & 6 deletions buildingmotif/database/table_connection.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
DBTemplate,
DepsAssociation,
)
from buildingmotif.database.errors import LibraryNotFound, TemplateNotFound


class TableConnection:
Expand Down Expand Up @@ -216,7 +217,10 @@ def get_db_library(self, id: int) -> DBLibrary:
:return: DBLibrary
:rtype: DBLibrary
"""
db_library = self.bm.session.query(DBLibrary).filter(DBLibrary.id == id).one()
try:
db_library = self.bm.session.query(DBLibrary).filter(DBLibrary.id == id).one()
except NoResultFound:
raise LibraryNotFound(f"Library with id {id} not found")
return db_library

def get_db_library_by_name(self, name: str) -> DBLibrary:
Expand All @@ -227,7 +231,10 @@ def get_db_library_by_name(self, name: str) -> DBLibrary:
:return: DBLibrary
:rtype: DBLibrary
"""
return self.bm.session.query(DBLibrary).filter(DBLibrary.name == name).one()
try:
return self.bm.session.query(DBLibrary).filter(DBLibrary.name == name).one()
except NoResultFound:
raise LibraryNotFound(name=name)

def update_db_library_name(self, id: int, name: str) -> None:
"""Update database library name.
Expand Down Expand Up @@ -298,9 +305,12 @@ def get_db_template(self, id: int) -> DBTemplate:
:return: DBTemplate
:rtype: DBTemplate
"""
db_template = (
self.bm.session.query(DBTemplate).filter(DBTemplate.id == id).one()
)
try:
db_template = (
self.bm.session.query(DBTemplate).filter(DBTemplate.id == id).one()
)
except NoResultFound:
raise TemplateNotFound(idnum=id)
return db_template

def get_db_template_by_name(self, name: str) -> DBTemplate:
Expand All @@ -316,7 +326,7 @@ def get_db_template_by_name(self, name: str) -> DBTemplate:
self.bm.session.query(DBTemplate).filter(DBTemplate.name == name).one()
)
except NoResultFound:
raise NoResultFound(f"No template found with name {name}")
raise TemplateNotFound(name=name)
return db_template

def get_library_defining_db_template(self, id: int) -> DBLibrary:
Expand Down
32 changes: 23 additions & 9 deletions buildingmotif/database/tables.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
from typing import Dict, List

from sqlalchemy import Column, ForeignKey, Integer, String, Text, UniqueConstraint
from sqlalchemy import Column, ForeignKey, Integer, String, Text, UniqueConstraint, event
from sqlalchemy.engine import Engine
from sqlalchemy.orm import Mapped, declarative_base, relationship

# from sqlalchemy.dialects.postgresql import JSON
Expand All @@ -9,6 +10,13 @@
Base = declarative_base()


# https://docs.sqlalchemy.org/en/14/dialects/sqlite.html#foreign-key-support
@event.listens_for(Engine, "connect")
def set_sqlite_pragma(dbapi_connection, connection_record):
cursor = dbapi_connection.cursor()
cursor.execute("PRAGMA foreign_keys=ON")
cursor.close()

class DBModel(Base):
"""A Model is a metadata model of all or part of a building."""

Expand All @@ -18,12 +26,13 @@ class DBModel(Base):
description: Mapped[str] = Column(Text(), default="", nullable=False)
graph_id: Mapped[str] = Column(String())
manifest_id: Mapped[int] = Column(
Integer, ForeignKey("shape_collection.id"), nullable=False
Integer, ForeignKey("shape_collection.id", ondelete="CASCADE"), nullable=False
)
manifest: "DBShapeCollection" = relationship(
"DBShapeCollection",
uselist=False,
cascade="all,delete",
cascade="all",
passive_deletes=True,
)


Expand All @@ -45,16 +54,17 @@ class DBLibrary(Base):
name: Mapped[str] = Column(String(), nullable=False, unique=True)

templates: Mapped[List["DBTemplate"]] = relationship(
"DBTemplate", back_populates="library", cascade="all,delete"
"DBTemplate", back_populates="library", cascade="all", passive_deletes=True
)

shape_collection_id = Column(
Integer, ForeignKey("shape_collection.id"), nullable=False
Integer, ForeignKey("shape_collection.id", ondelete="CASCADE"), nullable=False
)
shape_collection: DBShapeCollection = relationship(
"DBShapeCollection",
uselist=False,
cascade="all,delete",
cascade="all",
passive_deletes=True,
)


Expand All @@ -64,8 +74,8 @@ class DepsAssociation(Base):
__tablename__ = "deps_association_table"

id: Mapped[int] = Column(Integer, primary_key=True)
dependant_id: Mapped[int] = Column(ForeignKey("template.id"))
dependee_id: Mapped[int] = Column(ForeignKey("template.id"))
dependant_id: Mapped[int] = Column(ForeignKey("template.id", ondelete="CASCADE"))
dependee_id: Mapped[int] = Column(ForeignKey("template.id", ondelete="CASCADE"))
# args are a mapping of dependee args to dependant args
args: Mapped[Dict[str, str]] = Column(JSONType) # type: ignore

Expand All @@ -89,21 +99,25 @@ class DBTemplate(Base):
body_id: Mapped[str] = Column(String())
optional_args: Mapped[List[str]] = Column(JSONType) # type: ignore

library_id: Mapped[int] = Column(Integer, ForeignKey("library.id"), nullable=False)
library_id: Mapped[int] = Column(Integer, ForeignKey("library.id", ondelete="CASCADE"), nullable=False)
library: Mapped[DBLibrary] = relationship("DBLibrary", back_populates="templates")
dependencies: Mapped[List["DBTemplate"]] = relationship(
"DBTemplate",
secondary="deps_association_table",
primaryjoin=id == DepsAssociation.dependant_id,
secondaryjoin=id == DepsAssociation.dependee_id,
back_populates="dependants",
cascade="all",
passive_deletes=True,
)
dependants: Mapped[List["DBTemplate"]] = relationship(
"DBTemplate",
secondary="deps_association_table",
primaryjoin=id == DepsAssociation.dependee_id,
secondaryjoin=id == DepsAssociation.dependant_id,
back_populates="dependencies",
cascade="all",
passive_deletes=True,
)

__table_args__ = (
Expand Down
Loading
Loading