Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update CAmp to use the new ACE release #3

Open
wants to merge 64 commits into
base: apl-fy24
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 56 commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
85cedd3
Update readme for yang adms and non-ION DTNMA C agent
jennylcao May 1, 2024
d90b674
change AdmFile to AdmModule
karenhaining Aug 13, 2024
87b0006
use most up-to-date ace branch in CI
karenhaining Aug 13, 2024
496d9fd
change Mdat to MetadataItem
karenhaining Aug 13, 2024
a69e7fd
try AdmObjMixin instead
karenhaining Aug 13, 2024
9ed099a
fix Mdat => Ident
karenhaining Aug 14, 2024
b90cbfe
use other branch for ace test env
karenhaining Aug 14, 2024
a6fce0e
update tests to use yang files instead of json
karenhaining Aug 14, 2024
cb9ed64
extract adms repo into env var
karenhaining Aug 14, 2024
b7d2dd4
change the minimum required metadata
karenhaining Aug 14, 2024
ce1e4f7
remove unnecessary check
karenhaining Aug 14, 2024
c1fefbe
change namespaces to name
karenhaining Aug 14, 2024
5656cac
change mdat to ident?
karenhaining Aug 14, 2024
ad4d74f
Merge branch 'ep2-dev' into haining-ace-updates
karenhaining Aug 15, 2024
0170b19
missed namespaces, temp take out rpt, tblt, macs
karenhaining Aug 15, 2024
f846274
change obj.type to obj.typeobj
karenhaining Aug 15, 2024
41d7934
fix const value => init_value
karenhaining Aug 15, 2024
19bce9a
fix oper result_type => obj.result.typeobj
karenhaining Aug 15, 2024
36334f9
temp ignore meta, mac, rptt, tblt
karenhaining Aug 15, 2024
d0665fa
fix type obj to strings
karenhaining Aug 15, 2024
d2ed0cb
ignore var for now
karenhaining Aug 15, 2024
6bc2fae
fix mostly parmspec => parameters
karenhaining Aug 15, 2024
d21cc76
fix get_child w new ace
karenhaining Aug 15, 2024
537b57e
more type fixes
karenhaining Aug 15, 2024
99ea611
cont ignore var
karenhaining Aug 15, 2024
cae92fe
compiles and runs! 🎉
karenhaining Aug 15, 2024
686b64e
fix const value
karenhaining Aug 15, 2024
54fc749
at least type shows an actual value now... change it to more readable?
karenhaining Aug 15, 2024
30b087e
mostly jenny's changes from ep2-dev
karenhaining Aug 15, 2024
3749ad0
revert enum names to norm_name + upper() when applicable
karenhaining Aug 15, 2024
1a48272
handle yang identifier to c string translation
karenhaining Aug 16, 2024
b9ed967
metadata -- sanitize newlines so c output isn't broken across lines
karenhaining Aug 16, 2024
393e1d4
fix the nickname (should've still been enum)
karenhaining Aug 16, 2024
33e1084
remove mac, rptt, tblt
karenhaining Aug 16, 2024
64deb58
dummy amp type
karenhaining Aug 16, 2024
077264e
add var back in, remove more unused
karenhaining Aug 16, 2024
7391212
working c var
karenhaining Aug 23, 2024
a3ebc59
no error running camp on all yang examples in dtnma-adms! 🎉
karenhaining Aug 23, 2024
c60c0f5
touch up sql file
karenhaining Aug 23, 2024
599c972
fix wrong attr name
karenhaining Aug 23, 2024
ae68e16
condense yang to sql name logic
karenhaining Aug 23, 2024
716c6a2
pull out more yang_to_c calls
karenhaining Aug 23, 2024
4f7eceb
remove expected json failures
karenhaining Aug 26, 2024
d3c5cf7
install ace before this repo is checked out
karenhaining Aug 26, 2024
d2c3699
add "# TODO type fixes" to the hacks
karenhaining Aug 27, 2024
1df3366
standardize naming
karenhaining Aug 27, 2024
d16269d
remove mac, rptt, tblt
karenhaining Aug 27, 2024
c8de8c8
update sql integration test for yang
karenhaining Aug 27, 2024
d827089
passing sql integration tests (w/ type fix TODOs)
karenhaining Aug 28, 2024
1d3e64e
fix typos
karenhaining Aug 28, 2024
2a2e8ae
default git branch
karenhaining Aug 28, 2024
ecaaa42
add var back in
karenhaining Aug 29, 2024
320c744
xfail ietf-amm.yang (doesn't have unique enum)
karenhaining Aug 29, 2024
13d87b7
switch back branch
karenhaining Aug 29, 2024
ee7e500
add debug
karenhaining Aug 29, 2024
4def230
passing tests w var added back in
karenhaining Aug 29, 2024
4fdcd61
update docs sha
karenhaining Aug 29, 2024
63c084a
remove debug print
karenhaining Aug 29, 2024
e550511
fix comment
karenhaining Aug 29, 2024
27bf3f0
capture stdout for debug
karenhaining Aug 30, 2024
3eb3d24
clear cache
karenhaining Aug 30, 2024
f688672
force?
karenhaining Aug 30, 2024
c023173
remove the cache clear
karenhaining Aug 30, 2024
6168e9a
remove rpt tblt mac from database creation
karenhaining Aug 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions .github/workflows/build-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ name: Build and run tests
on: [push]

env:
ACE_REPO: "git+https://github.com/JHUAPL-DTNMA/dtnma-ace.git@aa879be"
ACE_REPO: "git+https://github.com/JHUAPL-DTNMA/dtnma-ace.git@haining-ace-yang-updates"
ADMS_REPO: "https://github.com/JHUAPL-DTNMA/dtnma-adms.git"

jobs:
build:
Expand Down Expand Up @@ -57,6 +58,10 @@ jobs:
matrix:
python-version: ["3.10"]
steps:
- name: Prep
run: |
pip3 install ${ACE_REPO}
pip3 install -e '.[test]'
- name: Checkout repository
uses: actions/checkout@v4
with:
Expand All @@ -65,10 +70,6 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Prep
run: |
pip3 install ${ACE_REPO}
pip3 install -e '.[test]'
- name: Test
run: python3 -m pytest -v --cov=camp tests/unit_test

Expand Down Expand Up @@ -135,7 +136,7 @@ jobs:
run: |
pip3 install ${ACE_REPO}
pip install -e '.[test]'
git clone https://github.com/JHUAPL-DTNMA/dtnma-adms.git tests/integration_test/anms-adms
git clone ${ADMS_REPO} tests/integration_test/anms-adms
- name: Test
run: python3 -m pytest -v --cov=camp tests/integration_test/test_sql_integration.py

Expand All @@ -161,7 +162,7 @@ jobs:
pip3 install -e '.[test]'
- name: Prep repos
run: |
git clone https://github.com/JHUAPL-DTNMA/dtnma-adms.git tests/integration_test/anms-adms
git clone ${ADMS_REPO} tests/integration_test/anms-adms
git clone https://github.com/JHUAPL-DTNMA/dtnma-tools.git tests/integration_test/dtnma-tools
sudo apt-get update
sudo apt-get install -y cmake ninja-build ruby build-essential autoconf libtool civetweb libcivetweb-dev libssl-dev libcjson-dev
Expand Down
74 changes: 21 additions & 53 deletions src/camp/generators/create_agent_c.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,16 @@ class Writer(AbstractWriter, CHelperMixin):

def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self._g_var_idx = "g_" + self.adm.norm_namespace.lower() + "_idx"
self.c_norm_name = cu.yang_to_c(self.adm.norm_name)
self._g_var_idx = "g_" + self.c_norm_name.lower() + "_idx"

def file_path(self) -> str:
# Interface for AbstractWriter
return os.path.join(self.out_path, "agent", f"adm_{self.adm.norm_name}_agent.c")
return os.path.join(self.out_path, "agent", f"adm_{self.c_norm_name}_agent.c")

def write(self, outfile: TextIO):
# Interface for AbstractWriter
campch.write_c_file_header(outfile, f"adm_{self.adm.norm_name}_agent.c")
campch.write_c_file_header(outfile, f"adm_{self.c_norm_name}_agent.c")
self.write_includes(outfile)

outfile.write("vec_idx_t {}[11];\n\n".format(self._g_var_idx))
Expand All @@ -57,9 +58,6 @@ def write(self, outfile: TextIO):
self.write_init_op_function(outfile)
self.write_init_var_function(outfile)
self.write_init_control_function(outfile)
self.write_init_macro_function(outfile)
self.write_init_reports_function(outfile)
self.write_init_tables_function(outfile)

#
# Writes all of the #includes for this c file
Expand All @@ -72,11 +70,11 @@ def write_includes(self, outfile):
files = [
"ion.h",
"platform.h",
f"adm_{self.adm.norm_name}.h",
f"adm_{self.c_norm_name}.h",
"shared/utils/utils.h",
"shared/primitives/report.h",
"shared/primitives/blob.h",
f"adm_{self.adm.norm_name}_impl.h",
f"adm_{self.c_norm_name}_impl.h",
"agent/rda.h",
]
outfile.write(campch.make_includes(files))
Expand Down Expand Up @@ -127,13 +125,13 @@ def write_init_metadata_function(self, outfile):
body = ""
add_str_template = self.make_std_meta_adm_build_template(cs.META)

for obj in self.adm.mdat:
for obj in self.adm.metadata_list.items:
_,fname,_ = campch.make_meta_function(self.adm, obj)
ari = cu.make_ari_name(self.adm.norm_namespace, cs.META, obj)
ari = cu.make_ari_name(self.c_norm_name, cs.META, obj)

body += add_str_template.format("0", ari, fname)

campch.write_formatted_init_function(outfile, self.adm.norm_namespace, cs.META, body)
campch.write_formatted_init_function(outfile, self.c_norm_name, cs.META, body)

#
# Constructs and writes the init_constants function
Expand All @@ -148,15 +146,15 @@ def write_init_constant_function(self, outfile):
for obj in self.adm.const:
parms_tf = "0"
_,fname,_ = campch.make_constant_function(self.adm, obj)
ari = cu.make_ari_name(self.adm.norm_namespace, cs.CONST, obj)
ari = cu.make_ari_name(self.c_norm_name, cs.CONST, obj)

#FIXME: can const have parameters?
# if obj.parmspec:
# if obj.paramspec:
# parms_tf = "1"

body += add_str.format(parms_tf, ari, fname)

campch.write_formatted_init_function(outfile, self.adm.norm_namespace, cs.CONST, body)
campch.write_formatted_init_function(outfile, self.c_norm_name, cs.CONST, body)

#
# Constructs and writes the init_edd function
Expand All @@ -171,14 +169,14 @@ def write_init_edd_function(self, outfile):
for obj in self.adm.edd:
parms_tf = "0"
_,fname,_ = campch.make_collect_function(self.adm, obj)
ari = cu.make_ari_name(self.adm.norm_namespace, cs.EDD, obj)
ari = cu.make_ari_name(self.c_norm_name, cs.EDD, obj)

if obj.parmspec and obj.parmspec.items:
if hasattr(obj, 'parameters') and obj.parameters.items:
parms_tf = "1"

body += add_str.format(parms_tf, ari, fname)

campch.write_formatted_init_function(outfile, self.adm.norm_namespace, cs.EDD, body)
campch.write_formatted_init_function(outfile, self.c_norm_name, cs.EDD, body)


#
Expand All @@ -193,12 +191,12 @@ def write_init_op_function(self, outfile):
adm_add_op_template = "\n\tadm_add_"+cs.get_sname(cs.OP)+"(" + self._g_var_idx + "["+cs.get_adm_idx(cs.OP)+"], {0}, {1}, {2});"

for obj in self.adm.oper:
ari = cu.make_ari_name(self.adm.norm_namespace, cs.OP, obj)
in_types = obj.in_type if obj.in_type else []
ari = cu.make_ari_name(self.c_norm_name, cs.OP, obj)
in_types = obj.operands.items if obj.operands.items else []

body += adm_add_op_template.format(ari, len(in_types), ari.lower())

campch.write_formatted_init_function(outfile, self.adm.norm_namespace, cs.OP, body)
campch.write_formatted_init_function(outfile, self.c_norm_name, cs.OP, body)

#
# Writes the init_variables body
Expand All @@ -220,39 +218,9 @@ def write_init_control_function(self, outfile):
adm_add_template = "\n\tadm_add_ctrldef(" + self._g_var_idx + "["+cs.get_adm_idx(cs.CTRL)+"], {0}, {1}, {2});"

for obj in self.adm.ctrl:
ari = cu.make_ari_name(self.adm.norm_namespace, cs.CTRL, obj)
parms = obj.parmspec.items if obj.parmspec else []
ari = cu.make_ari_name(self.c_norm_name, cs.CTRL, obj)
parms = obj.parameters.items if hasattr(obj, 'parameters') else []
body += adm_add_template.format(ari, len(parms), ari.lower())

campch.write_formatted_init_function(outfile, self.adm.norm_namespace, cs.CTRL, body)

#
# Constructs and writes the init_macros function
#
# c_file is an open file descriptor to write to,
# name is the value returned from get_adm_names()
# macros is a list of macros to add
#
def write_init_macro_function(self, outfile):
campch.write_init_macro_function(outfile, self.adm, self._g_var_idx, False)

#
# Constructs and writes the init reports function
#
# c_file is an open file descriptor to write to
# name is the name returned by the call to get_adm_names()
# retriever is the Retriever class instance for this ADM
#
def write_init_reports_function(self, outfile):
campch.write_parameterized_init_reports_function(outfile, self.adm, self._g_var_idx, False)

#
# Constructs and writes the init tables function
#
# c_file is an open file descriptor to write to
# name is the name returned by the call to get_adm_names()
# retriever is the Retriever class instance for this ADM
#
def write_init_tables_function(self, outfile):
campch.write_init_tables_function(outfile, self.adm, self._g_var_idx, False)
campch.write_formatted_init_function(outfile, self.c_norm_name, cs.CTRL, body)

Loading
Loading