feature/progressive-function-extractors #190

ndortega · 2024-04-25T03:22:38Z

added support for query params w/ default values
Added support for Extractors
Global and local validators for extractors

…ters from function and building structs from plain dictionaries (useful in queryparams and forms) 2.) added deps for the above feature and also included Revise for future autoreloading

…es from function signatures

… from the HTTP.Request object 2.) updated extractor tests

…o a route with no path params.

2.) added protobuf extractor & tests 3.) struct_builder now works recursively with deeply nested json and types

…-function-extractors

2.) Extrators are now defined utilizing a macro 3.) added a validate and type properties to Extractors

…pes from extractors

2.) removed unused code 3.) added more tests 4.) Added a TestPackage to load the oxygen as a dependency & included tests to run against it to check for precompilation issues

2.) updated readme docs

JanisErdmanis · 2024-04-29T12:16:13Z

This errors on a following line in PeaceFounderAdmin:

spec = Mapper.BRAID_CHAIN.ledger[index]

Here Mapper is imported from PeaceFounder and BRAID_CHAIN is a global mutable variable there initialized at runtime. The full error is as follows:

julia> using PeaceFounderAdmin
Precompiling PeaceFounderAdmin
        Info Given PeaceFounderAdmin was explicitly requested, output will be shown live 
ERROR: LoadError: UndefVarError: `BRAID_CHAIN` not defined
Stacktrace:
  [1] getargvalue
    @ ~/.julia/packages/Oxygen/IQnes/src/reflection.jl:12 [inlined]
  [2] splitargs(args::Vector{GlobalRef}, func_name::Symbol)
    @ Oxygen.Core.Reflection ~/.julia/packages/Oxygen/IQnes/src/reflection.jl:79
  [3] extract_defaults(info::Vector{Core.CodeInfo}, func_name::Symbol, param_names::Vector{Symbol}, kwarg_names::Vector{Symbol}; start::Int64)
    @ Oxygen.Core.Reflection ~/.julia/packages/Oxygen/IQnes/src/reflection.jl:238
  [4] extract_defaults
    @ ~/.julia/packages/Oxygen/IQnes/src/reflection.jl:225 [inlined]
  [5] splitdef(f::Function; start::Int64)
    @ Oxygen.Core.Reflection ~/.julia/packages/Oxygen/IQnes/src/reflection.jl:274
  [6] splitdef
    @ ~/.julia/packages/Oxygen/IQnes/src/reflection.jl:262 [inlined]
  [7] parse_func_params(route::String, func::Function)
    @ Oxygen.Core ~/.julia/packages/Oxygen/IQnes/src/core.jl:435
  [8] register(ctx::Oxygen.Core.AppContext.Context, httpmethod::String, route::String, func::Function)
    @ Oxygen.Core ~/.julia/packages/Oxygen/IQnes/src/core.jl:493
  [9] route(methods::Vector{String}, path::String, func::Function)
    @ PeaceFounderAdmin.AdminService ~/.julia/packages/Oxygen/IQnes/src/methods.jl:216
 [10] top-level scope
    @ ~/BtSync/PeaceFounder/Github/PeaceFounderAdmin/src/braidchain.jl:78
 [11] include(mod::Module, _path::String)
    @ Base ./Base.jl:495
 [12] include(x::String)
    @ PeaceFounderAdmin ~/BtSync/PeaceFounder/Github/PeaceFounderAdmin/src/PeaceFounderAdmin.jl:1
 [13] top-level scope
    @ ~/BtSync/PeaceFounder/Github/PeaceFounderAdmin/src/PeaceFounderAdmin.jl:22
 [14] include
    @ ./Base.jl:495 [inlined]
 [15] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2222
 [16] top-level scope
    @ stdin:3
in expression starting at /Users/jerdmanis/BtSync/PeaceFounder/Github/PeaceFounderAdmin/src/braidchain.jl:78
in expression starting at /Users/jerdmanis/BtSync/PeaceFounder/Github/PeaceFounderAdmin/src/PeaceFounderAdmin.jl:1
in expression starting at stdin:3
  ✗ PeaceFounderAdmin
  0 dependencies successfully precompiled in 4 seconds. 73 already precompiled.

ERROR: The following 1 direct dependency failed to precompile:

PeaceFounderAdmin [71cbcc19-d5de-42ce-8f1e-a2372a8e1f1c]

Failed to precompile PeaceFounderAdmin [71cbcc19-d5de-42ce-8f1e-a2372a8e1f1c] to "/Users/jerdmanis/.julia/compiled/v1.10/PeaceFounderAdmin/jl_3yKc5Z".
ERROR: LoadError: UndefVarError: `BRAID_CHAIN` not defined
Stacktrace:
  [1] getargvalue
    @ ~/.julia/packages/Oxygen/IQnes/src/reflection.jl:12 [inlined]
  [2] splitargs(args::Vector{GlobalRef}, func_name::Symbol)
    @ Oxygen.Core.Reflection ~/.julia/packages/Oxygen/IQnes/src/reflection.jl:79
  [3] extract_defaults(info::Vector{Core.CodeInfo}, func_name::Symbol, param_names::Vector{Symbol}, kwarg_names::Vector{Symbol}; start::Int64)
    @ Oxygen.Core.Reflection ~/.julia/packages/Oxygen/IQnes/src/reflection.jl:238
  [4] extract_defaults
    @ ~/.julia/packages/Oxygen/IQnes/src/reflection.jl:225 [inlined]
  [5] splitdef(f::Function; start::Int64)
    @ Oxygen.Core.Reflection ~/.julia/packages/Oxygen/IQnes/src/reflection.jl:274
  [6] splitdef
    @ ~/.julia/packages/Oxygen/IQnes/src/reflection.jl:262 [inlined]
  [7] parse_func_params(route::String, func::Function)
    @ Oxygen.Core ~/.julia/packages/Oxygen/IQnes/src/core.jl:435
  [8] register(ctx::Oxygen.Core.AppContext.Context, httpmethod::String, route::String, func::Function)
    @ Oxygen.Core ~/.julia/packages/Oxygen/IQnes/src/core.jl:493
  [9] route(methods::Vector{String}, path::String, func::Function)
    @ PeaceFounderAdmin.AdminService ~/.julia/packages/Oxygen/IQnes/src/methods.jl:216
 [10] top-level scope
    @ ~/BtSync/PeaceFounder/Github/PeaceFounderAdmin/src/braidchain.jl:78
 [11] include(mod::Module, _path::String)
    @ Base ./Base.jl:495
 [12] include(x::String)
    @ PeaceFounderAdmin ~/BtSync/PeaceFounder/Github/PeaceFounderAdmin/src/PeaceFounderAdmin.jl:1
 [13] top-level scope
    @ ~/BtSync/PeaceFounder/Github/PeaceFounderAdmin/src/PeaceFounderAdmin.jl:22
 [14] include
    @ ./Base.jl:495 [inlined]
 [15] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2222
 [16] top-level scope
    @ stdin:3
in expression starting at /Users/jerdmanis/BtSync/PeaceFounder/Github/PeaceFounderAdmin/src/braidchain.jl:78
in expression starting at /Users/jerdmanis/BtSync/PeaceFounder/Github/PeaceFounderAdmin/src/PeaceFounderAdmin.jl:1
in expression starting at stdin:
Stacktrace:
  [1] pkgerror(msg::String)
    @ Pkg.Types ~/.julia/juliaup/julia-1.10.2+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/Types.jl:70
  [2] precompile(ctx::Pkg.Types.Context, pkgs::Vector{…}; internal_call::Bool, strict::Bool, warn_loaded::Bool, already_instantiated::Bool, timing::Bool, _from_loading::Bool, kwargs::@Kwargs{…})
    @ Pkg.API ~/.julia/juliaup/julia-1.10.2+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/API.jl:1659
  [3] precompile(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::@Kwargs{_from_loading::Bool})
    @ Pkg.API ~/.julia/juliaup/julia-1.10.2+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/API.jl:159
  [4] precompile
    @ ~/.julia/juliaup/julia-1.10.2+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/API.jl:147 [inlined]
  [5] #precompile#114
    @ ~/.julia/juliaup/julia-1.10.2+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/API.jl:146 [inlined]
  [6] #invokelatest#2
    @ ./essentials.jl:894 [inlined]
  [7] invokelatest
    @ ./essentials.jl:889 [inlined]
  [8] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1963
  [9] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1812
 [10] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [11] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [12] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1803
 [13] macro expansion
    @ ./loading.jl:1790 [inlined]
 [14] macro expansion
    @ ./lock.jl:267 [inlined]
 [15] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1753
 [16] #invoke_in_world#3
    @ ./essentials.jl:926 [inlined]
 [17] invoke_in_world
    @ ./essentials.jl:923 [inlined]
 [18] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1746
Some type information was truncated. Use `show(err)` to see complete types.

ndortega · 2024-04-29T12:55:35Z

Hi @JanisErdmanis,

Thanks for the error report. It looks like my reflection code is trying to parse a CodeInfo object that represents the main body of a function.

Oxygen only cares about the function signature, so I've added some checks to try to skip everything else. Clearly, this check isn't working as intended - I'll have to see why its failing in this case.

Feel free to actually open an issue for this since this will probably affect other package authors.

I've also added a new test package in this update to check for precompilation issues. Once I figure out the reason, I'll try to recreate that case inside that test package

ndortega · 2024-04-29T21:13:27Z

Hi @JanisErdmanis,

I've updated the piece of code that's causing this issue and opened a PR. Can you manually test this branch and let me know if the issue still persists

PR: #192
Branch: hotfix/signature-parsing-bugifix

JanisErdmanis · 2024-04-30T08:02:53Z

This fixed the error, and everything else seems to work as expected.

ndortega added 22 commits April 6, 2024 23:31

added example to parse out info from function and from structs

3711a52

1.) added reflection module in oxygen to help with parsing out parame…

539f010

…ters from function and building structs from plain dictionaries (useful in queryparams and forms) 2.) added deps for the above feature and also included Revise for future autoreloading

added extractors

8f3113e

added missing export

4be8c2e

saving current progress

bf8994a

replaced external deps with custom reflection utils

ce3c1cb

saving current progress towards alternate route parser

e576542

saving current progress

46cdb67

fixed reflection bugs, when parsing signatures with globalrefs

8049fbc

fixed bug where duplicate types weren't getting tracked in function sig

ea6e705

add reflection tests for a regular function

4e4ac7e

added extractor tests and fixed more bugs around pulling default valu…

6aedaa8

…es from function signatures

1.) added a lazily initialized wrapper for storing commonly used data…

f5d3da0

… from the HTTP.Request object 2.) updated extractor tests

1.) Now properly handling cases where no additional args are mapped t…

819a622

…o a route with no path params.

1.) renamed extrator() to extract()

9a29bbb

2.) added protobuf extractor & tests 3.) struct_builder now works recursively with deeply nested json and types

removed revise

3a07741

Merge remote-tracking branch 'origin/master' into feature/progressive…

e623c2d

…-function-extractors

1.) added a new IR parsing strategy

5f9f123

2.) Extrators are now defined utilizing a macro 3.) added a validate and type properties to Extractors

cleaned up reflection and added helper functions to extract nested ty…

8400b18

…pes from extractors

all test cases now run with the new function parsing strategy

3b04ce5

removed unused functions

cadad84

added more extractor tests

231c498

ndortega linked an issue Apr 25, 2024 that may be closed by this pull request

feature: ergonomic extractors #183

Closed

added more test cases and cleaned up unused code

9022ca2

ndortega changed the title ~~feature/experimental-progressive-function-extractors~~ feature/progressive-function-extractors Apr 25, 2024

ndortega added 4 commits April 25, 2024 19:03

1.) now global and local validators can be used in tandem

c85845c

2.) removed unused code 3.) added more tests 4.) Added a TestPackage to load the oxygen as a dependency & included tests to run against it to check for precompilation issues

updated test coverage

bf5d12a

started to include docs in the readme

d9dc5b7

1.) Fixed extrator constructors

bfbf527

2.) updated readme docs

ndortega self-assigned this Apr 29, 2024

updated readme

9d128cf

ndortega merged commit 1e14091 into master Apr 29, 2024
3 of 6 checks passed

ndortega deleted the feature/experimental-progressive-function-extractors branch April 29, 2024 03:14

ndortega mentioned this pull request Apr 29, 2024

Reflection Parsing Issue #191

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feature/progressive-function-extractors #190

feature/progressive-function-extractors #190

ndortega commented Apr 25, 2024

JanisErdmanis commented Apr 29, 2024

ndortega commented Apr 29, 2024

ndortega commented Apr 29, 2024

JanisErdmanis commented Apr 30, 2024 •

edited

Loading

feature/progressive-function-extractors #190

feature/progressive-function-extractors #190

Conversation

ndortega commented Apr 25, 2024

JanisErdmanis commented Apr 29, 2024

ndortega commented Apr 29, 2024

ndortega commented Apr 29, 2024

JanisErdmanis commented Apr 30, 2024 • edited Loading

JanisErdmanis commented Apr 30, 2024 •

edited

Loading