Skip to content

Commit

Permalink
Rebase staging (#343)
Browse files Browse the repository at this point in the history 
* Release/1.1.3 (#290)

* [MLOP-634] Butterfree dev workflow, set triggers for branches staging and master (#280)

* Change github actions pipelines.

* Change pipeline logic.

* [BUG] Fix Staging GithubActions Pipeline (#283)

* New step on pipelie.

* Some adjusts.

* Apply only wheel. (#285)

* [BUG] Change version on setup.py to PyPI (#286)

* Add new make command to change version.

* Change command order.

* Change desc and variable name.

* Change command name.

* Keep milliseconds when using 'from_ms' argument in timestamp feature (#284)

* changed timestamp resolution

* fix import

* simple refactor

Co-authored-by: Henrique Camargo <[email protected]>

* Change trigger for pipeline staging (#287)

* Change trigger to publish dev pipeline.

* Some fix.

* Create a dev package. (#288)

* [MLOP-633] Butterfree dev workflow, update documentation (#281)

* Update workflow doc.

* Update README

* Add pre-release.

* Fix typo.

* [MLOP-632] Butterfree dev workflow, automate release description (#279)

* release 1.1.4

* update changelog

Co-authored-by: Mayara Moromisato <[email protected]>
Co-authored-by: Henrique Camargo <[email protected]>
Co-authored-by: AlvaroMarquesAndrade <[email protected]>

* Release/1.2.0 (#328)

* [MLOP-636] Create migration classes (#282)

* [MLOP-635] Rebase Incremental Job/Interval Run branch for test on selected feature sets (#278)

* Add interval branch modifications.

* Add interval_runs notebook.

* Add tests.

* Apply style (black, flack8 and mypy).

* Fix tests.

* Change version to create package dev.

* Allow slide selection (#293)

* Fix Slide Duration Typo (#295)

* [MLOP-637] Implement diff method (#292)

* [MLOP-640] Create CLI with migrate command (#298)

* [MLOP-645] Implement query method, cassandra (#291)

* [MLOP-671] Implement get_schema on Spark client (#301)

* [MLOP-648] Implement query method, metastore (#294)

* Fix Validation Step (#302)

* [MLOP-647] [MLOP-646] Apply migrations (#300)

* add apply migration method

* add test apply migration

* add migrate actor with tests

* mypy compliant

* fix test interaction with mocked object

* Rebase and some adjusts.

Co-authored-by: Mayara Moromisato <[email protected]>

* [BUG] Apply create_partitions to historical validate (#303)

* Apply create_partitions to historical validate.

* Remove comments and adjusts.

* [BUG] Fix key path for validate read (#304)

* Fix key path

* bump version

Co-authored-by: AlvaroMarquesAndrade <1a789766b1c4c8b679e80f11fa6d63d42fa4bcdf>

* [FIX] Add Partition types for Metastore (#305)

* [MLOP-639] Track logs in S3 (#306)

* Apply tracking logs and logging config.

* Adjusts in CLI and logging.conf.

* Some adjusts.

* Change version to generate new dev package

* Fix version.

* Apply style.

* Add new assert in the migrate unit test.

* [BUG] Change logging config (#307)

* Change logging config.

* Some adjusts.

* Remove a code smell.

* Change solution for tracking logs (#308)

* Change tracking logs method.

* Change version to generate dev package.

* Change path name in S3

* Read and write consistency level options (#309)

* modify cassandra client to be region aware

* add option for the user to set read and write consistency levels on cassandra config

* add tests

* use env vars instead

* Update butterfree/configs/db/cassandra_config.py

Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>

* Update butterfree/configs/db/cassandra_config.py

Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>

Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>

* Fix kafka reader. (#310)

* Fix path validate. (#311)

* Add local dc property (#312)

* add local dc property

* update version

* Remove metastore migrate (#313)

* Remove metastore migrate.

* Change version to create a dev package.

* Fix link in our docs. (#315)

* [BUG] Fix Cassandra Connect Session (#316)

* Fix Cassandra Connect Session.

* Apply style.

* Fix migration query. (#318)

* Fix migration query add type key. (#319)

* Fix db-config condition (#321)

* Fix db-config condition.

* Apply style.

* MLOP-642 Document migration in Butterfree (#320)

* update docs

* add more information and reference new cli.md file

* [MLOP-702] Debug mode for Automate Migration (#322)

* Create flag debug-mode.

* Fix tests.

* Fix migrate test.

* [MLOP-727] Improve logging messages (#325)

* Fix logging message for local file

* Remove json import

* [MLOP-728] Improve logging messages (#324)

* Improve logs.

* Revert debug-mode condition.

* Fix method to generate agg feature name. (#326)

* [MLOP-691]  Include step to add partition to SparkMetastore during writing of Butterfree (#327)

* Change writer type for interval mode.

* Some adjusts.

* Release 1.2.0

Co-authored-by: AlvaroMarquesAndrade <[email protected]>
Co-authored-by: Igor Gustavo Hoelscher <[email protected]>
Co-authored-by: Felipe Victorino Caputo <[email protected]>
Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>
Co-authored-by: Gabriel Brandão <[email protected]>

* Release 1.2.1 (#338)

* Add the missing link for H3 geohash (#330)

* Add the missing link for H3 geohash

* Update the H3 geohash link.

* Update the same link 

Update the same link in in spark_function_and_window.ipynb example

* Update README.md (#331)

* Update Github Actions Workflow runner (#332)

* Update Workflow runner version

* bump flake8-bandit

* chore: bypass false positive for S105

Co-authored-by: Lucas Cardozo <[email protected]>

* Delete sphinx version. (#334)

* Update files to staging (#336)

Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>

* Update butterfree/configs/db/cassandra_config.py

Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>

Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>

* Fix kafka reader. (#310)

* Fix path validate. (#311)

* Add local dc property (#312)

* release 1.2.1

Co-authored-by: Jay Vala <[email protected]>
Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>
Co-authored-by: Lucas Fonseca <[email protected]>
Co-authored-by: Lucas Cardozo <[email protected]>
Co-authored-by: Felipe Victorino Caputo <[email protected]>

* [BUG] Fix Staging GithubActions Pipeline (#283)

* New step on pipelie.

* Some adjusts.

* Apply only wheel. (#285)

* [BUG] Change version on setup.py to PyPI (#286)

* Add new make command to change version.

* Change command order.

* Change desc and variable name.

* Change command name.

* [MLOP-636] Create migration classes (#282)

* [MLOP-635] Rebase Incremental Job/Interval Run branch for test on selected feature sets (#278)

* Add interval branch modifications.

* Add interval_runs notebook.

* Add tests.

* Apply style (black, flack8 and mypy).

* Fix tests.

* Change version to create package dev.

* [MLOP-637] Implement diff method (#292)

* [MLOP-640] Create CLI with migrate command (#298)

* [MLOP-645] Implement query method, cassandra (#291)

* [MLOP-648] Implement query method, metastore (#294)

* Fix Validation Step (#302)

* [MLOP-647] [MLOP-646] Apply migrations (#300)

* add apply migration method

* add test apply migration

* add migrate actor with tests

* mypy compliant

* fix test interaction with mocked object

* Rebase and some adjusts.

Co-authored-by: Mayara Moromisato <[email protected]>

* [MLOP-639] Track logs in S3 (#306)

* Apply tracking logs and logging config.

* Adjusts in CLI and logging.conf.

* Some adjusts.

* Change version to generate new dev package

* Fix version.

* Apply style.

* Add new assert in the migrate unit test.

* [BUG] Change logging config (#307)

* Change logging config.

* Some adjusts.

* Remove a code smell.

* Change solution for tracking logs (#308)

* Change tracking logs method.

* Change version to generate dev package.

* Change path name in S3

* Remove metastore migrate (#313)

* Remove metastore migrate.

* Change version to create a dev package.

* [MLOP-727] Improve logging messages (#325)

* Fix logging message for local file

* Remove json import

* [MLOP-728] Improve logging messages (#324)

* Improve logs.

* Revert debug-mode condition.

* [MLOP-691]  Include step to add partition to SparkMetastore during writing of Butterfree (#327)

* Change writer type for interval mode.

* Some adjusts.

* Update files to staging (#336)

* Release/1.1.3 (#290)

* [MLOP-634] Butterfree dev workflow, set triggers for branches staging and master (#280)

* Change github actions pipelines.

* Change pipeline logic.

* [BUG] Fix Staging GithubActions Pipeline (#283)

* New step on pipelie.

* Some adjusts.

* Apply only wheel. (#285)

* [BUG] Change version on setup.py to PyPI (#286)

* Add new make command to change version.

* Change command order.

* Change desc and variable name.

* Change command name.

* Keep milliseconds when using 'from_ms' argument in timestamp feature (#284)

* changed timestamp resolution

* fix import

* simple refactor

Co-authored-by: Henrique Camargo <[email protected]>

* Change trigger for pipeline staging (#287)

* Change trigger to publish dev pipeline.

* Some fix.

* Create a dev package. (#288)

* [MLOP-633] Butterfree dev workflow, update documentation (#281)

* Update workflow doc.

* Update README

* Add pre-release.

* Fix typo.

* [MLOP-632] Butterfree dev workflow, automate release description (#279)

* release 1.1.4

* update changelog

Co-authored-by: Mayara Moromisato <[email protected]>
Co-authored-by: Henrique Camargo <[email protected]>
Co-authored-by: AlvaroMarquesAndrade <[email protected]>

* Release/1.2.0 (#328)

* [MLOP-636] Create migration classes (#282)

* [MLOP-635] Rebase Incremental Job/Interval Run branch for test on selected feature sets (#278)

* Add interval branch modifications.

* Add interval_runs notebook.

* Add tests.

* Apply style (black, flack8 and mypy).

* Fix tests.

* Change version to create package dev.

* Allow slide selection (#293)

* Fix Slide Duration Typo (#295)

* [MLOP-637] Implement diff method (#292)

* [MLOP-640] Create CLI with migrate command (#298)

* [MLOP-645] Implement query method, cassandra (#291)

* [MLOP-671] Implement get_schema on Spark client (#301)

* [MLOP-648] Implement query method, metastore (#294)

* Fix Validation Step (#302)

* [MLOP-647] [MLOP-646] Apply migrations (#300)

* add apply migration method

* add test apply migration

* add migrate actor with tests

* mypy compliant

* fix test interaction with mocked object

* Rebase and some adjusts.

Co-authored-by: Mayara Moromisato <[email protected]>

* [BUG] Apply create_partitions to historical validate (#303)

* Apply create_partitions to historical validate.

* Remove comments and adjusts.

* [BUG] Fix key path for validate read (#304)

* Fix key path

* bump version

Co-authored-by: AlvaroMarquesAndrade <1a789766b1c4c8b679e80f11fa6d63d42fa4bcdf>

* [FIX] Add Partition types for Metastore (#305)

* [MLOP-639] Track logs in S3 (#306)

* Apply tracking logs and logging config.

* Adjusts in CLI and logging.conf.

* Some adjusts.

* Change version to generate new dev package

* Fix version.

* Apply style.

* Add new assert in the migrate unit test.

* [BUG] Change logging config (#307)

* Change logging config.

* Some adjusts.

* Remove a code smell.

* Change solution for tracking logs (#308)

* Change tracking logs method.

* Change version to generate dev package.

* Change path name in S3

* Read and write consistency level options (#309)

* modify cassandra client to be region aware

* add option for the user to set read and write consistency levels on cassandra config

* add tests

* use env vars instead

* Update butterfree/configs/db/cassandra_config.py

Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>

* Update butterfree/configs/db/cassandra_config.py

Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>

Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>

* Fix kafka reader. (#310)

* Fix path validate. (#311)

* Add local dc property (#312)

* add local dc property

* update version

* Remove metastore migrate (#313)

* Remove metastore migrate.

* Change version to create a dev package.

* Fix link in our docs. (#315)

* [BUG] Fix Cassandra Connect Session (#316)

* Fix Cassandra Connect Session.

* Apply style.

* Fix migration query. (#318)

* Fix migration query add type key. (#319)

* Fix db-config condition (#321)

* Fix db-config condition.

* Apply style.

* MLOP-642 Document migration in Butterfree (#320)

* update docs

* add more information and reference new cli.md file

* [MLOP-702] Debug mode for Automate Migration (#322)

* Create flag debug-mode.

* Fix tests.

* Fix migrate test.

* [MLOP-727] Improve logging messages (#325)

* Fix logging message for local file

* Remove json import

* [MLOP-728] Improve logging messages (#324)

* Improve logs.

* Revert debug-mode condition.

* Fix method to generate agg feature name. (#326)

* [MLOP-691]  Include step to add partition to SparkMetastore during writing of Butterfree (#327)

* Change writer type for interval mode.

* Some adjusts.

* Release 1.2.0

Co-authored-by: AlvaroMarquesAndrade <[email protected]>
Co-authored-by: Igor Gustavo Hoelscher <[email protected]>
Co-authored-by: Felipe Victorino Caputo <[email protected]>
Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>
Co-authored-by: Gabriel Brandão <[email protected]>

Co-authored-by: hmeretti <[email protected]>
Co-authored-by: Henrique Camargo <[email protected]>
Co-authored-by: AlvaroMarquesAndrade <[email protected]>
Co-authored-by: Igor Gustavo Hoelscher <[email protected]>
Co-authored-by: Felipe Victorino Caputo <[email protected]>
Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>
Co-authored-by: Gabriel Brandão <[email protected]>

* Revert "Update files to staging (#336)" (#337)

This reverts commit 35bcd30.

* Less strict requirements (#333)

* bump a few requirements; increase lower bound for h3 version range; adding pyarrow dev dependency

* fix type repr for spark types; fix: broken tests (pyspark 3.4)

---------

Co-authored-by: Ralph Rassweiler <[email protected]>

* feat: optional row count validation (#340)

* fix: parameter, libs (#341)

---------

Co-authored-by: hmeretti <[email protected]>
Co-authored-by: Mayara Moromisato <[email protected]>
Co-authored-by: Henrique Camargo <[email protected]>
Co-authored-by: AlvaroMarquesAndrade <[email protected]>
Co-authored-by: Igor Gustavo Hoelscher <[email protected]>
Co-authored-by: Felipe Victorino Caputo <[email protected]>
Co-authored-by: Rodrigo Martins de Oliveira <[email protected]>
Co-authored-by: Gabriel Brandão <[email protected]>
Co-authored-by: Jay Vala <[email protected]>
Co-authored-by: Lucas Fonseca <[email protected]>
Co-authored-by: Lucas Cardozo <[email protected]>
Co-authored-by: Mayara Moromisato <[email protected]>
  • Loading branch information
@ralphrass @hmeretti
@moromimay @AlvaroMarquesAndrade @roelschr @github-felipe-caputo @GaBrandao @jdvala @lecardozo
13 people authored Aug 21, 2023
1 parent 60c7ee4 commit f35d665
Showing 0 changed files with 0 additions and 0 deletions.

1 comment on commit f35d665

@chip-n-dale
Copy link

@chip-n-dale chip-n-dale bot commented on f35d665 Aug 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ralphrass!

The GitLeaks SecTool reported some possibly exposed credentials/secrets, how about giving them a look?

GitLeaks Alert Sync 
[
  {
    "line": "    webhook: REDACTED",
    "lineNumber": 141,
    "offender": "REDACTED",
    "offenderEntropy": -1,
    "commit": "b6a5daf28abc035f74b9685aab573d384680b9d1",
    "repo": "butterfree",
    "repoURL": "",
    "leakURL": "",
    "rule": "Slack Webhook",
    "commitMessage": "initial commit\n",
    "author": "Alvaro",
    "email": "[email protected]",
    "file": ".drone.yml",
    "date": "2020-01-03T14:21:51-03:00",
    "tags": "key, slack"
  },
  {
    "line": "    webhook: REDACTED",
    "lineNumber": 159,
    "offender": "REDACTED",
    "offenderEntropy": -1,
    "commit": "b6697aa708fec0c5a9e3af0b2713cee6f45ff675",
    "repo": "butterfree",
    "repoURL": "",
    "leakURL": "",
    "rule": "Slack Webhook",
    "commitMessage": "hail to the butterfree\n",
    "author": "Alvaro",
    "email": "[email protected]",
    "file": ".drone.yml",
    "date": "2020-01-03T11:07:44-03:00",
    "tags": "key, slack"
  }
]

In case of false-positives, more information is available on GitLeaks FAQ
If you had any other problem or question during this process, be sure to contact us on the Security space on GChat!

Please sign in to comment.