Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci(fuzz tests): graphql fuzz testing #3760

Draft
wants to merge 72 commits into
base: main
Choose a base branch
from
Draft

ci(fuzz tests): graphql fuzz testing #3760

wants to merge 72 commits into from

Conversation

iainsproat
Copy link
Contributor

Description & motivation

⚠️ Draft, do not merge yet.

This adds fuzz testing of the graphql API.

Changes:

  • GitHub Action workflow to fuzz test the speckle server graphql API

To-do before merge:

Screenshots:

Validation of changes:

Checklist:

  • My pull request follows the guidelines in the Contributing guide?
  • My pull request does not duplicate any other open Pull Requests for the same update/change?
  • My commits are related to the pull request and do not amend unrelated code or documentation.
  • My code follows a similar style to existing code.
  • I have added appropriate tests.
  • I have updated or added relevant documentation.

References

@iainsproat
hack(tests): add fuzz testing workflow
d65e435
@iainsproat
fixes
e4bbf38
@iainsproat
Update path of binary
a026354
@iainsproat
Update binary path again, and add names to steps
e66d2c2
@iainsproat
More path fixes, too much nesting
3e18153
@iainsproat
Add debugging output
c8b96e4
@iainsproat
it is case-sensitive fml
48b7d2f
@iainsproat
Restler documentation is inconsistent on the output dir and there are…
ea36cd9 
… no logs
@iainsproat
Now output the file content to allow configuration
8ea40b3
@iainsproat
Attempt to run speckle-server and test the API
968e4e7
@iainsproat
fix minio service
45674f0
@iainsproat
Fix paths for yarn
bc4ca17
@iainsproat
set cache dependency path
95fbf4a
@iainsproat
Add a yarn start to server package.json
b2ebd47
@iainsproat
Caching
8686c00
@iainsproat
configure env vars for fuzz test
3d20d7a
@iainsproat
Printing results
a030b22
@iainsproat
caching was too agressive
272b9fa 
- override default settings
@iainsproat
overwrite minio entrypoint
d0c0c60
@iainsproat
More bug fixing
e19abdd
@iainsproat
remove network alias
2460fb9
@iainsproat
Bump OpenAPI specification version and add servers property
6cba30d
@iainsproat
Override host when testing
198a8cc
@iainsproat
Swap to docker compose
b76f20e
@iainsproat
fix postgres creds
e68da6a
@iainsproat
Wait for server to start running, and always print restler output
c8002de
@iainsproat
Wait until server is responding
fe446d5
@iainsproat
Do not fail on decoding of responses being unexpected unicode
ca46336
@iainsproat
Two step creation of grammar
f09ce25 
- update openapi specification for images in response body
@iainsproat
Fix restler config
62abdcb
@iainsproat
Print compile logs
c00c3bb
@iainsproat
Cache the restler binary and do not attempt to save cache if cache hit
5791e5a
@iainsproat
Attempt to fix dictionary
1f6bb11
@iainsproat
Update OpenAPI specification
6ff569b
@iainsproat
provide proper path to compiler
5f86467
@iainsproat
Bust cache to retry building
080a478
@iainsproat
feat(server): allow ratelimiting to be explicitly disabled
a3e17b8 
- allows rate limiter to be enabled or disabled explicitly
- previously implied based on whether NODE_ENV is 'test', which may not be desired behaviour
- example .env file for testing now explicitly disables it
@iainsproat
Disable rate limiter in CI
d2d6210
@iainsproat
Merge branch 'main' into iain/web-511-fuzz-test-speckle-server-rest-api
96bc9fc
@iainsproat
Merge branch 'iain/ratelimiter-can-be-explicitly-disabled' into iain/…
a5f15ed 
…web-511-fuzz-test-speckle-server-rest-api
@iainsproat
Explicitly disable ratelimiter
c188fa9
@iainsproat
More fixes to openapi specification
fef3198
@iainsproat
Remove caching
ed09a8c
@iainsproat
fix(server/blobstorage): handles errors with missing content-type header
49884b5
@iainsproat
More openapi specification improvements
b56ba47
@iainsproat
Provide default app Ids
29ee4f4
@iainsproat
Enhance OpenAPI specification
39efe09
@iainsproat
ci(fuzz tests): graphql fuzz testing
f30b04e
@linear Linear
Copy link

linear bot commented Jan 3, 2025

WEB-512 Fuzz test Speckle server Graphql API

@gjedlicska
Copy link
Contributor

What is our goal with this?

@iainsproat
Copy link
Contributor Author

What is our goal with this?

It was a personal festive season project.

Partially it's about discovering incorrect status codes (especially 5XX) or graphql errors we're returning in API responses. And partially about discovering actually-exploitable security vulnerabilities, if any.

It discovered 9 potential issues in the short 3 minutes test it ran. (And the equivalent REST API fuzzer caught a couple of issues I've fixed already.)

If it's useful, it could run weekly against HEAD of main branch rather than being a check (and delay/blocker) on PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@iainsproat @gjedlicska