Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prometheus Remote write - save report for later "remote write" #4117

Closed
JCzz opened this issue Dec 16, 2024 · 5 comments
Closed

Prometheus Remote write - save report for later "remote write" #4117

JCzz opened this issue Dec 16, 2024 · 5 comments
Assignees
Labels
awaiting user waiting for user to respond feature

Comments

@JCzz
Copy link

JCzz commented Dec 16, 2024

Feature Description

Hi

Feature Describing/scenario:
saving "--out json=output.json" for later - to comparing reports generated in pipeline, unable to reach Prometheus at pipeline runtime.

Suggested Solution (optional)

I am having issues writing "k6 run --out json=output.json"data to prometheus. I need to write a script that can upload the json file to Prometheus, maybe this could be a feature for k6 "remote write"?

Already existing or connected issues / PRs (optional)

No response

@ankur22
Copy link
Contributor

ankur22 commented Dec 17, 2024

Hi @JCzz,

What is your end goal of working with both the json output file and prometheus? As far as I'm aware, they are two ways of outputting the metrics that k6 generates when running a test.

Maybe an example test script, json output file and some POC screenshots would help me understand your use case.

@ankur22 ankur22 added awaiting user waiting for user to respond and removed triage labels Dec 17, 2024
@ankur22 ankur22 removed their assignment Dec 17, 2024
@JCzz
Copy link
Author

JCzz commented Dec 17, 2024

Hi @ankur22

Thanks for asking.

I am trying to convince my team and company to use k6 and Grafana for visualising performance tests. Problem is that we will not get a dedicated Prometheus server running(at first), so only option is to have Azure pipeline running k6 with output to json.

Then when we want to analyse performance tests, we need to load all the json files, from each pipeline k6 run, into Grafana and as I understand, the only way is via Prometheus... k6 >> json >> Prometheus >> Grafana.

Hope it makes more sense... please as again, if I am not making it any more clear.

Regards
Christian

@oleiade
Copy link
Member

oleiade commented Dec 17, 2024

Hi @JCzz 👋🏻

Based on my understanding of what you're after, and keeping in mind that I might be missing some context, and/or misunderstood something: I believe your objective is not achievable as-is with k6.

The JSON output is emitting a format that's specific to k6, with metric or sample per-line, and is not meant to be compatible with Prometheus specifically.

Furthermore as far as I'm aware, there is no official way to store prometheus metrics/samples to a file in a format that could be later replayed using the prometheus remote-write protocol (there might be a gap in my knowledge though, so please feel free to correct me if you believe I missed something).

So I don't believe there is neither an immediate way to allow to replay JSON results to Prometheus, nor is there a way to save a report to replay later using the remote write output, out of the box.

However, if you or anyone in the community has the capacity to experiment around that capability, it might be possible to build a script or small program that replays a k6 JSON output file and uses the prometheus remote-write functionality to a Prometheus instance. But I think this is unfortunately not something we'd build ourselves, nor bake directly into k6.

I hope this is helpful, let me know if you have further questions, happy to help 🙇🏻

@JCzz
Copy link
Author

JCzz commented Dec 17, 2024

Thanks @oleiade

Good points. In the light of what has been commented on this, I will try moving forward with some golang code, that will read in k6 json output files, and "remote write" to Prometheus. Thanks for looking at this issue and please let me know if you have any insights on how to read the k6 json data.

You are welcome to close this.

Regards & thanks for great products
Christian

@oleiade oleiade removed the triage label Dec 17, 2024
@oleiade
Copy link
Member

oleiade commented Dec 17, 2024

You're very welcome 🤝

If you reach a solution you like and you think might be beneficial to the community, feel free to come back around to show us and share what you came up with.

Will close this issue in the meantime 🙇🏻

@oleiade oleiade closed this as completed Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting user waiting for user to respond feature
Projects
None yet
Development

No branches or pull requests

4 participants