Thresholds with tags not capturing metric values for some API calls, returning zero instead #3518
Assignees
Labels
Comments
luissalgadofreire
changed the title
|
Hi @luissalgadofreire 👋🏻 Thanks for reporting this 🙇🏻 I can confirm I am able to reproduce the behavior you're observing in a simplified script that isolates it: import { check, group } from 'k6';
import http from 'k6/http';
export const options = {
iterations: 10,
thresholds: {
'http_req_duration{name: Get}': ["p(95)<1000"],
'http_req_duration{name: Delete}': ["p(95)<1000"],
'http_req_duration{name: Purge}': ["p(95)<1000"],
}
}
export default function () {
group('Get', () => {
const response = http.get('http://httpbin.org/get', { tags: { name: 'Get' } });
check(response, {
"Get response status code is 200": (r) => r.status === 200
});
});
group('Delete', () => {
const response = http.del('http://httpbin.org/status/204', { tags: { name: 'Delete' } });
check(response, {
"Purge response status code is 204": (r) => r.status === 204
});
});
group('Purge', () => {
const response = http.del('http://httpbin.org/status/204', { tags: { name: 'Purge' } });
check(response, {
"Purge response status code is 204": (r) => r.status === 204
});
});
}Leading to the following output, where indeed the metrics linked to the requests using the |
|
I've found out that only the Directing the k6 output to a JSON file, shows that some actual data points are actually recorded, even when using the Which seems to hint that the issue might be at the |
|
Hi, @oleiade. Your example indeed shows that HTTP requests that do not return a body (hence the 204 - No Content HTTP status code) end up zeroed in the summary output. It could be that the problem resides in My only hint, however, is 1) the absence of a body in the response or 2) the HTTP method (DELETE) or 3) the HTTP status code (204) somehow triggering this behaviour somewhere in k6... |
|
I'm not convinced that this bug happens because of the absence of a body. With a quick and dirty Python-based HTTP server returning 200 on DELETE, with a JSON body, I get the same result. server script: #!/usr/bin/env python3
from http.server import BaseHTTPRequestHandler, HTTPServer
import json
class SimpleHTTPRequestHandler(BaseHTTPRequestHandler):
def do_DELETE(self):
# Respond with 204 No Content status
# self.send_response(204)
self.send_response(200)
# Send headers
self.send_header("Content-type", "application/json")
self.end_headers()
# Send a basic JSON response
response = json.dumps({"message": "Delete request processed"})
self.wfile.write(response.encode())
def run(server_class=HTTPServer, handler_class=SimpleHTTPRequestHandler, port=8000):
server_address = ("", port)
httpd = server_class(server_address, handler_class)
print(f"Starting httpd server on port {port}")
httpd.serve_forever()
if __name__ == "__main__":
run()Replaced script line: And same result: |
|
I think we have three alternative clues:
• It could be the absence of a response body
• It could be the status code (204)
• It could be the request method (DELETE)
It seems that with the tests you made, it should not be the first two. Maybe the DELETE method itself is the culprit?
…On Jan 15, 2024 at 15:45 +0000, Théo Crevon ***@***.***>, wrote:
I'm not convinced that this bug happens because of the absence of a body. With a quick and dirty Python-based HTTP server returning 200 on DELETE, with a JSON body, I get the same result.
server script:
#!/usr/bin/env python3
from http.server import BaseHTTPRequestHandler, HTTPServer
import json
class SimpleHTTPRequestHandler(BaseHTTPRequestHandler):
def do_DELETE(self):
# Respond with 204 No Content status
# self.send_response(204)
self.send_response(200)
# Send headers
self.send_header("Content-type", "application/json")
self.end_headers()
# Send a basic JSON response
response = json.dumps({"message": "Delete request processed"})
self.wfile.write(response.encode())
def run(server_class=HTTPServer, handler_class=SimpleHTTPRequestHandler, port=8000):
server_address = ("", port)
httpd = server_class(server_address, handler_class)
print(f"Starting httpd server on port {port}")
httpd.serve_forever()
if __name__ == "__main__":
run()
Replaced script line:
const response = http.del('http://localhost:8000', { tags: { name: 'Delete' } });
And same result:
✓ { name: Delete }.............: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
|
Hey @luissalgadofreire, So, it turns out the issue is probably much simpler than I had anticipated, and would have had a link to the body, but, the request's. I always forget about it, but the |
|
Hi again, @oleiade. Yup, that seems to solve it: /\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/ .io
execution: local
script: main.js
output: -
scenarios: (100.00%) 1 scenario, 100 max VUs, 1m30s max duration (incl. graceful stop):
* rps_50: 10.00 iterations/s for 1m0s (maxVUs: 50-100, gracefulStop: 30s)
█ Create
✓ Create response status code is 201
█ Get
✓ Get response status code is 200
█ List
✓ List response status code is 200
█ Delete
✓ Delete response status code is 204
█ Purge
✓ Purge response status code is 204
█ teardown
checks.........................: 100.00% ✓ 3000 ✗ 0
data_received..................: 5.5 MB 86 kB/s
data_sent......................: 1.1 MB 17 kB/s
group_duration.................: avg=507.3ms min=3.05ms med=47.76ms max=1.81s p(90)=1.01s p(95)=1.02s
http_req_blocked...............: avg=258.6µs min=1µs med=3µs max=37.83ms p(90)=5µs p(95)=6µs
http_req_connecting............: avg=4.53µs min=0s med=0s max=1.92ms p(90)=0s p(95)=0s
✓ http_req_duration..............: avg=15.36ms min=2.87ms med=6.48ms max=875.98ms p(90)=21.5ms p(95)=25.49ms
{ expected_response:true }...: avg=15.36ms min=2.87ms med=6.48ms max=875.98ms p(90)=21.5ms p(95)=25.49ms
✓ { name:Create }..............: avg=12.17ms min=2.87ms med=4.41ms max=875.98ms p(90)=6.01ms p(95)=7.61ms
✓ { name:Delete }..............: avg=21.53ms min=12.58ms med=17.78ms max=152.44ms p(90)=25.35ms p(95)=33.84ms
✓ { name:Get }.................: avg=26.97ms min=10.65ms med=18.27ms max=695.72ms p(90)=26.4ms p(95)=37.14ms
✓ { name:List }................: avg=9.94ms min=3.22ms med=4.97ms max=516.25ms p(90)=7.85ms p(95)=9.26ms
✓ { name:Purge }...............: avg=6.17ms min=3.73ms med=5.45ms max=52.48ms p(90)=8.09ms p(95)=10.34ms
✓ http_req_failed................: 0.00% ✓ 0 ✗ 3001
✓ { name:Create }..............: 0.00% ✓ 0 ✗ 600
✓ { name:Delete }..............: 0.00% ✓ 0 ✗ 600
✓ { name:Get }.................: 0.00% ✓ 0 ✗ 600
✓ { name:List }................: 0.00% ✓ 0 ✗ 600
✓ { name:Purge }...............: 0.00% ✓ 0 ✗ 600
http_req_receiving.............: avg=216.3µs min=15µs med=65µs max=9.61ms p(90)=411µs p(95)=723µs
http_req_sending...............: avg=23.38µs min=9µs med=21µs max=169µs p(90)=34µs p(95)=39µs
http_req_tls_handshaking.......: avg=249.36µs min=0s med=0s max=37.29ms p(90)=0s p(95)=0s
http_req_waiting...............: avg=15.12ms min=2.79ms med=6.25ms max=866.64ms p(90)=21.23ms p(95)=25.28ms
http_reqs......................: 3001 46.53064/s
iteration_duration.............: avg=2.53s min=31.38ms med=2.28s max=5.27s p(90)=4.05s p(95)=4.06s
iterations.....................: 600 9.303027/s
vus............................: 1 min=1 max=32
vus_max........................: 50 min=50 max=50
running (1m04.5s), 000/050 VUs, 600 complete and 0 interrupted iterations
rps_50 ✓ [======================================] 000/050 VUs 1m0s 10.00 iters/sIt is a little tricky, though;). Thanks, nevertheless, it does work perfectly now, with that correction. |
The team agrees, and we are in the process of providing a new HTTP module which will surely improve on this #2461. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Brief summary
While testing a CRUD API, thresholds with tags are used to get the metrics per tag. However, some calls, namely Delete and Purge, which are the only ones that don't return a body in the response (and therefore their
response.json()is null) return 0 instead of the actual values. Using custom groups worked. Using thresholds with tags didn't.k6 version
v0.48.0
OS
macOS Sonoma (14.1.2)
Docker version and image (if applicable)
No response
Steps to reproduce the problem
Use the following configuration to test the performance of a CRUD API.
Expected behaviour
The configuration should return the below, with
DeleteandPurgecalls returning the proper values instead of0.Actual behaviour
The configuration returns the below, with
DeleteandPurgecalls being basically ignored and returning 0 instead. The one thing where they differ from the other calls is that they return no body, soresponse.json()is null and trying to call it will throw an error. I suspectthresholdswithtagsmay be callingresponse.json()somewhere in the source code and ends up not returning the proper metric.The text was updated successfully, but these errors were encountered: