Using the EdcHttpClient without fallbacks in HttpDataSource

[AndrYurk]

In the HttpDataSource class, we execute requests using EdcHttpClient without any fallbacks. This means that FailsafeCall will return a FailsafeException after unsuccessful attempts for any kind of response error. However, errors like 404, 401, and 403 (and possibly others) do not need to be retried and should be returned after the first attempt.

From a code perspective, it was designed to receive and handle such errors, but currently, this functionality is not reachable.

Connector/extensions/data-plane/data-plane-http/src/main/java/org/eclipse/edc/connector/dataplane/http/pipeline/HttpDataSource.java

Lines 62 to 91 in 0802581

	var response = httpClient.execute(request);
	if (response.isSuccessful()) {
	var body = response.body();
	if (body == null) {
	throw new EdcException(format("Received empty response body transferring HTTP data for request %s: %s", requestId, response.code()));
	}
	var stream = body.byteStream();
	responseBodyStream.set(new ResponseBodyStream(body, stream));
	var mediaType = Optional.ofNullable(body.contentType()).map(MediaType::toString).orElse(OCTET_STREAM);
	return success(Stream.of(new HttpPart(name, stream, mediaType)));
	} else {
	try {
	if (NOT_AUTHORIZED == response.code() || FORBIDDEN == response.code()) {
	return StreamResult.notAuthorized();
	} else if (NOT_FOUND == response.code()) {
	return StreamResult.notFound();
	} else {
	return error(format("Received code transferring HTTP data: %s - %s.", response.code(), response.message()));
	}
	} finally {
	try {
	response.close();
	} catch (Exception e) {
	monitor.severe("Error closing failed response", e);
	}
	}
	}
	} catch (IOException e) {
	throw new EdcException(e);
	}

The proposal is to use fallbacks, but the question is: which error codes should not be retried? Should it include all error codes from 400 to 500, or only specific ones?

[ndr-brt]

It really depends, I can see cases when at the first call the service returns 404 because the resource is not available "yet", and so at the second time it works.

My take on this would be to add a configuration setting in the data-plane-http extension, to exclude certain response codes from the retry policy, so eventually providers can decide to enable this feature or not.