-
Notifications
You must be signed in to change notification settings - Fork 5.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introducing ClientRequestFilter.java: A New Plugin for Merging Additional Headers into Client Requests in the Authentication Filter #23380
base: master
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This PR needs some test cases.
Nit rephrasing of release note entries following the Order of changes in the Release Notes Guidelines.
|
I have added a test case to verify setting values in the request header and committed it. |
assertEquals("CustomValue", wrappedRequest.getHeader("X-Custom-Header")); | ||
} | ||
|
||
abstract static class HttpServletRequestAdapter |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's already a MockHttpServletRequest
, can you use that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tdcmeehan
I have refactored the code to use MockHttpServletRequest. Could you check it?
Thanks.
While we don't use Mockito, the stubs in the test are a little out of control. We need to reduce the size of this simple test. I have a couple of suggestions that you can research to make the testing simpler.
|
@tdcmeehan |
@@ -825,4 +830,25 @@ private static int driftServerPort(DriftServer server) | |||
{ | |||
return ((DriftNettyServerTransport) server.getServerTransport()).getPort(); | |||
} | |||
|
|||
public RequestModifierManager getRequestModifierManager() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please, just add a method to return the RequestModifierManager
, and let users inject whatever custom modifier they wish.
|
||
public class RequestModifierManager | ||
{ | ||
private final List<RequestModifier> requestModifiers; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needs to be thread safe. I recommend using a CopyOnWriteArrayList
.
// authentication succeeded | ||
nextFilter.doFilter(withPrincipal(request, principal), response); | ||
CustomHttpServletRequestWrapper wrappedRequest = withPrincipal(request, principal); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of modifying the authentication filter, this should just go in its own filter that is applied after the authentication filter.
this.customHeaders = new HashMap<>(); | ||
} | ||
|
||
public void addHeader(String name, String value) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unused
presto-main/src/main/java/com/facebook/presto/server/security/AuthenticationFilter.java
Show resolved
Hide resolved
import java.util.Map; | ||
import java.util.Optional; | ||
|
||
public interface RequestModifier |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's call this something more descriptive, such as ClientRequestFilter
.
@tdcmeehan |
|
||
public class ClientRequestFilterManager | ||
{ | ||
private final CopyOnWriteArrayList<ClientRequestFilter> clientRequestFilters; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
private final CopyOnWriteArrayList<ClientRequestFilter> clientRequestFilters; | |
private final List<ClientRequestFilter> clientRequestFilters; |
private final CopyOnWriteArrayList<ClientRequestFilter> clientRequestFilters; | ||
public ClientRequestFilterManager() | ||
{ | ||
this.clientRequestFilters = new CopyOnWriteArrayList<>(); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
private final CopyOnWriteArrayList<ClientRequestFilter> clientRequestFilters; | |
public ClientRequestFilterManager() | |
{ | |
this.clientRequestFilters = new CopyOnWriteArrayList<>(); | |
} | |
private final List<ClientRequestFilter> clientRequestFilters = new CopyOnWriteArrayList<>(); |
|
||
public List<ClientRequestFilter> getClientRequestFilters() | ||
{ | ||
return Collections.unmodifiableList(clientRequestFilters); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return Collections.unmodifiableList(clientRequestFilters); | |
return ImmutableList.copyOf(clientRequestFilters); |
extraHeaderValueMap.ifPresent(map -> { | ||
for (Map.Entry<String, String> extraHeaderEntry : map.entrySet()) { | ||
if (request.getHeader(extraHeaderEntry.getKey()) == null) { | ||
extraHeadersMap.putIfAbsent(extraHeaderEntry.getKey(), extraHeaderEntry.getValue()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we should silently not put the header if there's an existing header. I think we should consider the following:
- The SPI contract mandates that any potential header is known upfront.
- The runtime will validate that the headers that are requested to be modified match with what is actually returned.
- There is a blocklist of headers that the plugin should not modify (such as query ID).
- The runtime will validate that all registered filters only return a disjoint set of headers being added. If there is a conflict, this is a fatal error that will require an administrator to fix (but un-registering one of these plugins).
- If there is a conflict here, then an exception is thrown.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tdcmeehan
I have gone through your comments and added a few validations in the current implementation. I can verify that with you. However, I have to know about the headers in the blocklist that should not be modified.
Could you help?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's start with the query ID, trace ID and transaction ID in PrestoHeaders.java.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have incorporated the changes mentioned in your comment. Could you please review them and share your suggestions?
private final Map<String, String> customHeaders; | ||
|
||
public CustomHttpServletRequestWrapper(HttpServletRequest request) | ||
{ | ||
super(request); | ||
this.customHeaders = new ConcurrentHashMap<>(); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
private final Map<String, String> customHeaders; | |
public CustomHttpServletRequestWrapper(HttpServletRequest request) | |
{ | |
super(request); | |
this.customHeaders = new ConcurrentHashMap<>(); | |
} | |
private final Map<String, String> customHeaders = new ConcurrentHashMap<>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some feedback is still unadressed.
for (Map.Entry<String, String> extraHeaderEntry : map.entrySet()) { | ||
String headerKey = extraHeaderEntry.getKey(); | ||
if (headersBlockList.contains(headerKey)) { | ||
throw new RuntimeException("Modification attempt detected: The header " + headerKey + " is present in the blocked headers list."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please throw a PrestoException and introduce a new error code for this. It should be a system error.
while (originalHeaderNames.hasMoreElements()) { | ||
headerNames.add(originalHeaderNames.nextElement()); | ||
} | ||
return Collections.enumeration(headerNames); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please use Immutable collections.
@tdcmeehan Could you help me identify which feedback is still pending? I believe I have addressed feedback items 2, 3, 4, and 5 from the list above. |
Please scroll up in this PR and examine feedback on the code that is still remaining. |
Yeah. @tdcmeehan I have incorporated all feedback items now. Kindly review them and share your suggestions? Thanks. |
ClientRequestFilter customModifier = new ClientRequestFilter() | ||
{ | ||
@Override | ||
public List<String> getHeaderNames() | ||
{ | ||
return Collections.singletonList("X-Custom-Header"); | ||
} | ||
@Override | ||
public <T> Optional<Map<String, String>> getExtraHeaders(T additionalInfo) | ||
{ | ||
Map<String, String> headers = new HashMap<>(); | ||
headers.put("X-Custom-Header", "CustomValue_1"); | ||
return Optional.of(headers); | ||
} | ||
}; | ||
|
||
ClientRequestFilter customModifierConflict = new ClientRequestFilter() | ||
{ | ||
@Override | ||
public List<String> getHeaderNames() | ||
{ | ||
return Collections.singletonList("X-Custom-Header"); | ||
} | ||
@Override | ||
public <T> Optional<Map<String, String>> getExtraHeaders(T additionalInfo) | ||
{ | ||
Map<String, String> headers = new HashMap<>(); | ||
headers.put("X-Custom-Header", "CustomValue_2"); | ||
return Optional.of(headers); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please reduce duplicate code. Consider an inner helper class.
} | ||
|
||
static class ConcreteHttpServletRequest | ||
extends MockHttpServletRequest |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not modify MockHttpServletRequest
directly?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In ConcreteHttpServletRequest, I am storing additional headers in the customHeaders map and overriding the getHeaders method to manage them. This functionality isn't natively provided by MockHttpServletRequest, so extending it makes sense.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just add a new constructor to MockHttpServletRequest
that overrides the headers.
@tdcmeehan |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please refactor tests to use the code under test.
for (Map.Entry<String, String> extraHeaderEntry : map.entrySet()) { | ||
String headerKey = extraHeaderEntry.getKey(); | ||
if (headersBlockList.contains(headerKey)) { | ||
throw new PrestoException(HEADER_MODIFICATION_ATTEMPT, "Modification attempt detected: The header " + headerKey + " is present in the blocked headers list."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of mentioning a "blocked headers list", just list out the headers that are not allowed to be modified.
throw new PrestoException(HEADER_MODIFICATION_ATTEMPT, "Modification attempt detected: The header " + headerKey + " is present in the blocked headers list."); | ||
} | ||
if (globallyAddedHeaders.contains(headerKey)) { | ||
throw new RuntimeException("Header conflict detected: " + headerKey + " already added by another filter."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not throw RuntimeException, throw PrestoException.
for (Map.Entry<String, String> extraHeaderEntry : map.entrySet()) { | ||
String headerKey = extraHeaderEntry.getKey(); | ||
if (headersBlockList.contains(headerKey)) { | ||
throw new RuntimeException("Modification attempt detected: The header " + headerKey + " is present in the blocked headers list."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are we duplicating code here? I am very confused about how this test works.
@tdcmeehan |
Please squash commits and ensure the commit format follows our guidelines in contributing.md. |
d797300
to
9fa7ae2
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Almost there. Please also add documentation for this plugin to https://github.com/prestodb/presto/tree/master/presto-docs/src/main/sphinx/develop
ClientRequestFilterContext context = new ClientRequestFilterContext(principal); | ||
Optional<Map<String, String>> extraHeaderValueMap = requestFilter.getExtraHeaders(context); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I realize now that the principal is per request. Context objects typically are used to create the plugin. So please revert this, and just add the principal as a plain old parameter to getExtraHeaders
, no need for fancy parameterization.
Map<String, String> extraHeadersMap = extraHeadersMapBuilder.build(); | ||
return new ModifiedHttpServletRequest(request, extraHeadersMapBuilder.build()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Map<String, String> extraHeadersMap = extraHeadersMapBuilder.build(); | |
return new ModifiedHttpServletRequest(request, extraHeadersMapBuilder.build()); | |
return new ModifiedHttpServletRequest(request, extraHeadersMapBuilder.build()); |
Remove duplicate build.
return new ClientRequestFilter() { | ||
@Override | ||
public List<String> getHeaderNames() | ||
{ | ||
return ImmutableList.of(headerName); | ||
} | ||
|
||
@Override | ||
public Optional<Map<String, String>> getExtraHeaders(ClientRequestFilterContext context) | ||
{ | ||
return Optional.of(ImmutableMap.of(headerName, headerValue)); | ||
} | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Make this an inner class.
9d848d1
to
b49b2c1
Compare
Presto facilitates seamless integration of additional headers into client requests using the | ||
Client Request Filter plugin via the AuthenticationFilter class, enabling efficient and dynamic header management. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This sounds AI generated. Please keep the documentation factual and concise. Example, instead of"Efficient and dynamic header management", say, "allowing operators of Presto to customize the headers used for query processing by the Presto runtime." "Presto facilitates the seamless integration of additional headers" is another example. "Seamless" has no meaning in this context.
Just say something like "Presto allows operators to customize the headers used by the Presto runtime to process queries. Some example use cases include customized authentication workflows, or enriching query attributes such as the query source." Please use original wording, think about it, and avoid tools to write this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure.
The ``ClientRequestFilterFactory`` is responsible for creating instances of ``ClientRequestFilter``. It also defines | ||
the name of the filter, which is subsequently used as the header name in client requests. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this true? The name of the filter is the header name in the client request?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Corrected.
|
||
After installing a plugin that implements ``ClientRequestFilterFactory`` on the coordinator, the ``AuthenticationFilter`` class passes the ``principal`` object to the request filter, which returns the header values as a map. | ||
|
||
Presto uses the name of the request filter to determine whether the header is present in the blocklist. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What blocklist? Mention the headers which are not allowed to be overidden. Link to the Java source of headers that may be overidden.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added.
5a959b0
to
18e06fe
Compare
|
||
public interface ClientRequestFilter | ||
{ | ||
List<String> getHeaderNames(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
List<String> getHeaderNames(); | |
Set<String> getExtraHeaderKeys(); |
{ | ||
List<String> getHeaderNames(); | ||
|
||
Optional<Map<String, String>> getExtraHeaders(Principal principal); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can it just return empty map and we don't return optional?
``ClientRequestFilter`` interface provides two methods: ``getExtraHeaders()``, | ||
which returns header values as key-value pairs in a map, and ``getHeaderNames()``, which returns a list of header names used as the header names in client requests. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I imagine getExtraHeaders
is used for the runtime to quickly check if it needs to apply a more expensive call to enrich the headers. If so, add this context to the documentation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added.
bc440bc
to
90678a2
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just one more thing, otherwise looks good.
@ThreadSafe | ||
public class ClientRequestFilterManager | ||
{ | ||
private List<ClientRequestFilterFactory> factories = new CopyOnWriteArrayList<>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's make this a map.
- The key of the map should be the name of the factory.
- We should not allow duplicate names.
} | ||
|
||
filters = factories.stream() | ||
.map(factory -> factory.create(factory.getName())) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This construct is strange. See above comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What I meant was, please don't pass in factory.getName()
into the create
method, there is no need because it can just internally call getName()
.
b80a4d5
to
28c5aa4
Compare
@tdcmeehan I have made the changes. Could you please take a look? |
} | ||
|
||
filters = factories.stream() | ||
.map(factory -> factory.create(factory.getName())) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What I meant was, please don't pass in factory.getName()
into the create
method, there is no need because it can just internally call getName()
.
a00c586
to
b21ad93
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the doc! A few suggestions.
Also, to have this new doc page be visible in the Developer Guide index page, you must edit presto-docs/src/main/sphinx/develop.rst
and add
develop/client-request-filter
.
02dba34
to
c2bd746
Compare
@steveburnett I have incorporated the changes requested. Thanks. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! (docs)
Pull updated branch, new local doc build, looks good. Thanks!
@tdcmeehan Could you take a look and approve? |
Please fix rebase conflicts. |
f7c2d9f
c2bd746
to
f7c2d9f
Compare
@tdcmeehan |
f7c2d9f
to
9ace465
Compare
If it looks good, can we merge this PR? @tdcmeehan |
@SthuthiGhosh9400 the CI is read. Please investigate. If you find it's not related, rebase to rerun the test. |
d5df908
to
7d72a35
Compare
Add ClientRequestFilter.java interface in Presto-spi module. Improve Request Headers in the Authentication Filter Class.
7d72a35
to
634d36c
Compare
@tdcmeehan I reran the test, and it looks good now. |
Description
Motivation and Context
#23997
Impact
This feature enables the merging of headers of any type, offering a versatile solution for header management.
Test Plan
Contributor checklist
Release Notes
Please follow release notes guidelines and fill in the release notes below.