-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathATS-Cork-WG-Summary.en.html
316 lines (309 loc) · 21.5 KB
/
ATS-Cork-WG-Summary.en.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Traffic Server Cork Working Group — AMC Traffic Server Documentation</title>
<link rel="stylesheet" href="_static/sphinxdoc.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT: './',
VERSION: '8.0.x',
COLLAPSE_INDEX: false,
FILE_SUFFIX: '.html',
HAS_SOURCE: true,
SOURCELINK_SUFFIX: '.txt'
};
</script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Diagrams" href="reference.en.html" />
<link rel="prev" title="Partial Caching Initial Design" href="poc/archive-partial-object-caching-prototype.en.html" />
</head>
<body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="reference.en.html" title="Diagrams"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="poc/archive-partial-object-caching-prototype.en.html" title="Partial Caching Initial Design"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">SWOC Docs</a> »</li>
</ul>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="index.html">
<img class="logo" src="_static/balcora-gate-400x400.jpg" alt="Logo"/>
</a></p>
<h3><a href="index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Traffic Server Cork Working Group</a><ul>
<li><a class="reference internal" href="#configuration">Configuration</a></li>
<li><a class="reference internal" href="#building">Building</a></li>
<li><a class="reference internal" href="#process-management">Process Management</a></li>
<li><a class="reference internal" href="#layer-7-routing">Layer 7 Routing</a></li>
<li><a class="reference internal" href="#hash-tables">Hash Tables</a></li>
<li><a class="reference internal" href="#plugins">Plugins</a></li>
<li><a class="reference internal" href="#proxy-protocol">Proxy Protocol</a></li>
<li><a class="reference internal" href="#memory-allocation">Memory Allocation</a></li>
<li><a class="reference internal" href="#testing">Testing</a></li>
<li><a class="reference internal" href="#fine-grained-debugging">Fine grained debugging</a></li>
<li><a class="reference internal" href="#other-notes">Other Notes</a></li>
<li><a class="reference internal" href="#amc-tech">AMC Tech</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="poc/archive-partial-object-caching-prototype.en.html"
title="previous chapter">Partial Caching Initial Design</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="reference.en.html"
title="next chapter">Diagrams</a></p>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="_sources/ATS-Cork-WG-Summary.en.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<form class="search" action="search.html" method="get">
<div><input type="text" name="q" /></div>
<div><input type="submit" value="Go" /></div>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="traffic-server-cork-working-group">
<h1>Traffic Server Cork Working Group<a class="headerlink" href="#traffic-server-cork-working-group" title="Permalink to this headline">¶</a></h1>
<p>My view on the working group Cork, heavily biased toward things of importance to me.</p>
<div class="section" id="configuration">
<h2>Configuration<a class="headerlink" href="#configuration" title="Permalink to this headline">¶</a></h2>
<p>Lua configuration has been abandoned. It will be replaced by YAML. The TsLuaConfig project will
be abandoned and the schema work converted to YAML. It was agreed that YAML based schemas will be
supported if not required to provide more accurate configuration feedback. In the longer term I
would like to build editor support logic for editing Traffic Server configurations such as for
Visual Code that can provide direct schema based feedback and error checking. It was also agreed
that we would strive for a zero configuration capability, to enable local configurations to
contain only locallly relevant configuration options.</p>
<p>One advantage of YAML is it lends itself to REST style API control more easily. It was the
concensus we should move forward on extending and broadening external API control of the
configuration using YAML.</p>
</div>
<div class="section" id="building">
<h2>Building<a class="headerlink" href="#building" title="Permalink to this headline">¶</a></h2>
<p>Upgrade to Clang 6 static analyzer. This created around 50 new errors which we fixed during teh
working group. There appear to still be a few issues that are problems with the analyzer we are
working around but this was determined to be overall an improvement.</p>
<p>The compiler requirement will be moved to C++17 starting with 8.0. The language changes are
minor, the primary advantages will be to simplify the build environment to a single version of
C++ and to gain access to many useful standard library mechanisms, such as <code class="docutils literal"><span class="pre">std::filesystem</span></code>
and <code class="docutils literal"><span class="pre">std::string_view</span></code>.</p>
</div>
<div class="section" id="process-management">
<h2>Process Management<a class="headerlink" href="#process-management" title="Permalink to this headline">¶</a></h2>
<p><code class="docutils literal"><span class="pre">traffic_cop</span></code> has been removed for 8.0. This removes internal health checking from Traffic
Server. The concensus view is that this is done better using other mechanisms that are commonly
available now but weren’t back in the early days of Traffic Server. Some additional documentation
will be required for groups wanting to use Traffic Server but not aware of specific tools needed.</p>
<p>It is planned to remove <code class="docutils literal"><span class="pre">traffic_manager</span></code> for 9.0. The latter will be significantly more
difficult because <code class="docutils literal"><span class="pre">traffic_manager</span></code> performs useful functions. Among these are</p>
<ul class="simple">
<li>Opening of the proxy ports. This can be shifted without much loss to <code class="docutils literal"><span class="pre">traffic_server</span></code> which
is already capable of doing this. This will mean lack of open ports while <code class="docutils literal"><span class="pre">traffic_server</span></code> is
restarting which was considered a minor issue.</li>
<li>External communcation with command lines tools. The ongoing RPC work will be needed to move
this functionality in to <code class="docutils literal"><span class="pre">traffic_server</span></code>.</li>
<li>Access to statistics and metrics. This data should already be present in <code class="docutils literal"><span class="pre">traffic_server</span></code>
and if the RPC work is completed this should be a relatively straight forward project.</li>
</ul>
<p>Internal RPC
The current <code class="docutils literal"><span class="pre">libRPC</span></code> project will be abandoned. Instead a third party RPC serialization
mechanism will be selected and used. This replacement will be required to achieve the design
goals of <code class="docutils literal"><span class="pre">libRPC</span></code> –</p>
<ul class="simple">
<li>Separately library without executable specific compilation or libraries.</li>
<li>Bidirectional.</li>
<li>Type safety.</li>
<li>External registration to enable subsystems to register their own RPC calls and handling.</li>
</ul>
<p>This is not as simple as it sounds. One key point of the current RPC is it requires no
allocations. Because it is a simple serialized list of fundamental types the reciever can provide
an array or argument list of those types to transfer data directly from the serialized data. It
is likley a new RPC will be YAML or (roughly equivalently) JSON based. How the memory for storing
this is to be handled will be something to be considered.</p>
</div>
<div class="section" id="layer-7-routing">
<h2>Layer 7 Routing<a class="headerlink" href="#layer-7-routing" title="Permalink to this headline">¶</a></h2>
<p>The general direction of the work as accepted. The main concern is providing intermediate
deliverables rather than trying for one big result. The current work on the extended host
database was well accepted. In fact there was some interest in extending this to the <code class="docutils literal"><span class="pre">HttpSM</span></code>
for other plugins. This could be an interesting idea even with regard to Layer 7 routing. There
is a problem in the design of the resolvers regarding maintaning state during resolution for a
transaction. The problem is the amount of state data depends on the resolver statck. If this data
could be added in the same way to the <code class="docutils literal"><span class="pre">HttpSM</span></code> then that would be a perfect spot to store that
state.</p>
</div>
<div class="section" id="hash-tables">
<h2>Hash Tables<a class="headerlink" href="#hash-tables" title="Permalink to this headline">¶</a></h2>
<p>For concurrent containers we will try TBB (Thread Building Blocks) first. The primary constraint is
the availability of packages from well known repositories. TBB is a bit heavy weight wih many things
not of direct use but if installable via normal package management this is acceptable.</p>
<p>If TBB doesn’t work out then we will look at either <code class="docutils literal"><span class="pre">libcds</span></code> (Concurrent Data Structures). A third
option is to do a fork of CK (Concurrency Kit) to make CK++, a C++ fronted version of CK.</p>
<p>For basic hash tables to replace the TCL hash table we will look at using the STL containers with a
custom allocator. It’s still a bit unclear to me what the goal of this is. As best I can tell the
objection is that common use of the STL based containers will have memory use characteristics. A
custom allocator should be able to do better in terms of locality and cleanup. Given the strictures
on allocators I am not confident this can be done in a reasonable way.</p>
</div>
<div class="section" id="plugins">
<h2>Plugins<a class="headerlink" href="#plugins" title="Permalink to this headline">¶</a></h2>
<p>Some plugins were promoted which will affect building but not release packages. The <code class="docutils literal"><span class="pre">gzip</span></code>
plugin was renamed to <code class="docutils literal"><span class="pre">compress</span></code>. This will affect packaging and configurations.</p>
<p>There was agreement that we should push forward on providing access to C++ based support present
in <code class="docutils literal"><span class="pre">libtsutil</span></code>. The primary issue will be setting up the header files to avoid pulling in
internal header files which are specialized for the core. Some work was done on this at the
Working Group but more may be required.</p>
<p>The pre-fetch plugin written for Tumblr was discussed. The plugin itself may not be accepted in
to open source but the basic functionality was supported and it was agreed to provide it in one
way or another. It might be by extending the <code class="docutils literal"><span class="pre">background_fetch</span></code> plugin or some other mechanism.
There were some suggestions about pre-fetching, including using 0 or 1 byte range requests to
trigger <code class="docutils literal"><span class="pre">background_fetch</span></code>. I’m stylistically opposed to this because it’s dependent on
installing a plugin to use in a non-obvious. it might be reasonable to expand <code class="docutils literal"><span class="pre">prefetch</span></code> to
handle background fetch operations or vice versa. We will need to look at the configuration to
see which is best, but the shift in configuration to YAML might be a good opportunity shift thist
around.</p>
</div>
<div class="section" id="proxy-protocol">
<h2>Proxy Protocol<a class="headerlink" href="#proxy-protocol" title="Permalink to this headline">¶</a></h2>
<p>It was agreed to commit the work on <a class="reference external" href="https://www.haproxy.com/blog/haproxy/proxy-protocol/">PROXY protocol</a> to 8.0 with one more change. This change
is to enable PROXY protocol on a per proxy port basis using the proxy port descriptors. The keyword
for this will be <code class="docutils literal"><span class="pre">pp</span></code>, therefore to enable PROXY protocol with TLS on port 443 the descriptor
would be <code class="docutils literal"><span class="pre">443:ssl:pp</span></code>.</p>
<p>The current version of PROXY protocol is inbound only. In the longer term it will be useful to
provide for it outbound as well. I want to look at doing a betterjob of handling protocols on the
proxy ports in order to prepare for non-HTTP proxying.</p>
</div>
<div class="section" id="memory-allocation">
<h2>Memory Allocation<a class="headerlink" href="#memory-allocation" title="Permalink to this headline">¶</a></h2>
<p>Comcast indicated some concerns about leaks due to disabling global allocators, which was a puzzle
both to them and to the rest of us. Nevertheless, it was agreed to proceed cautiously for this
reason. Fei’s work will be to provide flags for removing just the global allocators and for removing
global and proxy allocators. In addition he will need to test this and perform measurements to
validate the changes are not harmful.</p>
</div>
<div class="section" id="testing">
<h2>Testing<a class="headerlink" href="#testing" title="Permalink to this headline">¶</a></h2>
<p>The AuTest workshop went well – everyone at the Working Group has now written an AuTest. Jason
received some useful feedback on improving the infrastructure. I discussed off line potential
additions to the AuTest tooling with regard to utilities to drive traffic through Traffic Server.
Apple is working on a tool which could be used to provide HTTP specific traffic rather than using
the more blunt tool of <code class="docutils literal"><span class="pre">curl</span></code> and <code class="docutils literal"><span class="pre">grep</span></code>. In particular this would have the ability to focus on
specific headers rather than a generic text comparison, something I have wanted for a long time.</p>
<p>Apple also discussed their “CDN Checker” which is very similar to what I have been calling “live
testing”. In essence requests are done against an in production box to verify its status and
configuration. This is the basis for the HTTP traffic tool discussed just above. In the Apple setup
this is done in an automated fashion but both I and the Traffic Control group would like to be able
to do it on demand as well.</p>
</div>
<div class="section" id="fine-grained-debugging">
<h2>Fine grained debugging<a class="headerlink" href="#fine-grained-debugging" title="Permalink to this headline">¶</a></h2>
<p>This started from a request by Dave Carlin to be able to enable debug messages for a specific user
agent IP address. There was quite a bit of controversy about this, oddly about the configuration not
the implementation. The actual mechanism was put in 7.0 (ported from our internal patch) but 7.0 had
no way to actually use it (that was local to OATS). The concensus at the Working Group was to put
that in to 8.0 but there was some desire to have more features, such as subnets instead of a single
IP address and plugin control of the feature. There was also discussion of integration with another
debugging feature where a specific but limited set of debugging messages can be enabled on a per
transaction basis. This feature is a bit of a hack, we will need to look at how to do better. This
is a bit tricky because the per user agent IP address mechanism depends on tagging continuations
with a debug enabled flag, and by the time the <code class="code docutils literal"><span class="pre">HttpSM</span></code> is created many of those continuations
may already exist and not inherit the enable flag.</p>
</div>
<div class="section" id="other-notes">
<h2>Other Notes<a class="headerlink" href="#other-notes" title="Permalink to this headline">¶</a></h2>
<p>Susan’s design for outbound HTTP/2 was accepted. She was able to spend a good bit of time with
Kees, Masaori, and Masakazu to make sure everyone working in this area is on board with her
design.</p>
<p>For half closed connections, it was agreed these are not likely to be of significant use in real
production because this is not an issue for TLS or HTTP/2 connections. Susan agreed to try to
gather some actual data on how often this happens to verify.</p>
<p>WeAmp is preparing to release its <code class="docutils literal"><span class="pre">fastCGI</span></code> plugin. This enables remaps to redirect to a
fastCGI handler. This would among other enable serving static pages from Traffic Server directly
which has been a request from the operations teams and some properties for quite a while. This
could be integrated with the <code class="docutils literal"><span class="pre">escalate</span></code> plugin or the internal error page templates. Health
checks could use this feature. It does require an additional process to handle the fastCGI
requests which need to be managed.</p>
<p>Based on some work by Masoari for doing better connection draining in Traffic Server, Susan and I
are investigating doing clean cutovers from one <code class="docutils literal"><span class="pre">traffic_server</span></code> to another. This would involve</p>
<ul class="simple">
<li>Start the new instance, wait for it to become ready.</li>
<li>Mark the old instance to drain.</li>
<li>Disable cache writing in the old instance.</li>
<li>Enable cache writing in the new instance.</li>
<li>Start accepting connections on the new instance.</li>
<li>When the old instance has few enough connections, shut it down.</li>
</ul>
<p>The key point is to avoid having two processes write to the cache at the same time, while also
continuing to provide some level of protection to the upstreams.</p>
</div>
<div class="section" id="amc-tech">
<h2>AMC Tech<a class="headerlink" href="#amc-tech" title="Permalink to this headline">¶</a></h2>
<p>Resistance to my work on <code class="code docutils literal"><span class="pre">BufferWriter</span></code> finally collapsed and I will be able to merge in my
updates, which are already incorporated in to to the upstream connection limiting.</p>
<p><code class="code docutils literal"><span class="pre">MemArena</span></code> was accepted with little comment, I will be doing a few more fixes to that before
using it in other project. The same for <code class="code docutils literal"><span class="pre">MemSpan</span></code> and <code class="code docutils literal"><span class="pre">MemVec</span></code>. I do need to track the
emerging C++20 standard on this point to minimize disruption, or provide facilities that are needed
but won’t be in the future STL version.</p>
<p>The move to C++17 will enable me to remove some sub-optimal code, particularly in the Cache Tool.</p>
<p>There was a push to remove <code class="code docutils literal"><span class="pre">TSHashTable</span></code> in the process of doing the hash table upgrades. I
need to do some writing on that, I’m not sure it’s a good idea in retrospect after pondering it for
a bit. That class does provide some features that are not going to be available in the STL
containers. I need to be specific about that so a reasonable decision can be made.</p>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="reference.en.html" title="Diagrams"
>next</a> |</li>
<li class="right" >
<a href="poc/archive-partial-object-caching-prototype.en.html" title="Partial Caching Initial Design"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">SWOC Docs</a> »</li>
</ul>
</div>
<div class="footer" role="contentinfo">
© Copyright 2017, [email protected].
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.6.6.
</div>
</body>
</html>