-
Notifications
You must be signed in to change notification settings - Fork 0
/
CHANGES
24897 lines (15687 loc) · 869 KB
/
CHANGES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
3.3.0-dev.422 | 2020-10-13 16:26:24 -0700
* GH-1208: Use Dictionary validity assertions only during CI (Jon Siwek, Corelight)
3.3.0-dev.420 | 2020-10-13 15:34:02 -0700
* Change "Cluster::Node$p" field to use "&default=0/unknown" (Seth Hall, Corelight)
This effectively makes defining the listening port optional in the cluster
node layout config, with "0/unknown" meaning the node is not pre-configured
to listen for incoming connections from other cluster nodes.
3.3.0-dev.414 | 2020-10-13 13:49:05 -0700
* Update cmake submodule to pull in fix for building libkqueue (Tim Wojtulewicz, Corelight)
* GH-1063: Update libkqueue to fix pf_ring-zc failures (Tim Wojtulewicz, Corelight)
3.3.0-dev.411 | 2020-10-13 13:38:38 -0700
* Remove unused LoginConn type and variable in Conn.h (Tim Wojtulewicz, Corelight)
3.3.0-dev.409 | 2020-10-13 13:32:18 -0700
* Make it possible to pass command line options through to scripts. (Seth Hall, Corelight)
A new ``zeek_script_args`` variable contains a list of arguments passed
to a script. E.g. either when explicitly executing Zeek like
``zeek -- myscript.zeek -arg1 -arg2``, or when using Zeek to interpret
executable scripts that contain a hashbang line at the top like::
#!/usr/local/zeek/bin/zeek --
3.3.0-dev.403 | 2020-10-13 10:50:12 -0700
* Add new Pcap::findalldevs() BIF (Seth Hall, Corelight)
* Remove superfluous RuleCondition destructors (Jon Siwek, Corelight)
* Silence Clang's warning about ignoring GCC's maybe-uninitialized warning (Jon Siwek, Corelight)
* Add reference to network_time_init from zeek_init docs (Jon Siwek, Corelight)
3.3.0-dev.390 | 2020-10-12 17:43:15 -0700
* Improve documentation for zeek_init event scheduling pitfalls (Jon Siwek, Corelight)
3.3.0-dev.388 | 2020-10-12 17:02:20 -0700
* Add CaptureLoss::Too_Little_Traffic notice (Vlad Grigorescu)
* Add CaptureLoss::initial_watch_interval for a quick read on cluster health after startup. (Vlad Grigorescu)
* Improve capture-loss.zeek documentation. (Vlad Grigorescu)
* Fix whitespace in capture-loss.zek (Vlad Grigorescu)
3.3.0-dev.381 | 2020-10-12 11:15:29 -0700
* GH-779: Add "udp-state" signature condition (Jon Siwek, Corelight)
It accepts "originator" or "responder" states as a way to enforce that
the signature only matches packets in the associated direction.
The "established" state is rejected as an error since it doesn't
have a useful meaning like it does for the "tcp-state" condition.
* Rename RuleConditionTCPState::TCPState enum values (Jon Siwek, Corelight)
* Rename "tcp-state" signature parser tokens to not be TCP-specific (Jon Siwek, Corelight)
3.3.0-dev.377 | 2020-10-08 12:07:42 -0700
* GH-1200: ignore a maybe-uninitialized warning (Jon Siwek, Corelight)
3.3.0-dev.374 | 2020-10-07 10:44:11 -0700
* Fix multipart MIME leak of sub-part found after closing-boundary (Jon Siwek, Corelight)
After detecting a closing-boundary for a given multipart MIME entity, it
enters into an "end of data" state, however any subsequent boundary
delimiter could still cause the allocation of a sub-entity object that
is never released due to cleanup logic being bypassed upon finding the
"end of data" state already reached.
This change prevents allocation/processing of sub-entities after the
"end of data" state is reached (e.g. from detecting a multipart
closing-boundary). This new behavior still aligns with RFC 2046
expectations:
"There appears to be room for additional information prior to the first
boundary delimiter line and following the final boundary delimiter line.
These areas should generally be left blank, and implementations must
ignore anything that appears before the first boundary delimiter line or
after the last one."
Credit to OSS-Fuzz for discovery
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=26027
(Link to details becomes public 30 days after patch release)
* Btests for log filter policy hooks (Christian Kreibich, Corelight)
* Btest baseline updates to reflect new logging policy hooks (Christian Kreibich, Corelight)
* Migrate existing use of filter predicates to policy hooks (Christian Kreibich, Corelight)
* Support for log filter policy hooks (Christian Kreibich, Corelight)
This adds a "policy" hook into the logging framework's streams and
filters to replace the existing log filter predicates. The hook
signature is as follows:
hook(rec: any, id: Log::ID, filter: Log::Filter);
The logging manager invokes hooks on each log record. Hooks can veto
log records via a break, and modify them if necessary. Log filters
inherit the stream-level hook, but can override or remove the hook as
needed.
The distribution's existing log streams now come with pre-defined
hooks that users can add handlers to. Their name is standardized as
"log_policy" by convention, with additional suffixes when a module
provides multiple streams. The following adds a handler to the Conn
module's default log policy hook:
hook Conn::log_policy(rec: Conn::Info, id: Log::ID, filter: Log::Filter)
{
if ( some_veto_reason(rec) )
break;
}
By default, this handler will get invoked for any log filter
associated with the Conn::LOG stream.
The existing predicates are deprecated for removal in 4.1 but continue
to work.
3.3.0-dev.365 | 2020-10-06 12:19:49 -0700
* GH-425: Avoid temporary value while coercing records already of the right type. (Robin Sommer, Corelight)
The combination of this commit with the previous one now lets the examples
in GH-425 all execute with the same performance.
* GH-425: Optimize record constructor expression. (Robin Sommer, Corelight)
We remove the inheritance from UnaryExpr because we know the type of the
operand precisely and can skip a temporary when evaluating the expression.
* Unify type comparisions for records. (Robin Sommer, Corelight)
For records, same_type(r1, r2) would not check if the fields'
attributes match as well. That seems like an oversight, and some
callers of same_type() did indeed add that check on their end. This
commit moves the check into same_type() itself. That generally doesn't
seem make any differences except for a couple of places validating
code, which we update a bit. That in turn leans to slightly different
(better?) error messages for a couple of test cases.
3.3.0-dev.361 | 2020-10-06 10:13:37 -0700
* logging/ascii: Support leftover log rotation in non-supervisor setups (Arne Welzel, Corelight)
Allow enabling leftover log rotation through
LogAscii::enable_leftover_log_rotation and redef this for the
logger node in a supervisor setup individually.
3.3.0-dev.359 | 2020-10-02 10:49:35 -0700
* GH-1180: Add external dependency include paths to `zeek-config --include_dir` (Jon Siwek, Corelight)
This helps ensure successful compilation of a plugin that includes a
Zeek header which includes an external dependency header even if the
plugin itself doesn't necessarily depend on that external library.
3.3.0-dev.357 | 2020-10-02 10:26:11 -0700
* Switch one's complement checksum implementation (Jon Siwek, Corelight)
Borrows the `in_cksum` code from tcpdump, which borrowed from FreeBSD.
It handles unaligned data better and also unrolls the inner loop to
process 16 two-byte values at a time versus 2 one-byte values at a time
in the previous version. Generally measured as ~1.5x faster in a
release build. The new API should generally be more amenable to any
future optimization explorations since all relevant data blocks are
available within a single call rather than spread across multiple.
3.3.0-dev.355 | 2020-09-29 17:08:44 -0700
* Improve broker/ssl_auth_failure.zeek test (Jon Siwek, Corelight)
There was a race that previously may cause it to not truly test
whether the connecting-process exited due to SSL authentication
failure or just because the listening-process wasn't available yet
(e.g. due to process scheduling variability).
3.3.0-dev.352 | 2020-09-25 15:09:29 -0700
* GH-160: change find-filtered-trace to ignore pcaps with any non-TCP traffic (Robin Sommer, Corelight)
3.3.0-dev.350 | 2020-09-25 14:37:58 -0700
* Add dce_rpc_request_stub and dce_rpc_response_stub events (Yacin Nadji, Corelight)
* Fix namespace warning in fuzzer code (Tim Wojtulewicz, Corelight)
3.3.0-dev.346 | 2020-09-24 16:03:28 -0700
* Fix a Sphinx warning about misformatted packet analyzer comment (Jon Siwek, Corelight)
* Add Zeekygen documentation support for packet analyzers (Jon Siwek, Corelight)
* Move packet_mgr to the zeek namespace (Tim Wojtulewicz, Corelight)
3.3.0-dev.341 | 2020-09-24 08:16:45 -0700
* Fix negative-value-left-shift undefined behavior in patricia trie (Jon Siwek, Corelight)
* Improve negation of ConstExpr (Jon Siwek, Corelight)
* Instead of creating a NegExpr for negation of a literal/constant,
a ConstExpr is now created directly.
* For negation of integer literals, there's now an additional check
for whether the integer would be outside the range of possible 'int'
values. This can also help prevent the undefined behavior due to
overflow as a result of trying to represent the minimum 'int' value of
-9223372036854775808 as a literal in a script -- the unsigned value is
cast to signed yielding INT64_MIN, then INT64_MIN is negated.
* Avoid signed integer overflow when combining SMB header PID bits (Jon Siwek, Corelight)
Such an overflow invokes undefined behavior.
* Avoid unary negation of INT64_MIN in modp_litoa10 (Jon Siwek, Corelight)
Overlow can occur in that case, which is undefined behavior.
* Avoid double-to-int conversion overflows in modp_dtoa functions (Jon Siwek, Corelight)
Those methods already had a fallback to use sprintf() for large values
except:
* The check-for-large-value was unnecessarily done after many
operations that aren't relevant to the check and those operations can
result in a conversion overflow (undefined behavior).
* The check-for-large-value was using the literal value for a
32-bit INT_MAX instead of just using INT_MAX. For a platform where
`int` is less than 32-bits, the same conversion overflow from the
previous point could still occur (undefined behavior).
* The check-for-large-value was not inclusive of INT_MAX.
In a case where the conversion of INT_MAX itself to a double
can't be represented exactly, it's implementation-defined whether
the closest higher or closest lower representable-value is selected.
If the higher value is selected, then a `double` value comparing equal
to INT_MAX-as-converted-to-double would cause an overflow of an `int`
upon conversion (undefined behavior).
* Fix divide-by-zero in Entropy analyzer (Jon Siwek, Corelight)
* Fix divide-by-zero in stats/profiling memory usage calculation (Jon Siwek, Corelight)
* Fix uninitialized field in POP3 fuzzer (Jon Siwek, Corelight)
* Add framework for running UndefinedBehaviorSanitizer in CI (Jon Siwek, Corelight)
3.3.0-dev.329 | 2020-09-23 11:32:06 -0700
* Update NEWS (Tim Wojtulewicz, Corelight)
* Fix build on FreeBSD, which was missing full definition of sockaddr for ARP (Tim Wojtulewicz, Corelight)
* Update hashes for external testing repos (Tim Wojtulewicz, Corelight)
* Fix handling of session analysis.
This reverts commit 0e51be18ac59d7a176f18780f1f8df85561449b9. (Jan Grashoefer)
* Fix Sessions::DoNextPacket to ignore ARP packets and not report weirds for them (Tim Wojtulewicz, Corelight)
* Packet analysis cleanup. (Jan Grashoefer)
* Minor cleanup (Tim Wojtulewicz, Corelight)
* Simplify packet analyzer config. (Jan Grashoefer)
* Make default packet analyzer definition explicit. (Jan Grashoefer)
* Add explicit root analyzer for packet analysis. (Jan Grashoefer)
* Extend packet analysis test. (Jan Grashoefer)
* Simplify MPLS analysis. (Jan Grashoefer)
* Improve packet analysis data flow. (Jan Grashoefer)
* Improve packet analyzer API. (Jan Grashoefer)
* Rename DefaultAnalyzer to IP. (Jan Grashoefer)
* Move ARP analysis into packet analyzer. (Jan Grashoefer)
* Small cleanup of packet analysis. (Jan Grashoefer)
* Fix some build issues from the rebase (Tim Wojtulewicz, Corelight)
* Remove encap_hdr_size (replaced by skip analyzer). (Jan Grashoefer)
* Add skip analyzer test and update baselines. (Jan Grashoefer)
* Allow to overwrite packet analysis mappings. (Jan Grashoefer)
* Add SkipAnalyzer.
This is WIP: The test case would require a new pcap or the possibility
to overwrite analyzer mappings. The CustomEncapsulationSkip method and
the corresponding options need to be removed. (Jan Grashoefer)
* Suggested code improvements for packet analysis. (Jan Grashoefer)
* Improve naming in packet analysis. (Jan Grashoefer)
* Migrate all packet analyzers to new API. (Jan Grashoefer)
* Remove magic identifiers from Ethernet analyzer. (Jan Grashoefer)
* Bring back default packet analysis.
Default analyzers can be configured per packet analyzer by omitting the
identifier in the ConfigEntry. (Jan Grashoefer)
* Further simplified the packet analysis API.
This is still WIP and includes the following changes:
* Dispatchers are now part of analyzers (moving dispatching logic from
the manager to the analyzers)
* All available analyzers are instantiated on start up
* Removal of configuration class (Jan Grashoefer)
* Move dispatching into packet analyzers.
WIP that updates only the Ethernet analyzer. (Jan Grashoefer)
* Move cur_pos from packet into packet manager loop. (Jan Grashoefer)
* Remove packet_analysis/Defines.h
- Replace uses of identifier_t with uint32_t
- Replace repeated usage of tuple type for Analysis results with type alias (Tim Wojtulewicz, Corelight)
* Move VectorDispatcher to be the only dispatcher (Tim Wojtulewicz, Corelight)
* Remove Manager::Reset() method (Tim Wojtulewicz, Corelight)
* Remove enabled state from Components, ability to enable/disable from Manager (Tim Wojtulewicz, Corelight)
* Use shared_ptr instead of raw pointers in packet_analysis for analyzers and dispatchers (Tim Wojtulewicz, Corelight)
* Whitespace fixes from review (Tim Wojtulewicz, Corelight)
* Reorganize some pointer handling (Tim Wojtulewicz, Corelight)
* Fix ordering of debug logger strings to match the enum (Tim Wojtulewicz, Corelight)
* Fixes for various btest issues
- Fix handling of truncated ethernet headers, fix core.truncation test output
- Update commit hashes for external private test repo (Tim Wojtulewicz, Corelight)
* Merge ProtocolAnalyzerSet into Manager, remove AnalyzerSet base class (Tim Wojtulewicz, Corelight)
* Minor fixes for packet analyzer renaming. (Jan Grashoefer)
* Renamed LL-Analyzers to Packet Analyzers. (Jan Grashoefer)
* Initial implementation of Lower-Level analyzers (Peter Oettig)
3.3.0-dev.285 | 2020-09-22 16:01:51 -0700
* Improve fa_file/fa_metadata documentation (Duffy O'Craven, Corelight)
3.3.0-dev.284 | 2020-09-22 14:10:36 -0700
* Avoid race condition in control/id_value btest (Jon Siwek, Corelight)
3.3.0-dev.282 | 2020-09-22 11:08:56 -0700
* GH-927: Fix circumvention of evaluation order in 'when' conditions (Jon Siwek, Corelight)
3.3.0-dev.279 | 2020-09-21 15:22:45 -0700
* CI: Move to Fedora 32 (Christian Kreibich, Corelight)
Fedora 30 is EOL.
3.3.0-dev.275 | 2020-09-17 13:39:46 -0700
* GH-1164: Fix incorrect RSTOS0 conn_state determinations
The RSTOS0 `conn_state` label is documented as "Originator sent a SYN
followed by a RST, never saw SYN-ACK from responder", but was previously
applied to cases where no originator SYN exists, like a single RST-only
packet. (Jon Siwek, Corelight)
3.3.0-dev.273 | 2020-09-17 13:11:44 -0700
* GH-1119: add base/protcols/conn/removal-hooks.zeek
This adds two new functions: `Conn::register_removal_hook()` and
`Conn::unregister_removal_hook()` for registering a hook function to be
called back during `connection_state_remove`. The benefit of using hook
callback approach is better scalability: the overhead of unrelated
protocols having to dispatch no-op `connection_state_remove` handlers is
avoided. (Jon Siwek, Corelight)
* Avoid run-time cost of find-filtered-trace.zeek for live traffic (Jon Siwek, Corelight)
* Remove connection_successful and successful_connection_remove events
Related to https://github.com/zeek/zeek/issues/1119 (Jon Siwek, Corelight)
3.3.0-dev.269 | 2020-09-17 11:42:38 -0700
* GH-1155: Recursively check table index for unsupported types
Previously, container types used within a table/set index were not
deeply checked to ensure all constituents could be part of an index. (Jon Siwek, Corelight)
* GH-1159: Fix vector-of-interval multiplication/division arithmetic
Those operations done between a vector-of-interval and a
vector-of-arithmetic-type previously threw a runtime expression error
due to an incorrect coercion being used internally. (Jon Siwek, Corelight)
3.3.0-dev.265 | 2020-09-17 11:24:42 -0700
* Avoid passing address of member in packed struct #1074
This appeases -Waddress-of-packed-member warnings in some
compiler/platform combinations via use of local variables. (Christian Kreibich, Corelight)
3.3.0-dev.262 | 2020-09-14 12:27:52 -0700
* Improve notice/suppression-cluster btest to avoid race potential (Jon Siwek, Corelight)
3.3.0-dev.260 | 2020-09-10 11:53:03 -0700
* Fix generate-docs Action to stage autogenerated script-reference files (Jon Siwek, Corelight)
3.3.0-dev.258 | 2020-09-09 10:46:37 -0700
* Improve Reporter weird-sampling-whitelist getters/setters (Jon Siwek, Corelight)
- getter methods return const-ref
- setter methods pass by value and std::move()
- ranged-for loops over the whitelists access by const-ref
* Fix memory leak caused by re-entering GTPv1 parsing (Jon Siwek, Corelight)
If the inner packet of a decapsulated GTPv1 frame maps to the same
Connection as the outer packet and also contains another GTPv1
encapsulation, the same GTPv1 analyzer attempts to parse the inner
packet. In the process, the inner parsing method overwrites pointers to
memory resources the outer parsing method has not yet released and
results in a memory leak.
* Fix memory leak caused by re-entering AYIYA parsing (Jon Siwek, Corelight)
If the inner packet of a decapsulated AYIYA frame maps to the same
Connection as the outer packet and also contains another AYIYA
encapsulation, the same AYIYA analyzer attempts to parse the inner
packet. In the process, the inner parsing method overwrites pointers to
memory resources the outer parsing method has not yet released and
results in a memory leak.
Credit to OSS-Fuzz for discovery
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=25256
3.3.0-dev.254 | 2020-09-08 17:21:16 -0700
* GH-343: Change HTTP DPD signatures to trigger analyzer independent of peer state. (Robin Sommer, Corelight)
This is to avoid missing large sessions where a single side exceeds
the DPD buffer size. It comes with the trade-off that now the analyzer
can be triggered by anybody controlling one of the endpoints (instead
of both).
3.3.0-dev.252 | 2020-09-08 17:04:19 -0700
* GH-623: Extend weird sampling with option to track selected weirds globally. (Robin Sommer, Corelight)
The new set "sampling_global_list" lists weirds to rate-limit
globally instead of per connection/flow.
3.3.0-dev.249 | 2020-09-04 18:30:19 -0700
* Exclude installing "zeek -> ." include dir symlink (Jon Siwek, Corelight)
The symlink only exists for use within the source-tree and isn't needed
for the install-tree.
3.3.0-dev.247 | 2020-09-04 17:24:28 -0700
* GH-165: Fix global initializations that indirectly use builtin types (Jon Siwek, Corelight)
* Improve how primary/top-level BIFs get initialized (Jon Siwek, Corelight)
* GH-1122: Allow initializing globals with calls to subdir BIFs (Jon Siwek, Corelight)
* GH-1122: Improve error for global record initialization exceptions (Jon Siwek, Corelight)
3.3.0-dev.241 | 2020-09-03 15:02:29 -0700
* Change a script comment to use "code-block" directive (Jon Siwek, Corelight)
3.3.0-dev.240 | 2020-09-02 20:51:13 -0700
* Improve CI benchmark script's error handling/messaging (Jon Siwek, Corelight)
3.3.0-dev.238 | 2020-09-02 11:39:25 -0700
* GH-955: Prohibit `fallthrough` in typecasting `switch` cases (Jon Siwek, Corelight)
3.3.0-dev.236 | 2020-09-01 22:33:05 -0700
* Fix/finalize the generate-docs Github Action (Jon Siwek, Corelight)
3.3.0-dev.230 | 2020-09-01 20:32:01 -0700
* Fix generate-docs action to not fail when there's no changes (Jon Siwek, Corelight)
3.3.0-dev.229 | 2020-09-01 19:49:36 -0700
* Update generate-docs action to use access token for submodule update (Jon Siwek, Corelight)
3.3.0-dev.228 | 2020-09-01 18:37:33 -0700
* Update generate-docs action to install Sphinx dependency (Jon Siwek, Corelight)
3.3.0-dev.227 | 2020-09-01 17:58:22 -0700
* Add missing configure step for doc-generation action (Jon Siwek, Corelight)
3.3.0-dev.226 | 2020-09-01 17:53:55 -0700
* Improve documentation for rfind_str() (Jon Siwek, Corelight)
* GH-1149: Add GitHub Action to automate generation of zeek-docs (Jon Siwek, Corelight)
3.3.0-dev.224 | 2020-09-01 11:57:44 -0700
* Reformat strings.bif Zeekygen comments to fix Sphinx warnings (Jon Siwek, Corelight)
3.3.0-dev.221 | 2020-08-31 17:31:21 -0700
* GH-174: Treat ambiguous attribute duplication as an error (Tim Wojtulewicz, Corelight)
For example, a &default=1 and a &default=2 attribute are not valid when
used together, but two duplicate &log attributes together are acceptable
although redundant.
3.3.0-dev.216 | 2020-08-31 14:57:57 -0700
* Simplify a broker btest (Jon Siwek, Corelight)
3.3.0-dev.215 | 2020-08-28 16:51:40 -0700
* GH-926: Improve type-checking for ternary conditional operator (Jon Siwek, Corelight)
* Unbreak build on power architectures by fixing small cmake issue (Johanna Amann, Corelight)
3.3.0-dev.211 | 2020-08-27 18:00:55 -0700
* Potentially improve intel/remove-item-cluster btest (Jon Siwek, Corelight)
3.3.0-dev.210 | 2020-08-27 15:34:14 -0700
* GH-158: Ensure packet-filter framework is loaded if -f option is passed (Tim Wojtulewicz, Corelight)
3.3.0-dev.208 | 2020-08-27 11:12:22 -0700
* GH-1145: Fix a umask issue for processing-status file (Vlad Grigorescu)
3.3.0-dev.206 | 2020-08-26 11:20:58 -0700
* Rename bro-bif.h to zeek-bif.h (Tim Wojtulewicz, Corelight)
* Fix uses of bro.pac to use zeek.pac (Tim Wojtulewicz, Corelight)
* Rename the bro-related binpac files to zeek (Tim Wojtulewicz, Corelight)
3.3.0-dev.202 | 2020-08-26 10:59:59 -0700
* GH-1145: Have mkdir in ensure_dir respect umask. (Vlad Grigorescu)
This also aligns with the mkdir BIF.
3.3.0-dev.200 | 2020-08-25 19:47:02 -0700
* Update plugin btests for namespace changes (Tim Wojtulewicz, Corelight)
* Remove namespace qualifications where unnecessary. (Tim Wojtulewicz, Corelight)
3.3.0-dev.196 | 2020-08-25 17:04:12 -0700
* Fix closing timestamp of rotated log files in supervised-cluster mode (Jon Siwek, Corelight)
3.3.0-dev.195 | 2020-08-25 14:32:47 -0700
* Improve an Intel framework btest (Jon Siwek, Corelight)
Relying on non-atomic `cp` operation may not cooperate well with the
Input framework file re-reading logic.
3.3.0-dev.194 | 2020-08-24 19:20:29 -0700
* Mark Continuation.h and PacketDumper.h as deprecated (Tim Wojtulewicz, Corelight)
* Remove vector iterator type aliases from SmithWaterman code, fix uses of them (Tim Wojtulewicz, Corelight)
* Fix warning with usage of fmt() (Tim Wojtulewicz, Corelight)
3.3.0-dev.190 | 2020-08-24 14:43:06 -0700
* Make set_processing_status() signal-safe.
Closes #574. (Robin Sommer, Corelight)
3.3.0-dev.188 | 2020-08-24 14:30:43 -0700
* GH-710: Check that &expire_func is a function and not hook/event (Jon Siwek, Corelight)
3.3.0-dev.186 | 2020-08-24 14:28:25 -0700
* Fix a case where PktSrc gets processed twice in one runloop iteration
For a non-live PktSrc, it had a special-case to be considered "ready"
every iteration, but additionally every 1 in 100 iterations (the polling
frequency), if there were no other "ready" IOSources, it would get added
to the "ready" set a 2nd time.
This commit completely excludes PktSrc from being processed during the
1/100 runloop iteration where a Poll() happens. That exclusion is
desirable for a second reason: if reading a pcap happens to do its final
Process() during that 1/100 polling-iteration and there's other
IOSources ready to process like EventMgr/TimerMgr, those sources have
logic to advance network-time to current-time if a PktSrc is no longer
open. So in such a case, PktSrc::Process() closes, then
EventMgr::Process() sees there's no longer an active PktSrc and advances
to current-time, then EventMgr::Drain() happens and may dispatch
various events that were previous scheduled, with those events now
unexpectedly seeing a network_time() returning current-time. (Jon Siwek, Corelight)
3.3.0-dev.184 | 2020-08-24 14:27:31 -0700
* GH-594: Improve table initialization type-check error messages (Jon Siwek, Corelight)
3.3.0-dev.182 | 2020-08-21 13:46:16 -0700
* Fix `zeek -NN` not printing canonical names for file analyzers (Tim Wojtulewicz, Corelight)
3.3.0-dev.180 | 2020-08-21 13:28:23 -0700
* Remove definition-less ThreadVal<->BrokerData conversion functions (Jon Siwek, Corelight)
Their presence/deprecation may cause "undefined reference" build
failures for some compilers (GCC 9.3.0 in my case).
3.3.0-dev.177 | 2020-08-21 08:36:48 -0700
* Add missing namespace to usage of get_exe_path in fuzzer (Tim Wojtulewicz, Corelight)
* Rename methods in RunState to remove 'net' from their names (Tim Wojtulewicz, Corelight)
* Move zeek::net namespace to zeek::run_state namespace.
This also moves all of the code from Net.{h,cc} to RunState.{h,cc} and marks Net.h as deprecated (Tim Wojtulewicz, Corelight)
* Move ScannedFile class and associated globals into ScannedFile.h and out of Net.h and scan.l (Tim Wojtulewicz, Corelight)
* Rename types in ZeekList.h to be consistent with the style guide (Tim Wojtulewicz, Corelight)
* Move NetVar from zeek to zeek::detail namespace (Tim Wojtulewicz, Corelight)
* Remove PRI_PTR_COMPAT macros (Tim Wojtulewicz, Corelight)
* Fix indentation of namespaced aliases (Tim Wojtulewicz, Corelight)
* Move zeek-setup code into namespaces (Tim Wojtulewicz, Corelight)
* Move ZeekList types to zeek namespace (Tim Wojtulewicz, Corelight)
* Move __RegisterBif from zeek::detail::plugin to zeek::plugin::detail (Tim Wojtulewicz, Corelight)
* Remove unimplemented zeek_magic_path/bro_magic_path method (Tim Wojtulewicz, Corelight)
* Move all plugin classes into zeek::plugin::detail namespaces (Tim Wojtulewicz, Corelight)
* Rename BroList.h to ZeekList.h (Tim Wojtulewicz, Corelight)
* Move a few smaller files to zeek namespaces (Tim Wojtulewicz, Corelight)
* Tag the end of some namespaces for consistency (Tim Wojtulewicz, Corelight)
* Move a few of the zeek::util methods and variables to zeek::util::detail (Tim Wojtulewicz, Corelight)
* Move zeekygen code to zeek::zeekygen::detail namespace (Tim Wojtulewicz, Corelight)
* Mark zeek::util::pad_size as constexpr, which provides a small performance improvement (Tim Wojtulewicz, Corelight)
* Move everything in util.h to zeek::util namespace.
This commit includes renaming a number of methods prefixed with bro_ to be prefixed with zeek_. (Tim Wojtulewicz, Corelight)
* Deprecate ptr_compat_uint and ptr_compat_int in util.h (Tim Wojtulewicz, Corelight)
* Move NetVar.h to zeek namespace (Tim Wojtulewicz, Corelight)
* Move the functions and variables in Net.h to the zeek::net namespace. This includes moving network_time out of util.h. (Tim Wojtulewicz, Corelight)
* Add namespaced version of trigger_mgr, fix a couple uses of event_registry (Tim Wojtulewicz, Corelight)
* Move the remainder of the analyzers to zeek namespaces (Tim Wojtulewicz, Corelight)
* Move arp, tcp, udp, pia, and stepping stone analyzers (Tim Wojtulewicz, Corelight)
* Move bro_broker code to zeek::Broker namespace (Tim Wojtulewicz, Corelight)
* Move notifier code to zeek::notifier::detail namespace (Tim Wojtulewicz, Corelight)
* Move threading classes to zeek namespaces (Tim Wojtulewicz, Corelight)
* Move probabilistic code into zeek namespaces (Tim Wojtulewicz, Corelight)
* Move iosource code to zeek namespaces (Tim Wojtulewicz, Corelight)
* Move logging code to zeek namespaces (Tim Wojtulewicz, Corelight)
* Move input code to zeek namespaces (Tim Wojtulewicz, Corelight)
* Move file_analysis code to zeek namespaces (Tim Wojtulewicz, Corelight)
3.3.0-dev.142 | 2020-08-20 16:26:06 -0700
* Improve stability of config/cluster_resend btest (Jon Siwek, Corelight)
3.3.0-dev.141 | 2020-08-20 14:56:15 -0700
* Mark FileAssembler destructor as default, remove implementation (Tim Wojtulewicz, Corelight)
* Remove a few protected or private no-op constructors (Tim Wojtulewicz, Corelight)
* Properly initialize Timer::type. Fixes Coverity 1431144. (Tim Wojtulewicz, Corelight)
* Properly initialize members of Reassembler. Fixes Coverity 1413348. (Tim Wojtulewicz, Corelight)
* Fix a number of Coverity issues in Dict (Tim Wojtulewicz, Corelight)
1431186: Asserting that an unsigned value is >= 0 is pointless
1431188/1431189/1431191: Side effect of using an assignment operator in a call to ASSERT()
1431193: Dereference after null check could lead to null being used
1431195: Use of rand() replaced with random()
3.3.0-dev.135 | 2020-08-20 10:23:29 -0700
* remove variables in netvar, use tabs in DNS.h and polish comments in dns events. (FlyingWithJerome)
* add new line at the end of the test baseline (FlyingWithJerome)
* add data+=option_len to skip unknown bytes (FlyingWithJerome)
* better explanation to server cookie (FlyingWithJerome)
* add edns-cookie testcase (FlyingWithJerome)
* remove data+=option_len error in cookie and keepalive (FlyingWithJerome)
* add EDNS cookie parsing (FlyingWithJerome)
* add testcases (FlyingWithJerome)
* add units (100ms) to comments (FlyingWithJerome)
* initial adding EDNS TCP keepalive (FlyingWithJerome)
3.3.0-dev.119 | 2020-08-19 14:35:51 -0700
* Misc strings.bif adjustments
* Declare rfind_str() with correct return type
* Fix compiler warnings for signed/unsigned comparisons (Jon Siwek, Corelight)
3.3.0-dev.117 | 2020-08-19 14:12:06 -0700
* Use constexpr for IPAddr::v4_mapped_prefix declaration
The inconsistent use of constexpr in the definition but not declaration
causes an unrecoverable error for Coverity as seen in issue 1431151 (Jon Siwek, Corelight)
3.3.0-dev.115 | 2020-08-19 11:49:24 -0700
* Update docs for configuring CI email notifications (Jon Siwek, Corelight)
Secrets are now stored organization-wide rather than per-repository.
3.3.0-dev.113 | 2020-08-18 12:10:26 -0700
* GH-467: Catch use of &add_func/&delete_func on IDs without values. (Robin Sommer, Corelight)
* Fix compiler warning from missing namepsace qualification (Tim Wojtulewicz, Corelight)
3.3.0-dev.109 | 2020-08-14 10:00:07 -0700
* GH-178: Add new string bif methods based on python string utilities (Tim Wojtulewicz, Corelight)
3.3.0-dev.107 | 2020-08-12 20:01:37 +0000
* Accept bracketed ipv6 addresses (Justin Azoff, Corelight)
* Fix minimize_info in ftp/main not returning a value.
Fixes GH-1120 (Johanna Amann, Corelight)
* Fix config reader regular expression for MUSL (Seth Hall, Corelight)
* Moved verb ACTION_DROP to base/frameworks/notice/main.zeek.
ACTION_DROP is not only part of catch-n-release subsystem and can be useful in other cases (Aashish Sharma)
* Added new acld verbs of filter and nofilter (Aashish Sharma)
* Fix some printf warnings with size_t values (Tim Wojtulewicz, Corelight)
3.3.0-dev.97 | 2020-08-11 11:34:17 -0700
* Move Dict constants to detail namespace (Tim Wojtulewicz, Corelight)
* Add a few missing deprecation fixes (Tim Wojtulewicz, Corelight)
* Adjust Dict whitespace/style (Jon Siwek, Corelight)
* Adjust more btest timings (Jon Siwek, Corelight)
* Improve termination reliability/speed for brokerstore btests (Jon Siwek, Corelight)
* General btest cleanup
- Use `-b` most everywhere, it will save time.
- Start some intel tests upon the input file being fully read instead of
at an arbitrary time.
- Improve termination condition for some sumstats/cluster tests.
- Filter uninteresting output from some supervisor tests.
- Test for `notice_policy.log` is no longer needed. (Jon Siwek, Corelight)
* Update NEWS about change in Dictionary implementation (Jon Siwek, Corelight)
* Improve Intel expire-item btest to be less time-sensitive (Jon Siwek, Corelight)
* Improve btests with unstable table/set output ordering
Particularly, the final output order of a table/set is sensitive to
order of input/insertions and some tests were converting
std::unordered_{set,map} to Zeek table/set and iteration over those
standard containers may not always loop through elements in the same
order across all platforms. (Jon Siwek, Corelight)
* Update doc submodule (Jon Siwek, Corelight)
* Adjust a few btests that were unstable due to time-sensitivity (Jon Siwek, Corelight)
* Fix DNS script deleting a table element while iterating
Doesn't typically cause any problems since the loop breaks after
deleting, except there's now an assert in debug builds catching
potential problems like this. (Jon Siwek, Corelight)
* Improve a brokerstore btest to filter out Broker connection messages (Jon Siwek, Corelight)
* Sort output of a few SumStats cluster tests
The order of $epoch_result() function calls among all keys within a
single epoch isn't consequential. (Jon Siwek, Corelight)
* Fix extract_first_email_addr() to really return the first email
The use of find_all() in extract_email_addrs_vec() extracted occurrences
to an intermediate set and thus lost any sense of ordering.
This changes extract_email_addrs_vec() to use find_all_ordered() and
return all occurrences of email addresses found in the argument,
included duplicates, with their order of occurrence preserved. (Jon Siwek, Corelight)
* Add find_all_ordered() BIF
Operates similar to find_all(), except returns a vector instead of
set to allow preservation of order/duplicates. (Jon Siwek, Corelight)
* Extend external test suite canonifier with set-sorting logic
Two new canonifiers: one to sort the contents of conn.log "service"
field and another to sort the contents of any field of type "set". (Jon Siwek, Corelight)
* Update btests/baselines for OpenDict compat
Haven't checked different build configurations yet, but all except
a few SumStats tests are stable for me now. The external tests
are also completely failing, but haven't looked at those yet. (Jon Siwek, Corelight)
* Fix new/malloc/delete/free mismatches in Dictionary code (Jon Siwek, Corelight)
* Add explanation for a Dict TODO item (Jon Siwek, Corelight)
* Fix compiler warning in Dictionary debug/dump printf format string (Jon Siwek, Corelight)
* Overwrite old Dict.h/cc with new ones (Tim Wojtulewicz, Corelight)
* Fix issue with sumstats script and fix baselines that were crashing previously (Tim Wojtulewicz, Corelight)
* Extend the timeouts on a few intel tests, update baselines (Tim Wojtulewicz, Corelight)
* Updating test baselines for new dictionary code due to changes in ordering of fields in the dictionary (Tim Wojtulewicz, Corelight)
* Add namespaces for OpenDict files (Tim Wojtulewicz, Corelight)
* Add unit tests from Dict into OpenDict files (Tim Wojtulewicz, Corelight)
* Build fixups
- Fix some clang-tidy warnings
- Add std:: namespaces for ofstream and ios
- Remove some unused methods (Tim Wojtulewicz, Corelight)
* Massive formatting cleanup (jasonlue)
* Remove key reference after insertion because the key may become invalid afterwards. (jasonlue)
* add Clustered Hashing based Open Addressing Dict. To replace the existing dict, #define USE_OPEN_DICT (jasonlue)
3.3.0-dev.55 | 2020-08-10 09:57:36 -0700
* GH-1100: Fix reported body-length of HTTP messages w/ sub-entities (Jon Siwek, Corelight)
The body-lengths of sub-entities, like multipart messages, got counted
twice by mistake: once upon the end of the sub-entity and then again
upon the end of the top-level entity that contains all sub-entities.
The size of just the top-level entity is the correct one to use.
3.3.0-dev.53 | 2020-08-05 12:35:50 -0700
* Fix deprecation warnings in fuzz targets (Jon Siwek, Corelight)
* Remove dbg_read_internal_state() (Jon Siwek, Corelight)
It's never had a definition anyway and the namespacing/deprecation
process was now causing a compile error in debug/fuzzing builds w/ GCC
3.3.0-dev.50 | 2020-08-04 20:22:48 +0000
* Fix incorrect conflict detection of namespaced-enum-names
E.g. defining a `Foo::RED` enum name when a `GLOBAL::RED` identifier
already exists would previously be treated as an error, even though the
names don't truly conflict. (Jon Siwek, Corelight)
* GH-1080: Treat enum name re-use across different enum types as an error (Jon Siwek, Corelight)
* GH-1080: Rename conflicting NetControl::DROP enum definitions
``NetControl::DROP`` had 3 conflicting definitions that could potentially
be used incorrectly without any warnings or type-checking errors.
Such enum redefinition conflicts are now caught and treated as errors,
so the ``NetControl::DROP`` enums had to be renamed:
* The use as enum of type ``Log::ID`` is renamed to ``NetControl::DROP_LOG``
* The use as enum of type ``NetControl::CatchReleaseInfo`` is renamed to
``NetControl::DROP_REQUESTED``
* The use as enum of type ``NetControl::RuleType`` is unchanged and still
named ``NetControl::DROP`` (Jon Siwek, Corelight)
3.3.0-dev.44 | 2020-08-04 08:47:08 -0700
* Silence compiler warnings related to deprecated TYPE_COUNTER (Jon Siwek, Corelight)
3.3.0-dev.42 | 2020-08-03 10:37:38 -0700
* Remove counter type, only leaving compatibility enum. The type was unused/non-functional.
See GH-1095 (Johanna Amann, Corelight).
3.3.0-dev.38 | 2020-08-01 09:21:17 -0700
* Properly forward-declare zeek::plugin::plugin in DebugLogger (Tim Wojtulewicz, Corelight)
* Rename Brofiler to ScriptCoverageManager (Tim Wojtulewicz, Corelight)