Add-x, --recover-records option

[YamatoSecurity]

There is a PR that tries to recover more event records here: https://github.com/forensicmatt/evtx/tree/feature/parse-empty-pages
I would like to create a branch in the hayabusa-evtx repository and create an option --recover-records in Hayabusa that will try to recover records. I am not sure if data will be corrupted, etc.. so this needs testing.

[hitenkoku]

This issue must be updated in hayabusa-evtx. I have created a similar issue in hayabusa-evtx

[hitenkoku]

@YamatoSecurity I checked in https://github.com/Yamato-Security/hayabusa/tree/952-test-on-whether-to-add-recover-records-option

I tested file in evtx repository ( https://github.com/forensicmatt/evtx/blob/feature/parse-empty-pages/samples/Microsoft-Windows-WorkFolders%254WHC.evtx )

In the main branch, 0 cases were detected due to broken records, but we confirmed that 2 cases can be detected with this modification.

• main branch

./main.exe csv-timeline -f ..\Microsoft-Windows-WorkFolders-WHC.evtx 
...
Total event log files: 1
Total file size: 69.6 KB

Loading detections rules. Please wait.

Excluded rules: 31
Noisy rules: 12 (Disabled)

Deprecated rules: 165 (6.84%) (Disabled)
Experimental rules: 1269 (52.61%)
Stable rules: 196 (8.13%)
Test rules: 947 (39.26%)
Unsupported rules: 45 (1.87%) (Disabled)

Hayabusa rules: 159
Sigma rules: 2253
Total enabled detection rules: 2412

Output profile: standard

Scanning in progress. Please wait.

1 / 1   [========================================] 100% 0s

"..\\Microsoft-Windows-WorkFolders-WHC.evtx"



Results Summary:

Events with hits / Total events: 0 / 0 (Data reduction: 0 events (0.00%))

Total | Unique detections: 0 | 0
Total | Unique critical detections: 0 (0.00%) | 0 (0.00%)
Total | Unique high detections: 0 (0.00%) | 0 (0.00%)
Total | Unique medium detections: 0 (0.00%) | 0 (0.00%)
Total | Unique low detections: 0 (0.00%) | 0 (0.00%)
Total | Unique informational detections: 0 (0.00%) | 0 (0.00%)
...
Elapsed time: 00:00:02.289

• https://github.com/Yamato-Security/hayabusa/tree/952-test-on-whether-to-add-recover-records-option

.\952.exe csv-timeline -f ..\Microsoft-Windows-WorkFolders-WHC.evtx 

> .\952.exe csv-timeline -f ..\Microsoft-Windows-WorkFolders-WHC.evtx  -q
...
Total event log files: 1
Total file size: 69.6 KB

Loading detections rules. Please wait.

Excluded rules: 31
Noisy rules: 12 (Disabled)

Deprecated rules: 165 (6.84%) (Disabled)
Experimental rules: 1269 (52.61%)
Stable rules: 196 (8.13%)
Test rules: 947 (39.26%)
Unsupported rules: 45 (1.87%) (Disabled)

Hayabusa rules: 159
Sigma rules: 2253
Total enabled detection rules: 2412

Output profile: standard

Scanning in progress. Please wait.

[00:00:00] 1 / 1   [========================================] 100%                                                                                                                            
                                                                                                                                                                                              
Scanning finished. Please wait while the results are being saved.




Results Summary:

Events with hits / Total events: 0 / 2 (Data reduction: 2 events (100.00%))

Total | Unique detections: 0 | 0
Total | Unique critical detections: 0 (0.00%) | 0 (0.00%)
Total | Unique high detections: 0 (0.00%) | 0 (0.00%)
Total | Unique medium detections: 0 (0.00%) | 0 (0.00%)
Total | Unique low detections: 0 (0.00%) | 0 (0.00%)
Total | Unique informational detections: 0 (0.00%) | 0 (0.00%)

...

Elapsed time: 00:00:01.753

[YamatoSecurity]

@hitenkoku Oh great! Thanks so much. I will do some testing with this.

[YamatoSecurity]

@hitenkoku Is it possible to tell if a record was recovered from the empty pages or not? That is, I would like to count the number of recovered records.
For example, output Recovered records: 500

Also, is there a way to turn it on and off. For example, we can enable or disable it by default and have an option to turn on or off if it is causing problems.

[hitenkoku]

@YamatoSecurity The fix has been completed and can be implemented only when the --recover-records option is used.

The hayabusa-evtx repository needed to be modified, please check Yamato-Security/hayabusa-evtx#34 to see if this has been done.

I will update hayabusa submodule when hayabusa-evtx pull request is complete.