Replaying fails when converting big file into json format

[Swar2424]

Describe the bug

Hi, I have a common issue when I try to replay big capture files (like 300 Mo .kcap file)

When I use this command :

fibratus replay -k events.kcap --output.console.format json > capture.json

I get the following error :

panic: runtime error: slice bounds out of range [129756:109723]
        
goroutine 9 [running]:
github.com/rabbitstack/fibratus/pkg/kevent.(*Kevent).UnmarshalRaw(0xc0078956c0, {0xc007a3e000, 0x1ac9b, 0x1ac9b}, 0x2)
        D:/a/fibratus/fibratus/pkg/kevent/marshaller_windows.go:394 +0x224f
github.com/rabbitstack/fibratus/pkg/kevent.NewFromKcap({0xc007a3e000, 0x1ac9b, 0x1ac9b}, 0x2)
        D:/a/fibratus/fibratus/pkg/kevent/kevent.go:234 +0xa5
github.com/rabbitstack/fibratus/pkg/kcap.(*reader).Read.func1()
        D:/a/fibratus/fibratus/pkg/kcap/reader_windows.go:137 +0x306
created by github.com/rabbitstack/fibratus/pkg/kcap.(*reader).Read in goroutine 1
        D:/a/fibratus/fibratus/pkg/kcap/reader_windows.go:109 +0xc8

How to reproduce it

I have first realized a big capture, for testing purposes, by installing and removing programs, with the following command :

fibratus capture -o events.kcap

Then, I try to write it into a json file using the following command :

fibratus replay -k events.kcap --output.console.format json > capture.json

However, I get the error described above before I managed to load the complete capture into my json file

Expected behavior

I expected to get a full capture file without any incidents. When I tried the same steps with fibratus v2.3.0, it worked perfectly. However, after updating to v2.4.0, it does not work anymore.

Environment

• Fibratus version:

  Version : 2.4.0

  Commit : 6e9efb8

  Build date : 20-05-2025.17:13:06

  Go compiler : go1.23.9

• Configuration:

        aggregator ..................... [ flush-timeout=>4s flush-period=>500ms]
        alertsenders ................... [ mail=>[ port=>25 enabled=>false content-type=>text/html to=>[] use-template=>true] eventlog=>[ verbose=>true enabled=>true] slack=>[ enabled=>false] systray=>[ quiet-mode=>false enabled=>false sound=>true]]
        api ............................ [ timeout=>5s transport=>localhost:8482]
        config-file .................... C:\Program Files\fibratus\config\fibratus.yml
        debug-privilege ................ true
        filament ....................... [ path=>C:\Program Files\fibratus\filaments]
        filters ........................ [ rules=>[ enabled=>true from-urls=>[] from-paths=>[C:\Program Files\Fibratus\Rules\*]] macros=>[ from-paths=>[C:\Program Files\Fibratus\Rules\Macros\*]] match-all=>true]
        forward ........................ false
        handle ......................... [ init-snapshot=>false enumerate-handles=>false]
        kcap ........................... []
        kevent ......................... [ serialize-threads=>false serialize-handles=>false serialize-pe=>false serialize-envs=>false serialize-images=>false]
        kstream ........................ [ enable-thread=>true enable-dns=>true enable-registry=>true blacklist=>[ images=>[] events=>[CloseFile RegCloseKey]] enable-audit-api=>true enable-fileio=>true min-buffers=>16 enable-mem=>true enable-net=>true flush-interval=>1s enable-vamap=>true max-buffers=>36 enable-threadpool=>true enable-image=>true enable-handle=>false buffer-size=>512 stack-enrichment=>true]
        logging ........................ [ max-size=>100 max-age=>0 level=>info log-stdout=>false max-backups=>15 formatter=>text]
        output ......................... [ amqp=>[ routing-key=>fibratus exchange-type=>topic enabled=>false delivery-mode=>transient vhost=>/ url=>amqp://localhost:5672 exchange=>fibratus durable=>false tls-insecure-skip-verify=>false timeout=>5s passive=>false] http=>[ serializer=>json endpoints=>[] enabled=>false timeout=>5s enable-gzip=>false tls-insecure-skip-verify=>false method=>POST] eventlog=>[ level=>info enabled=>false] elasticsearch=>[ gzip-compression=>false servers=>[http://127.0.0.1:9200] sniff=>false flush-period=>1s healthcheck-timeout=>5s bulk-workers=>1 healthcheck=>true template-name=>fibratus trace-log=>false index-name=>fibratus enabled=>false healthcheck-interval=>10s timeout=>5s] console=>[ enabled=>true format=>pretty]]
        pe ............................. [ excluded-images=>[svchost.exe] enabled=>false read-symbols=>false read-resources=>false read-sections=>false]
        symbol-paths ................... srv*c:\\SymCache*https://msdl.microsoft.com/download/symbols
        symbolize-kernel-addresses ..... false
        transformers ................... [ trim=>[ enabled=>false] replace=>[ enabled=>false] remove=>[ kparams=>[] enabled=>false] tags=>[ enabled=>false] rename=>[ enabled=>false]]
        yara ........................... [ skip-registry=>false fastscan=>true scan-timeout=>10s enabled=>false skip-mmaps=>false skip-files=>false rule=>[ strings=>[map[namespace:<nil> string:<nil>]] paths=>[map[namespace: path:]]] excluded-procs=>[] excluded-files=>[] skip-allocs=>false]
  

• OS:
  Windows 11

[Swar2424]

Small correction here : I just got the same error with v2.3.0, with a 400 Mo .kcap file

[rabbitstack]

Hi @Swar2424 ,

Thanks for the detailed error report. Does the crash happens with any capture size or only capture with significant size?
Do you capture and reply on the same machine?

I'll try to repro the issue

[Swar2424]

I do the capture and replay on the same machine, but the issue only rises up with big files (i.e. over 300 Mo and 10 000 000 events). With smaller .kcap files, everything works properly. Also, when doing a replay without the --output.console.format json > capture.json flag, everything also works properly.

[rabbitstack]

Hi @Swar2424 ,

Does your capture contain sensitive data? If you don't have any objections to share the capture file, that would definitely streamline the investigation and troubleshooting on my side.

It is also odd that the process crashes only when you're replying with console output format set to JSON. The backtrace you attached in the issue might be a red herring then.

[Swar2424]

So I can't put the .kcap file here, since github doesn't support .kcap format for attachments. Do you have a way I could send it directly to you ?

[Swar2424]

Update : I got the same error WITHOUT any flags or filters. Maybe with my earlier tests I just wasn't waiting long enough to see this error. Just the command fibratus replay -k events.kcap makes the replay end after some time in the following way :

64509639 2025-06-03 16:51:06.2687968 +0200 CEST - 4 msiexec.exe (8444) - CreateFile (create_disposition➜ OPEN, file_name➜ C:\Program Files\Blender Foundation\Blender 4.4\4.4\python\lib\site-packages\numpy\core, type➜ File)
64509641 2025-06-03 16:51:06.2688335 +0200 CEST - 4panic: runtime error: slice bounds out of range [130000:109723]

goroutine 10 [running]:
github.com/rabbitstack/fibratus/pkg/kevent.(*Kevent).UnmarshalRaw(0xc00b830000, {0xc00b814000, 0x1ac9b, 0x1ac9b}, 0x2)
      D:/a/fibratus/fibratus/pkg/kevent/marshaller_windows.go:394 +0x224f
github.com/rabbitstack/fibratus/pkg/kevent.NewFromKcap({0xc00b814000, 0x1ac9b, 0x1ac9b}, 0x2)
      D:/a/fibratus/fibratus/pkg/kevent/kevent.go:234 +0xa5
github.com/rabbitstack/fibratus/pkg/kcap.(*reader).Read.func1()
      D:/a/fibratus/fibratus/pkg/kcap/reader_windows.go:137 +0x306
created by github.com/rabbitstack/fibratus/pkg/kcap.(*reader).Read in goroutine 1
      D:/a/fibratus/fibratus/pkg/kcap/reader_windows.go:109 +0xc8

Here the file events.kcap is 390 Mo.

[Swar2424]

I can also confirm it isn't a performance issue : according to windows task manager, with a processor usage of 55 % and a RAM usage of 45 % (approximately), I still have this error

[rabbitstack]

@Swar2424 could you upload the capture file to some file sharing platform (WeTransfer, Google Drive)? TIA

[Swar2424]

I uploaded 2 capture files on wetransfer that are unreplayable on my side, made with v2.3.0 I think (although the issue still remains for v2.4.0). You should receive the link by email.

[rabbitstack]

@Swar2424 got the capture files. Thanks! I'll investigate and come back to you with my findings.

[rabbitstack]

Hey @Swar2424,

I did a bit of triaging to narrow down the root cause. The problem occurs when a certain event type parameter is decoded. I'm suspecting the parameter is not written with the expected type modifier or either it is not treated in the parsing loop.

Could you try taking and then replaying some captures, but removing the RegSetValue event? For example:

fibratus capture "kevt.name != 'RegSetValue'" -o capture.kcap

[Swar2424]

I have narrowed it down a little, and it seems like it's only the RegSetValue events that pose an issue.

When I try to filter those that have an int-type value, i.e. REG_DWORD and REG_QWORD (I filter them with a registry.value !=0 or registry.value =0), there is no issue.

However I can't go much further since I can't manage to get any sort of event when I try to filter with registry.value.type. All kind of filters using registry.value.type return an empty kcap file (37 B, 0 events).

[Swar2424]

I have send you a kcap file that is unreplayable. Here's how I captured and replayed that file :

PS C:\Users\Félix\Documents\Test2> fibratus capture "kevt.name = 'RegSetValue'" -o events.kcap
┌─────────────────────────────────┐
│ Capture Statistics │
├───────────────────┬─────────────┤
│ File │ events.kcap │
├───────────────────┼─────────────┤
│ Events written │ 8306 │
│ Bytes written │ 5981227 │
│ Processes written │ 0 │
│ Handles written │ 0 │
├───────────────────┼─────────────┤
│ Capture size │ 536 kB │
└───────────────────┴─────────────┘

PS C:\Users\Félix\Documents\Test2> fibratus replay -k events.kcap
panic: runtime error: slice bounds out of range [130000:109723]

goroutine 52 [running]:
github.com/rabbitstack/fibratus/pkg/kevent.(*Kevent).UnmarshalRaw(0xc000cb6340, {0xc001280000, 0x1ac9b, 0x1ac9b}, 0x2)
D:/a/fibratus/fibratus/pkg/kevent/marshaller_windows.go:394 +0x224f
github.com/rabbitstack/fibratus/pkg/kevent.NewFromKcap({0xc001280000, 0x1ac9b, 0x1ac9b}, 0x2)
D:/a/fibratus/fibratus/pkg/kevent/kevent.go:234 +0xa5
github.com/rabbitstack/fibratus/pkg/kcap.(*reader).Read.func1()
D:/a/fibratus/fibratus/pkg/kcap/reader_windows.go:137 +0x306
created by github.com/rabbitstack/fibratus/pkg/kcap.(*reader).Read in goroutine 1
D:/a/fibratus/fibratus/pkg/kcap/reader_windows.go:109 +0xc8

PS C:\Users\Félix\Documents\Test2> fibratus version
┌─────────────┬─────────────────────┐
│ Version │ 2.4.0 │
│ Commit │ 6e9efb8 │
│ Build date │ 20-05-2025.17:13:06 │
├─────────────┼─────────────────────┤
│ Go compiler │ go1.23.9 │
└─────────────┴─────────────────────┘