OpenPGP: Mitigate some RuntimeExceptions uncovered using Fuzzing

[vanitasvitae]

Hey!

I did run junit-jazzer agains some PGPainless functions and uncovered some bugs in the bcpg codebase.

The commits in this PR harmonize the way malformed or strange packets are handled, by preventing runtime exceptions, such as IndexOutOfBounds, NegativeArraySize, IllegalArgumentExceptions and OutOfMemory errors.

I'm only just getting started with fuzzing, but I can highly recommend it!

[ligefeiBouncycastle]

Hi @vanitasvitae,
Thanks for your valuable PR in hardening the PGP packet handling against malformed inputs. Your fuzzing work is much appreciated.

Regarding the UserAttributeSubpacketInputStream.MAX_LEN setting (currently 5MB):

• We should revisit this limit – a prior fix in the C# library (PR #638) encountered a similar issue where packet lengths could theoretically reach 4,082,119,084 bytes (~3.8GB). The resolution there set the max to 2³⁰ bytes (1,073,741,824 bytes)

[vanitasvitae]

Personally I think 2^30 is still unreasonable (or at least unrealistic :D), but its still an improvement over the current 2^64 :D

Is it okay, if I leave it as a public static int such that applications can overwrite this value if needed?
Is there a configuration mechanism I could perhaps reuse? I think I saw something done with Properties at some other place in the code?

[ligefeiBouncycastle]

While 2³⁰ bytes (~1GB) still feels excessive for real-world use cases, it’s a practical upper bound and certainly more sane than the 2⁶⁴ max.

I updated the logic in UserAttributeSubpacketInputStream.readPacket() to check StreamUtil.flag_eof before evaluating bodyLen, since a true flag_eof may validly result in a bodyLen of -1. This avoids potential false positives when handling EOF-flagged packets.

Thank you.

[peterdettman]

Please do not add mutable statics. If some limit is warranted here, use org.bouncycastle.util.Properties class for configuration.

However limit checking should begin at the top/outermost level with a constraint on the total size of the input. Low-level classes could be optimized to spot that their length is greater than the remaining limit. It looks like SignatureSubpacketInputStream started to head in that direction.