cscli allowlists should support FQDN for dynamically changed addresses

[nvtkaszpir]

What would you like to be added?

/kind enhancement

cscli allowlists add my-private-address-list my-fqdn.tld -d 'home address list dynamic'

Why is this needed?

Right now cscli allowlist allows to add IP addresses to the allowlist.

What I would like to have is to pass FQDN which would be resolved to the given IP when evaluated.
I know resolving the address in the moment of the evaluation can be heavy, though DNS server should be actually responsible for handling that, so it may not be needed to additionally implement any kind of caching mechanism.

This would allow users with dynamic IP to be a bit safer in certain scenarios, such as your home IP is changed by the ISP and you unfortunately land on blocked IP address - and you have not much space to change it over next 24h.

Otherwise I need to use cron and a script to keep it up to date.

[github-actions]

@nvtkaszpir: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

1. Check Crowdsec Documentation to see if your issue can be self resolved.
2. You can also join our Discord.
3. Check Releases to make sure your agent is on the latest version.

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[github-actions]

@nvtkaszpir: There are no 'kind' label on this issue. You need a 'kind' label to start the triage process.

• /kind feature
• /kind enhancement
• /kind refactoring
• /kind bug
• /kind packaging

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[nvtkaszpir]

/kind enhancement

[LaurenceJJones]

Hey 👋🏻

We initially thought of this when we was designing the system, however, after a first prototype we decided it felt very hacky and wasnt very optimal as we had alot of variables to take into account:

• TTL?
• Custom DNS?
• Who does DNS in a multi server?

And we came to the conclusion it probably best we do not support this as a supported option, however, doesnt mean a user cannot do this at all a simple bash script can do this please note this relies on a feature not yet released #3684 but I can create one without:

#!/bin/bash

# Full paths to binaries (adjust if needed)
CSCLI="/usr/local/bin/cscli"
DIG="/usr/bin/dig"
JQ="/usr/bin/jq"

DOMAIN="$1"
ALLOWLIST_NAME="$2"

if [[ -z "$DOMAIN" || -z "$ALLOWLIST_NAME" ]]; then
    echo "Usage: $0 <domain> <allowlist_name>"
    exit 1
fi

# Resolve A and AAAA records
readarray -t CURRENT_IPS < <(for t in A AAAA; do "$DIG" +short "$t" "$DOMAIN"; done)

# DNS has no IP's do we flush the allowlist?
if [[ ${#CURRENT_IPS[@]} -eq 0 ]]; then
    echo "No IPs found for domain $DOMAIN"
    exit 1
fi

# Track whether allowlist was just created
ALLOWLIST_CREATED=false

# Check if allowlist exists
if ! "$CSCLI" allowlist list -ojson | "$JQ" -e ".[] | select(.name == \"$ALLOWLIST_NAME\")" > /dev/null; then
    echo "Allowlist '$ALLOWLIST_NAME' not found, creating it..."
    "$CSCLI" allowlist create "$ALLOWLIST_NAME" -d 'automated by dns script'
    ALLOWLIST_CREATED=true
fi

if [[ "$ALLOWLIST_CREATED" == true ]]; then
    echo "Adding all current IPs to newly created allowlist '$ALLOWLIST_NAME'"
    "$CSCLI" allowlist add "$ALLOWLIST_NAME" "${CURRENT_IPS[@]}"
    exit 0
fi

# Get existing IPs from the allowlist
readarray -t EXISTING_IPS < <("$CSCLI" allowlist list -ojson | "$JQ" -r ".[] | select(.name == \"$ALLOWLIST_NAME\") | .items[].value")

# Track changes
IPS_TO_ADD=()
IPS_TO_REMOVE=()

for ip in "${CURRENT_IPS[@]}"; do
    if ! "$CSCLI" allowlist check "$ip" >/dev/null 2>&1; then
        IPS_TO_ADD+=("$ip")
    fi
done

for ip in "${EXISTING_IPS[@]}"; do
    if [[ ! " ${CURRENT_IPS[*]} " =~ " $ip " ]]; then
        IPS_TO_REMOVE+=("$ip")
    fi
done

if [[ ${#IPS_TO_ADD[@]} -gt 0 ]]; then
    echo "Adding IPs to $ALLOWLIST_NAME: ${IPS_TO_ADD[*]}"
    "$CSCLI" allowlist add "$ALLOWLIST_NAME" "${IPS_TO_ADD[@]}"
fi

if [[ ${#IPS_TO_REMOVE[@]} -gt 0 ]]; then
    echo "Removing IPs from $ALLOWLIST_NAME: ${IPS_TO_REMOVE[*]}"
    "$CSCLI" allowlist remove "$ALLOWLIST_NAME" "${IPS_TO_REMOVE[@]}"
fi

Then you can create a cronjob which executes this script as:

*/30 * * * * /path/to/sync-allowlist.sh example.com my_allowlist

so this handles:

• Allowlist doesnt exist so create and add all IP's
• Check if IP's are new or old and add / remove if needed

However, if there is community appetite around it dont hesitate to 👍🏻 the OP and we can reconsider.

[nvtkaszpir]

Thanks for th details, will look into this.
Meanwhile I also noticed there may be an easier way with poatoverflows https://docs.crowdsec.net/docs/whitelist/create_fqdn/

[LaurenceJJones]

Thanks for th details, will look into this. Meanwhile I also noticed there may be an easier way with poatoverflows docs.crowdsec.net/docs/whitelist/create_fqdn

Yes postoverflows can already handle this but the only issue is if your in a multi server environment you will need to implement this across all nodes, as the goal with allowlist is you have a central place to control it, the other option is the hacky LAPI workaround which we did suggest until allowlist:

https://docs.crowdsec.net/docs/next/log_processor/whitelist/create_lapi