Add logic to identify insufficient resources causing timeout

[bpgould]

The step function kept failing on the lowest memory option that I provided. This was due to a timeout on the lambda. Either the powertuning step function should also increment timeout (not ideal) or it should recognize if every execution on a particular memory amount fails, it is not enough memory and proceed with the other values. I have achieved similar functionality before when using parallel states. Moreover, this greatly decreases the value of the tool because it is hard to implement in automation if I need to remove all of the lower memory values in order to achieve reliability of the step function.

[bpgould]

also for another issue, it only supports up to 3,008 mb but lambda supports 10,240.

[alexcasalboni]

Hi @bpgould - thanks for reaching out :)

Can you tell me more about your timeout issue? By default, the tool times out (each step) after 5 minutes. But you can change this by customizing the totalExecutionTimeout deployment parameter (this needs to be set when you deploy the application).

Regarding the memory requirements, you are correct. You do need to be aware of what's the minimum memory your function needs to operate properly (and not try to power-tune with lower memory). The tool currently works under the assumption that your function run properly with the given configurations.

About the 3,008MB limit, that sounds odd. Are you using a recent version of the app? As documented here, you can provide any power values between 128MB and 10,240MB, and please note that new AWS accounts have reduced concurrency and memory quotas, 3008MB max.

[bpgould]

@alexcasalboni thank you for your response. The memory limit was my issue, however, I think an enhancement is needed for the other issue. If I am deploying powertuning at scale to run in an account with 100 lambdas, how would I know what the min memory is for each lambda?

The tool should have an option to process timeouts = insufficient memory, this way for lambdas that can run with 128mb, the tool will run them and I don't have to set something much higher to accommodate other functions that require more. Perhaps this is a contribution that I could make.

[alexcasalboni]

@bpgould I think that adding support for this use case makes sense.

Practically, you still need to go function-by-function and craft a specific payload for each function if you want to power-tune 100+ functions in your account. But I agree that being forced to know the minimum memory each function requires is a bit too much :)

So what would you prefer to see in the state machine output and in the final visualization in case one (or more) memory configurations aren't enough to run your function? We wouldn't have any data point to plot on the chart, so we'd just leave it empty?

[alexcasalboni]

@bpgould I was thinking that we could add a new input parameter called ignoreTimeouts=true|false, so the state machine would ignore timeouts and simply not include them in the results.

Would that work for you?

[straygar]

Just to add to this, I also found it cumbersome to set up the experiment, due to having the state machine fail several times due to OOM in my case. I already know that at the lower bound it can fail due to timeouts or OOM, and I want the state machine to find the lowest lowest memory I can feasibly use.

Would be good for the visuals to also show the max memory used during the execution, just to see how close to the limit we're running. :)

[bpgould]

I will try and prioritize this enhancement on my side since this is still something I need to make the tool useful for me (and some others).