[FIXIT] Website Cron Tests in Cloud Deploy #5760
Conversation
…the cloud deploy pipeline
dwnoble
left a comment
dwnoble
left a comment
There was a problem hiding this comment.
Maybe for a future PR, but can we also deprecate:
build/website_cron_testing/Dockerfile
build/ci/cloudbuild.website_testing.yaml (and the related cloud build trigger)
| rm -rf ./output/* | ||
| } | ||
|
|
||
| echo "Starting tests in parallel..." |
There was a problem hiding this comment.
Instead of starting the tests in parallel in the same step, can we start them in 3 separate steps that cloud build runs in parallel? I think it will be easier to debug and dig through the logs if they're running in separate steps
There was a problem hiding this comment.
So there's a big argument for keeping them in the same one. Post deploy hooks are run sequentially which then means it would run nodejs, sanity and adv tests one after another making it WAY longer.
I think the extra logging I added in this script make it easy-ish to sift through the logs though I do agree it would have been better as separate steps :). I tried but I think the lack of parallelization among post-deploy hooks makes it worth keeping them in the same build. Do you agree?
There was a problem hiding this comment.
Can't the cloudbuild.cron_tests.yaml start the jobs in parallel though? Using something like example on this page: https://docs.cloud.google.com/build/docs/configuring-builds/configure-build-step-order - (i could be misunderstanding the example though)
There was a problem hiding this comment.
ah yes you're right! The only downside then is the cloud deploy pipeline's logs wont show the tests running but it will show a link to the other cloud build action, which then has the tests running. That's probably fine!
mind if we submit as is, and I ll send a follow up to take it out once ive verified it?
There was a problem hiding this comment.
Nice! Thanks @gmechali
last nit: remove the echo "Starting tests in parallel..." line
Yes will leave this for a follow up! |
| rm -rf ./output/* | ||
| } | ||
|
|
||
| echo "Starting tests in parallel..." |
There was a problem hiding this comment.
Nice! Thanks @gmechali
last nit: remove the echo "Starting tests in parallel..." line
| args: | ||
| - "-c" | ||
| - | | ||
| gcloud builds submit --config build/website_cron_testing/cloudbuild.cron_tests.yaml \ |
There was a problem hiding this comment.
why is this kicked off as a gcloud builds submit whereas as autopush and dev are ./run_website_cron_tests.sh?
3 participants
Refactors the Website Cron Tests to allow them to run in a mode that returns a success/failure.
This also parallelizes the sanity, adversarial and nodejs query tests.
We also modified the adversarial tests to allow sampling so we can run on a subset of the 12k+ queries.
This work now enables us to link the website cron tests to our Cloud Deploy pipeline. Here's how I made it work: