Added a section for bad monitoring examples

[laurianttila]

This was discussed in LCM related workshop, feel free to groom the text in case it's awful.

[ilkka]

A thought: since these "war stories" are a new thing, what do you think about putting them in a .md file or a section of their own, and then linking to them from a shorter text that has the basic problem description and recommendation? That way, in case the war stories include multiple examples of bad practices, they could be linked to from multiple places in the document. Maybe the stories themselves could even have a section that spells out the causes of the problems, and links to the relevant best practices.

[kypeli]

These kinds of war stories are really valuable! Would it make sense to link to a separate .md file from this document describing how things can do wrong, but then also add to this document solutions or proposals on how to do monitoring? Pingdom and New Relic kind of services come to mind, that can do "smart alerting" when something happens.

[phadej]

in case

[phadej]

war stories: write a blog post and link to it. Also discuss what should have been the right (or at least better) approach. I'm 👎 on merging this as is.

[laurianttila]

Maybe rewrite this as "word of warning - use email alerts with caution" (with some additional words about hurry/small budget/easy way)? War stories then as a blog post :).

[ilkka]

I'd like to see something that clearly states that email alerts should be
reserved for high priority, disruptive situations, and care should be taken
to avoid false positives and to throttle them appropriately — otherwise
they either get ignored, or cause more problems downstream with their
volume.

su 7. helmikuuta 2016 klo 20.24 Lauri Anttila notifications@github.com
kirjoitti:

Maybe rewrite this as "word of warning - use email alerts with caution"
(with some additional words about hurry/small budget/easy way)? War stories
then as a blog post :).

—
Reply to this email directly or view it on GitHub
#15 (comment)
.

Ilkka Laukkanen
Senior Specialist
Futurice Oy