Skip to content

Commit 44f5379

Browse files
cbruno10Priyanka-Chatterjee-2000
cbruno10
and
Priyanka-Chatterjee-2000
authored
Add Activity Dashboard and various benchmarks and detections (#1)
Co-authored-by: Priyanka Chatterjee <priyanka.chatterjee@turbot.com>
1 parent 24cc479 commit 44f5379

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

61 files changed

+2478
-83
lines changed

‎README.md‎

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,14 @@
11
# Apache Access Log Detections Mod for Powerpipe
22

3-
View dashboards, run detections and scan for anomalies across your Apache access logs.
3+
[Tailpipe](https://tailpipe.io) is an open-source CLI tool that allows you to collect logs and query them with SQL.
44

5-
<!--
6-
TODO: Insert images
7-
-->
5+
The [Apache Access Log Detections Mod](https://hub.powerpipe.io/mods/turbot/tailpipe-mod-apache-access-log-detections) contains pre-built dashboards and detections, which can be used to monitor and analyze activity across your Apache servers.
6+
7+
Run detection benchmarks:
8+
![image](docs/images/apache_access_log_owasp_dashboard.png)
9+
10+
View insights in dashboards:
11+
![image](docs/images/apache_access_log_activity_dashboard.png)
812

913
## Documentation
1014

@@ -102,13 +106,12 @@ List available benchmarks:
102106
powerpipe benchmark list
103107
```
104108

105-
<!-- TODO: add a benchmark name and uncomment
106109
Run a benchmark:
107110

108111
```sh
109-
powerpipe benchmark run apache_access_log_detections.benchmark.
112+
powerpipe benchmark run apache_access_log_detections.benchmark.owasp_top_10
110113
```
111-
-->
114+
112115
Different output formats are also available, for more information please see
113116
[Output Formats](https://powerpipe.io/docs/reference/cli/benchmark#output-formats).
114117

@@ -126,4 +129,4 @@ Want to help but don't know where to start? Pick up one of the `help wanted` iss
126129

127130
- [Powerpipe](https://github.com/turbot/powerpipe/labels/help%20wanted)
128131
- [Tailpipe](https://github.com/turbot/tailpipe/labels/help%20wanted)
129-
- [Apache Access Log Detections Mod](https://github.com/turbot/tailpipe-mod-apache-access-log-detections/labels/help%20wanted)
132+
- [Apache Access Log Detections Mod](https://github.com/turbot/tailpipe-mod-apache-access-log-detections/labels/help%20wanted)

‎dashboards/activity_dashboard.pp‎

Lines changed: 339 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,339 @@
1+
dashboard "activity_dashboard" {
2+
title = "Access Log Activity Dashboard"
3+
documentation = file("./dashboards/docs/activity_dashboard.md")
4+
5+
tags = {
6+
type = "Dashboard"
7+
service = "Apache/AccessLog"
8+
}
9+
10+
container {
11+
# Analysis
12+
card {
13+
query = query.activity_dashboard_total_logs
14+
width = 2
15+
}
16+
17+
card {
18+
query = query.activity_dashboard_success_count
19+
width = 2
20+
type = "ok"
21+
}
22+
23+
card {
24+
query = query.activity_dashboard_redirect_count
25+
width = 2
26+
type = "info"
27+
}
28+
29+
card {
30+
query = query.activity_dashboard_bad_request_count
31+
width = 2
32+
type = "alert"
33+
}
34+
35+
card {
36+
query = query.activity_dashboard_error_count
37+
width = 2
38+
type = "alert"
39+
}
40+
}
41+
42+
container {
43+
44+
chart {
45+
title = "Requests by Day"
46+
query = query.activity_dashboard_requests_by_day
47+
width = 6
48+
type = "line"
49+
}
50+
51+
chart {
52+
title = "Requests by HTTP Method"
53+
query = query.activity_dashboard_requests_by_http_method
54+
width = 6
55+
type = "bar"
56+
}
57+
58+
chart {
59+
title = "Requests by Status Code"
60+
query = query.activity_dashboard_requests_by_status_code
61+
width = 6
62+
type = "pie"
63+
}
64+
65+
chart {
66+
title = "Top 10 User Agents (Requests)"
67+
query = query.activity_dashboard_requests_by_user_agent
68+
width = 6
69+
type = "pie"
70+
}
71+
72+
chart {
73+
title = "Top 10 Clients (Requests)"
74+
query = query.activity_dashboard_top_10_clients
75+
width = 6
76+
type = "table"
77+
}
78+
79+
chart {
80+
title = "Top 10 URLs (Requests)"
81+
query = query.activity_dashboard_top_10_urls
82+
width = 6
83+
type = "table"
84+
}
85+
86+
chart {
87+
title = "Top 10 URLs (Successful Requests)"
88+
query = query.activity_dashboard_requests_by_successful_requests
89+
width = 6
90+
type = "table"
91+
}
92+
93+
chart {
94+
title = "Top 10 URLs (Errors)"
95+
query = query.activity_dashboard_requests_by_errors
96+
width = 6
97+
type = "table"
98+
}
99+
}
100+
101+
}
102+
103+
# Queries
104+
query "activity_dashboard_total_logs" {
105+
title = "Log Count"
106+
description = "Count the total Apache log entries."
107+
108+
sql = <<-EOQ
109+
select
110+
count(*) as "Total Requests"
111+
from
112+
apache_access_log;
113+
EOQ
114+
}
115+
116+
query "activity_dashboard_success_count" {
117+
title = "Successful Request Count"
118+
description = "Count of successful HTTP requests (status 2xx)."
119+
120+
sql = <<-EOQ
121+
select
122+
count(*) as "Successful (2xx)"
123+
from
124+
apache_access_log
125+
where
126+
status between 200 and 299;
127+
EOQ
128+
}
129+
130+
query "activity_dashboard_redirect_count" {
131+
title = "Redirect Request Count"
132+
description = "Count of redirect HTTP requests (status 3xx)."
133+
134+
sql = <<-EOQ
135+
select
136+
count(*) as "Redirections (3xx)"
137+
from
138+
apache_access_log
139+
where
140+
status between 300 and 399;
141+
EOQ
142+
}
143+
144+
query "activity_dashboard_bad_request_count" {
145+
title = "Bad Request Count"
146+
description = "Count of client error HTTP requests (status 4xx)."
147+
148+
sql = <<-EOQ
149+
select
150+
count(*) as "Bad Requests (4xx)"
151+
from
152+
apache_access_log
153+
where
154+
status between 400 and 499;
155+
EOQ
156+
}
157+
158+
query "activity_dashboard_error_count" {
159+
title = "Server Error Count"
160+
description = "Count of server error HTTP requests (status 5xx)."
161+
162+
sql = <<-EOQ
163+
select
164+
count(*) as "Server Errors (5xx)"
165+
from
166+
apache_access_log
167+
where
168+
status between 500 and 599;
169+
EOQ
170+
}
171+
172+
query "activity_dashboard_top_10_clients" {
173+
title = "Top 10 Clients (Requests)"
174+
description = "List the top 10 client IPs by request count."
175+
176+
sql = <<-EOQ
177+
select
178+
remote_addr as "Client IP",
179+
count(*) as "Request Count"
180+
from
181+
apache_access_log
182+
group by
183+
remote_addr
184+
order by
185+
count(*) desc,
186+
remote_addr
187+
limit 10;
188+
EOQ
189+
}
190+
191+
query "activity_dashboard_top_10_urls" {
192+
title = "Top 10 URLs (Requests)"
193+
description = "List the top 10 requested URLs by request count."
194+
195+
sql = <<-EOQ
196+
select
197+
request_uri as "URL",
198+
count(*) as "Request Count"
199+
from
200+
apache_access_log
201+
where
202+
request_uri is not null
203+
group by
204+
request_uri
205+
order by
206+
count(*) desc,
207+
request_uri
208+
limit 10;
209+
EOQ
210+
}
211+
212+
query "activity_dashboard_requests_by_day" {
213+
title = "Requests by Day"
214+
description = "Count of requests grouped by day."
215+
216+
sql = <<-EOQ
217+
select
218+
strftime(tp_timestamp, '%Y-%m-%d') as "Date",
219+
count(*) as "Request Count"
220+
from
221+
apache_access_log
222+
group by
223+
strftime(tp_timestamp, '%Y-%m-%d')
224+
order by
225+
strftime(tp_timestamp, '%Y-%m-%d');
226+
EOQ
227+
}
228+
229+
query "activity_dashboard_requests_by_status_code" {
230+
title = "Requests by Status Code"
231+
description = "Count of rqeuests grouped by status code."
232+
233+
sql = <<-EOQ
234+
select
235+
case
236+
when status between 200 and 299 then '2xx Success'
237+
when status between 300 and 399 then '3xx Redirect'
238+
when status between 400 and 499 then '4xx Client Error'
239+
when status between 500 and 599 then '5xx Server Error'
240+
else 'Other'
241+
end as "Status Category",
242+
count(*) as "Request Count"
243+
from
244+
apache_access_log
245+
where
246+
status is not null
247+
group by
248+
"Status Category"
249+
order by
250+
"Status Category";
251+
EOQ
252+
}
253+
254+
query "activity_dashboard_requests_by_http_method" {
255+
title = "Requests by HTTP Method"
256+
description = "Distribution of HTTP methods used in requests."
257+
258+
sql = <<-EOQ
259+
select
260+
request_method as "HTTP Method",
261+
count(*) as "Request Count"
262+
from
263+
apache_access_log
264+
where
265+
request_method is not null
266+
group by
267+
request_method
268+
order by
269+
count(*) asc,
270+
request_method;
271+
EOQ
272+
}
273+
274+
query "activity_dashboard_requests_by_successful_requests" {
275+
title = "Top 10 URLs (Successful Requests)"
276+
description = "List the top 10 requested URLs by successful request count."
277+
278+
sql = <<-EOQ
279+
select
280+
request_uri as "Path",
281+
count(*) as "Request Count",
282+
string_agg(distinct status::text, ', ' order by status::text) as "Status Codes"
283+
from
284+
apache_access_log
285+
where
286+
status between 200 and 299
287+
and request_uri is not null
288+
group by
289+
request_uri
290+
order by
291+
count(*) desc,
292+
request_uri
293+
limit 10;
294+
EOQ
295+
}
296+
297+
query "activity_dashboard_requests_by_errors" {
298+
title = "Top 10 URLs (Errors)"
299+
description = "List the top 10 requested URLs by error count."
300+
301+
sql = <<-EOQ
302+
select
303+
request_uri as "Path",
304+
count(*) as "Error Count",
305+
string_agg(distinct status::text, ', ' order by status::text) as "Status Codes"
306+
from
307+
apache_access_log
308+
where
309+
status between 400 and 599
310+
and request_uri is not null
311+
group by
312+
request_uri
313+
order by
314+
count(*) desc,
315+
request_uri
316+
limit 10;
317+
EOQ
318+
}
319+
320+
query "activity_dashboard_requests_by_user_agent" {
321+
title = "Top 10 User Agents (Requests)"
322+
description = "Distribution of user agents in requests."
323+
324+
sql = <<-EOQ
325+
select
326+
http_user_agent as "User Agent",
327+
count(*) as "Request Count"
328+
from
329+
apache_access_log
330+
where
331+
http_user_agent is not null
332+
group by
333+
http_user_agent
334+
order by
335+
count(*) desc,
336+
http_user_agent
337+
limit 10;
338+
EOQ
339+
}

‎dashboards/docs/activity_dashboard.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
This dashboard answers the following questions:
2+
3+
- How many HTTP requests has the Apache server handled?
4+
- What is the distribution of HTTP status codes (success, redirect, client errors, server errors)?
5+
- What HTTP methods are being used most frequently?
6+
- How has request volume changed over time?
7+
- Which browsers and tools are accessing the server?
8+
- Which client IPs are generating the most traffic?
9+
- Which URIs are most frequently requested?
10+
- Which paths have the most successful requests?
11+
- Which paths are generating the most errors?

0 commit comments

Comments
 (0)