Skip to content

Commit 431d320

Browse files
danielme85danielme85
authored
Merge pull request #17 from danielme85/dev
Add log-cleaner-upper
2 parents cc2b3ae + a0f49a6 commit 431d320

File tree

9 files changed

+291
-81
lines changed

9 files changed

+291
-81
lines changed

‎.travis.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ services:
66
- mysql
77
- mongodb
88
before_script:
9-
- pecl install mongodb
9+
- pecl install -f mongodb
1010
- mysql -e 'CREATE DATABASE testing;'
1111
- composer self-update
1212
- composer install --no-interaction

‎readme.md

Lines changed: 44 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,15 @@
1010
Custom Laravel 6 and >=5.6 Log channel handler that can store log events to SQL or MongoDB databases.
1111
Uses Laravel native logging functionality.
1212

13+
14+
* [Installation](#installation)
15+
* [Configuration](#configuration)
16+
* [Usage](#usage)
17+
* [Fetching Logs](#fetching-logs)
18+
* [Log Cleanup](#log-cleanup)
19+
* [Processors](#processors)
20+
21+
1322
## Installation
1423
Use the composer require or add to composer.json.
1524
```
@@ -29,7 +38,7 @@ You will need to add an array under 'channels' for Log-to-DB here like so:
2938
'stack' => [
3039
'name' => 'Log Stack',
3140
'driver' => 'stack',
32-
'channels' => ['database', 'mongodb'],
41+
'channels' => ['database', 'file'],
3342
],
3443

3544
'database' => [
@@ -43,6 +52,8 @@ You will need to add an array under 'channels' for Log-to-DB here like so:
4352
'queue' => false,
4453
'queue_name' => '',
4554
'queue_connection' => '',
55+
'max_records' => false,
56+
'max_hours' => false,
4657
'processors' => [
4758
//Monolog\Processor\HostnameProcessor::class
4859
]
@@ -65,39 +76,31 @@ More info about some of these options: https://laravel.com/docs/5.6/logging#cust
6576
There are some default settings and more information about configuring the logger in the 'logtodb.php' config file.
6677
This could be copied to your project if you would like edit it with the vendor publish command.
6778
```
68-
php artisan vendor:publish
79+
php artisan vendor:publish --provider="danielme85\LaravelLogToDB\ServiceProvider"
6980
```
70-
You can also set default log-to-db config settings in your .env file, for ex:
81+
You can also change these settings in your env file.
7182
```
7283
LOG_DB_CONNECTION='default'
7384
LOG_DB_DETAILED=false
7485
LOG_DB_MAX=100
7586
LOG_DB_QUEUE=false
7687
LOG_DB_QUEUE_NAME='logToDBQueue'
7788
LOG_DB_QUEUE_CONNECTION='default'
78-
79-
## Usage
80-
Use the default Laravel Facade "Log"
81-
```php
82-
Log::channel()->info("This thing just happened");
83-
Log::channel()->warning("This kind of bad thing happened...");
84-
```
85-
You can give the logging channels whatever name you want instead of: 'database', as well as the log levels.
86-
The naming can be used later if you want to send a Log event to a specific channel:
87-
```php
88-
Log::channel('database')->info("This thing just happened");
89-
Log::channel('mongodb')->info("This thing just happened");
89+
LOG_DB_MAX_COUNT=false
90+
LOG_DB_MAX_HOURS=false
9091
```
91-
This logger works the same as any other across Laravel, for example you can add it to a stack.
92-
You can log multiple levels to multiple DB connections... the possibilities are ENDLESS! 😎
9392

9493
#### Config priority order
95-
Lowest number has highest priority (overrides the one below);
96-
1. Log Channel config array in config/logging.php overrides logtodb.php
97-
2. .env File overrides config/logtodb.php file
98-
3. config/logtodb.php is the default config.
94+
There are three places you can change different options when using log-to-db:
95+
1. The config file: config/logtodb.php (after doing vendor:publish).
96+
2. Your .env file will override settings in the logtodb.php config file.
97+
3. The Laravel logging config file: config/logging.php. You need to add a custom array here as mentioned above,
98+
in this same array you can specify/override config settings specifically for that log channel.
9999

100-
### Log Worker Queue
100+
Config values set in point 1 & 2 would work as default for all new log channels you add in the "channels" array for the
101+
Laravel logging configuration (config/logging.php).
102+
103+
#### Log Worker Queue
101104
It might be a good idea to save the log events with a Queue Worker. This way your server does not have to wait for
102105
the save process to finish. You would have to configure the Laravel Queue settings and run the Queue listener.
103106
https://laravel.com/docs/5.6/queues#running-the-queue-worker
@@ -124,7 +127,7 @@ You can also log to specific log channels:
124127
Log::channel('database')debug("This is an test DEBUG log event");
125128

126129

127-
### Fetching Logs
130+
## Fetching Logs
128131
The logging by this channel is done trough the Eloquent Model builder.
129132
LogToDB::model($channel, $connection, $collection);
130133
You can skip all function variables and the default settings from the config/logtodb.php will be used.
@@ -211,7 +214,19 @@ No migrations needed for MongoDB.
211214

212215
No indexes are added per default, so if you fetch a lot of log results based on specific time ranges or types: it might be a good idea to add some indexes.
213216

214-
#### Log Cleanup
217+
## Log Cleanup
218+
There are config values that you can set to specify the max number of log records to keep, or the max record age in hours.
219+
220+
* logging.php channel array -> (max_records, max_hours).
221+
* .env file -> (LOG_DB_MAX_COUNT, LOG_DB_MAX_HOURS).
222+
223+
<b>These option is set to *false* per default, these have to be set to desired integers before you can run the "log:delete" artisan command.</b>
224+
```
225+
php artisan log:delete
226+
```
227+
This command will delete records based on settings described above. Add this command to your Console/kernel.php, or run manually in cron etc to enable automatic cleanup.
228+
229+
#### Manual Cleanup
215230
There is a helper function to remove the oldest log events and keep a specified number
216231
```php
217232
LogToDB::removeOldestIfMoreThan(100);
@@ -224,7 +239,7 @@ LogToDB::model()->removeOlderThan('2019-01-01');
224239
LogToDB::model()->removeOlderThan('2019-01-01 23:00:00');
225240
```
226241

227-
#### Processors
242+
## Processors
228243
Monolog ships with a set of [processors](https://github.com/Seldaek/monolog/tree/master/src/Monolog/Processor), these will generate additional data and populate the 'extra' field.
229244

230245
You could also create your own custom processor, make sure they implement [Monolog\Processor\ProcessorInterface](https://github.com/Seldaek/monolog/blob/master/src/Monolog/Processor/ProcessorInterface.php).
@@ -250,7 +265,7 @@ class PhpVersionProcessor implements ProcessorInterface {
250265

251266
```
252267

253-
#### Advanced /config/logging.php example
268+
## More logging.php config examples
254269
```php
255270
'default' => env('LOG_CHANNEL', 'stack'),
256271

@@ -269,7 +284,9 @@ class PhpVersionProcessor implements ProcessorInterface {
269284
'detailed' => true,
270285
'queue' => true
271286
'queue_name' => 'logQueue'
272-
'queue_connection' => 'redis'
287+
'queue_connection' => 'redis',
288+
'max_records' => 1000,
289+
'max_hours' => 24,
273290
],
274291

275292
'mongodb' => [

‎src/Commands/LogCleanerUpper.php

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
<?php
2+
/**
3+
* Created by PhpStorm.
4+
* User: dmellum
5+
* Date: 2/2/20
6+
* Time: 10:38 PM
7+
*/
8+
9+
namespace danielme85\LaravelLogToDB\Commands;
10+
11+
use Carbon\Carbon;
12+
use danielme85\LaravelLogToDB\LogToDbHandler;
13+
use danielme85\LaravelLogToDB\LogToDB;
14+
use Illuminate\Console\Command;
15+
16+
class LogCleanerUpper extends Command
17+
{
18+
/**
19+
* The name and signature of the console command.
20+
*
21+
* @var string
22+
*/
23+
protected $signature = 'log:delete';
24+
25+
/**
26+
* The console command description.
27+
*
28+
* @var string
29+
*/
30+
protected $description = 'Cleanup/delete/prune/trim log records.';
31+
32+
/**
33+
* Create a new command instance.
34+
*
35+
* @return void
36+
*/
37+
public function __construct()
38+
{
39+
parent::__construct();
40+
}
41+
42+
/**
43+
* Execute the console command.
44+
*
45+
*/
46+
public function handle()
47+
{
48+
$config = config('logtodb');
49+
$channels = $this->getLogToDbChannels();
50+
51+
if (!empty($channels)) {
52+
foreach ($channels as $channel) {
53+
$maxRecords = $channel['max_records'] ?? $config['purge_log_when_max_records'] ?? false;
54+
$maxHours = $channel['max_hours'] ?? $config['purge_log_when_max_records'] ?? false;
55+
$connection = $channel['connection'] ?? 'default';
56+
$collection = $channel['collection'] ?? 'log';
57+
58+
//delete based on numbers of records
59+
if (!empty($maxRecords) && $maxRecords > 0) {
60+
if (LogToDB::model($connection, $collection)->removeOldestIfMoreThan($maxRecords)) {
61+
$this->warn("Deleted oldest log records on channel: {$connection}->{$collection}, keep max number of records: {$maxRecords}");
62+
}
63+
}
64+
65+
//delete based on age
66+
if (!empty($maxHours) && $maxHours > 0) {
67+
$time = Carbon::now()->subHours($maxHours)->toDateTimeString();
68+
if (LogToDB::model($connection, $collection)->removeOlderThan($time)) {
69+
$this->warn("Deleted log records on channel: {$connection}->{$collection}, older than: {$time}");
70+
}
71+
}
72+
}
73+
}
74+
}
75+
76+
/**
77+
* @return array
78+
*/
79+
private function getLogToDbChannels()
80+
{
81+
$list = [];
82+
$logging = config('logging');
83+
if (!empty($logging) && isset($logging['channels']) && !empty($logging['channels'])) {
84+
foreach ($logging['channels'] as $log) {
85+
if (isset($log['via']) && $log['via'] === LogToDbHandler::class) {
86+
$list[] = $log;
87+
}
88+
}
89+
}
90+
91+
return $list;
92+
}
93+
}

‎src/LogToDB.php

Lines changed: 44 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -36,64 +36,85 @@ class LogToDB
3636
public $collection;
3737

3838
/**
39-
* @var mixed
39+
* Use queue driver to save record.
40+
* @var bool
4041
*/
4142
public $saveWithQueue;
4243

4344
/**
44-
* @var mixed
45+
* Name of the queue to use.
46+
* @var string
4547
*/
4648
public $saveWithQueueName;
4749

4850
/**
49-
* @var mixed
51+
* Queue connection to use
52+
* @var string
5053
*/
5154
public $saveWithQueueConnection;
5255

56+
/**
57+
* Purge after number of records
58+
* @var int
59+
*/
60+
private $purgeAfterNumber;
61+
62+
/**
63+
* Purge after number of hours old
64+
* @var int
65+
*/
66+
private $purgeAfterNumberOfHours;
67+
5368
/**
5469
* The DB config details
5570
* @var null
5671
*/
5772
public $database;
5873

74+
5975
/**
6076
* LogToDB constructor.
6177
*
62-
* @param string|null $channelConnection
63-
* @param string|null $collection
64-
* @param bool|null $detailed
65-
* @param bool|null $queue
66-
* @param string|null $queueName
67-
* @param string|null $queueConnection
78+
* @param string $connection The DB connection name to use.
79+
* @param string $collection The name of the collection/table to use.
80+
* @param array $config Array of combined config values.
6881
*/
6982
function __construct(string $channelConnection = null,
7083
string $collection = null,
7184
bool $detailed = null,
7285
bool $queue = null,
7386
string $queueName = null,
74-
string $queueConnection = null)
87+
string $queueConnection = null,
88+
int $purgeAfterNumber = null,
89+
int $purgeAfterNumberOfHours = null)
7590
{
7691
//Log default config if present
7792
$config = config('logtodb');
7893
if (!empty($config)) {
79-
if (isset($config['connection'])) {
94+
if (isset($config['connection']) && !empty($config['connection'])) {
8095
$this->connection = $config['connection'];
8196
}
82-
if (isset($config['collection'])) {
97+
if (isset($config['collection']) && !empty($config['collection'])) {
8398
$this->collection = $config['collection'];
8499
}
85-
if (isset($config['detailed'])) {
100+
if (isset($config['detailed']) && !empty($config['detailed'])) {
86101
$this->detailed = $config['detailed'];
87102
}
88-
if (isset($config['queue_db_saves'])) {
103+
if (isset($config['queue_db_saves']) && !empty($config['queue_db_saves'])) {
89104
$this->saveWithQueue = $config['queue_db_saves'];
90105
}
91-
if (isset($config['queue_db_name'])) {
106+
if (isset($config['queue_db_name']) && !empty($config['queue_db_name'])) {
92107
$this->saveWithQueueName = $config['queue_db_name'];
93108
}
94-
if (isset($config['queue_db_connection'])) {
109+
if (isset($config['queue_db_connection']) && !empty($config['connection'])) {
95110
$this->saveWithQueueConnection = $config['queue_db_connection'];
96111
}
112+
if (isset($config['purge_log_when_max_records']) && !empty($config['purge_log_when_max_records'])) {
113+
$this->purgeAfterNumber = $config['purge_log_when_max_records'];
114+
}
115+
if (isset($config['purge_log_when_max_hours']) && !empty($config['purge_log_when_max_hours'])) {
116+
$this->purgeAfterNumberOfHours = $config['purge_log_when_max_hours'];
117+
}
97118
}
98119

99120
//Set config based on specified config from the Log handler
@@ -115,6 +136,12 @@ function __construct(string $channelConnection = null,
115136
if (!empty($queueConnection)) {
116137
$this->saveWithQueueConnection = $queueConnection;
117138
}
139+
if (!empty($purgeAfterNumber)) {
140+
$this->purgeAfterNumber = $purgeAfterNumber;
141+
}
142+
if (!empty($purgeAfterNumberOfHours)) {
143+
$this->purgeAfterNumberOfHours = $purgeAfterNumberOfHours;
144+
}
118145

119146
//Get the DB connections
120147
$dbconfig = config('database.connections');
@@ -232,7 +259,7 @@ public function newFromMonolog(array $record)
232259
dispatch(new SaveNewLogEvent($this, $record))
233260
->onQueue($this->saveWithQueueName);
234261
}
235-
} else {
262+
} else {
236263
$log = CreateLogFromRecord::generate(
237264
$this->connection,
238265
$this->collection,

0 commit comments

Comments
 (0)