AppenderDbi: Log to databases via DBI

Description

Log to a database table with any DBI compatible backend. Please be aware that AppenderDbi does not support case sensitive / quoted column names, and you advised to only use all-lowercase names for custom fields (see ... argument of LogEvent). When appending to a database table all LogEvent values for which a column exists in the target table will be appended, all others are ignored.

Arguments

Buffered Logging

AppenderDbi does not write directly to the database but to an in memory buffer. With the default settings, this buffer is written to the database whenever the buffer is full (buffer_size, default is 10 LogEvents), whenever a LogEvent with a level of fatal or error is encountered (flush_threshold) or when the Appender is garbage collected (flush_on_exit), i.e. when you close the R session or shortly after you remove the Appender object via rm(). If you want to disable buffering, just set buffer_size to 0.

Usage

x <- AppenderDbi$new(conn, table, threshold = NA_integer_, layout =
  select_dbi_layout(conn, table), close_on_exit = TRUE, buffer_size = 10,
  flush_threshold = "error", flush_on_exit = TRUE, flush_on_rotate = TRUE,
  should_flush = default_should_flush, filters = NULL)
x$add_filter(filter, name = NULL)
x$append(event)
x$filter(event)
x$flush()
x$format(color = FALSE, ...)
x$remove_filter(pos)
x$set_buffer_size(x)
x$set_close_on_exit(x)
x$set_conn(conn)
x$set_filters(filters)
x$set_flush_on_exit(x)
x$set_flush_on_rotate(x)
x$set_flush_threshold(level)
x$set_layout(layout)
x$set_should_flush(x)
x$set_threshold(level)
x$show(threshold = NA_integer_, n = 20)
x$show(threshold = NA_integer_, n = 20L)
x$buffer_df
x$buffer_dt
x$buffer_events
x$buffer_size
x$close_on_exit
x$col_types
x$conn
x$data
x$destination
x$dt
x$filters
x$flush_on_exit
x$flush_on_rotate
x$flush_threshold
x$layout
x$should_flush
x$table
x$table_id
x$table_name
x$threshold

Creating a New Appender

An AppenderDbi is linked to a database table via its table argument. If the table does not exist it is created either when the Appender is first instantiated or (more likely) when the first LogEvent would be written to that table. Rather than to rely on this feature, it is recommended that you create the target log table first manually using an SQL CREATE TABLE statement as this is safer and more flexible. See also LayoutDbi.

New Appenders are instantiated with <AppenderSubclass>$new(). For the arguments to new() please refer to the section Fields. You can also modify those fields after the Appender has been created with setters in the form of appender$set_<fieldname>(value)

Fields

Note: $data and show() query the data from the remote database and might be slow for very large logs.

close_on_exit, set_close_on_exit(): TRUE or FALSE. Close the Database connection when the Logger is removed?
conn, set_conn(conn): a DBI connection
table: Name of the target database table

buffer_size, set_buffer_size(x): integer scalar >= 0 Number of LogEvents to buffer.

buffer_events, buffer_df, buffer_dt

The contents of the buffer as a list of LogEvents, a data.frame or a data.table.

flush_threshold, set_flush_threshold()

integer or character log level. Minimum event level that will trigger flushing of the buffer. This behaviour is implemented through should_flush(), and you can modify that function for different behaviour.

should_flush(event), set_should_flush(x)

A function with exactly one arguments: event. If the function returns TRUE, flushing of the buffer is triggered. Defaults to flushing if an event of level error or higher is registered.

dt: Get the log recorded by this Appender as a data.table with a maximum of buffer_size rows

data: Get the log recorded by this Appender as a data.frame

threshold, set_threshold(level): character or integer scalar. The minimum log level that triggers this logger. See log_levels
layout, set_layout(layout): a Layout that will be used for formatting the LogEvents passed to this Appender
destination: The output destination of the Appender in human-readable form (mainly for print output)

filters, set_filters(filters): a list that may contain functions or any R object with a filter() method. These functions must have exactly one argument: event which will get passed the LogEvent when the Filterable's filter() method is invoked. If all of these functions evaluate to TRUE the LogEvent is passed on. Since LogEvents have reference semantics, filters can also be abused to modify them before they are passed on. Look at the source code of with_log_level() or with_log_value() for examples.

Choosing the Right DBI Layout

Layouts for relational database tables are tricky as they have very strict column types and further restrictions. On top of that implementation details vary between database backends.

To make setting up AppenderDbi as painless as possible, the helper function select_dbi_layout() tries to automatically determine sensible LayoutDbi settings based on conn and - if it exists in the database already - table. If table does not exist in the database and you start logging, a new table will be created with the col_types from layout.

Methods

flush(): Manually trigger flushing of the buffer

show(n, threshold): Show the last n log entries with a log level bellow threshold. The log entries will be formatted for console output via this Appenders Layout

append(event): Tell the Appender to process a LogEvent event. This method is usually not called by the user, but invoked by a Logger

filter(event): Determine whether the LogEvent x should be passed on to Appenders (TRUE) or not (FALSE). See also the active binding filters
add_filter(filter, name = NULL), remove_filter(pos): Add or remove a filter. When adding a filter an optional name can be specified. remove_filter() can remove by position or name (if one was specified)