logging (파이썬 로깅 모듈)

00 개요

• 회사에서 코드 분석하고 있는데 logging에 관한 코드가 너무 너무너무너무넘누머누 많이 나와서 잘 알아야겠다 하여 이해만 하고 넘어가는 것보다 제대로 정리하고자 함

01 Logging이란

1. Logging 정의

• Logging is a means of tracking events (이벤트 추적) that happen when some software runs. 
• The software’s developer adds logging calls to their code to indicate that certain events have occurred. 
• An event is described by a descriptive message which can optionally contain variable data (i.e. data that is potentially different for each occurrence of the event)
• Events also have an importance which the developer ascribes to the event; the importance can also be called the level or severity.
• 추적된 이벤트 다루는 방법 (handling tracked events, i.e., log): 콘솔에 출력, 디스크 파일에 쓰기 등

 

2. Logging Level/Severity (로깅 레벨)

• The logger methods are named after the level or severity of the events they are used to track. 
• 기본값 = WARNING
  • → only events of this level and above will be tracked, unless the logging package is configured to do otherwise.

• The standard levels and their applicability are described below (in increasing order of severity):

Level	Numeric Values	언제 사용 되는지 / 설명
logging.NOTSET	0	When set on a logger, indicates that ancestor loggers are to be consulted to determine the effective level.  If that still resolves to NOTSET, then all events are logged.  When set on a handler, all events are handled.
logging.DEBUG	10	Detailed info, typically of interest only when diagnosing problems
logging.INFO	20	Confirmation that things are working as expected
logging.WARNING (기본값)	30	An indication that something unexpected happened, or indicative of some problem in the near future (e.g., 'disk space low') The software is still working as expected
logging.ERROR	40	Due to a more serious problem, the software has not been able to perform some function
logging.CRITICAL	50	A serious error, indicating that the program itself may be unable to continue running

 

3. Logging을 사용해야하는 경우

• You can access logging functionality by creating a logger via logger = getLogger(__name__), and then calling the logger’s debug(), info(), warning(), error() and critical() methods
  • 번역: logger = getLogger(__name__)을 통해 Logger 객체를 생성 후 logger의 debug(), info() ,warning(), error(), critical() 메소드들을 통해 logger의 기능에 접근할 수 있음
• To determine when to use logging, and to see which logger methods to use when, see the table below
  • 번역: 언제 logging을 사용하고 어느 상황에 어느 logger 메소드를 봐야하는 지에 대해 알기 위해 다음 테이블을 참조

하고 싶은 일	알맞은 도구
Display console output for ordinary usage of a command line script or program	print()
Report events that occur during normal operation of a program  (e.g. for status monitoring or fault investigation)	Logger의 info() 메소드  또는 debug() 메소드 - for very detailed output for diagnostic purposes
Issue a warning regarding a particular runtime event	warnings.warn() in library code if the issue is avoidable and the client application should be modified to eliminate the warning Logger의 warning() 메소드 if there is nothing the client application can do about the situation, but the event should still be noted
Report an error regarding a particular runtime event	Raise an exception
Report suppression of an error w/o raising an exception (e.g. error handler in a long-running server process)	Logger의 error(), exception() or critical() 메소드 as appropriate for the specific error and application domain

 

01  logging 모듈

0. pip 설치

• logging이 설치 안 되어있다면 설치 필요 

# pip install --upgrade logging

1. logging 모듈 역할

• defines functions and classes which implement a flexible event logging system for applications and libraries
• The key benefit of having the logging API provided by a standard library module is that all Python modules can participate in logging, so your application log can include your own messages integrated with messages from third-party modules.
• logging tutorials: (https://docs.python.org/3/howto/logging.html#logging-basic-tutorial)

2. 간단한 사용법 예시

• Starts with creating a module level logger with getLogger(__name__), and using that logger to do any needed logging 

1) myapp.y 및 mylib.py 적기

# myapp.py 내용
import logging
import mylib
logger = logging.getLogger(__name__)   # Logger 객체 생성

def main():
    logging.basicConfig(
        filename='myapp.log',      # 로그가 기록될 파일, 존재하지 않을 경우 생성, 존재할 경우 추가됨
        level=logging.DEBUG,       # 기록될 로깅 레벨
                    )
    logger.debug('DEBUG message1')
    logger.info('Started')
    mylib.do_something()
    logger.info('Finished')
    logger.debug('DEBUG message2')

if __name__ == '__main__':
    main()

# mylib.py 파일 내용
import logging
logger = logging.getLogger(__name__)
def do_something():
    logger.info('Doing something')

2) myapp.py 실행

// myapp.py 실행
# python ./myapp.py

• shell에서 위의 명령어로 myapp.py 실행 시 myapp.log라는 파일이 생김과 동시 myapp.log 파일 안에 다음과 같은 내용이 적혀져있을 것

• myapp.py 한 번 더 실행 시 (→ # python ./myapp.py) myapp.log 파일 안에 myapp.py 모듈의 main() 함수의 실행 결과가 myapp.log 파일에 다음과 같이 추가되어 있을 것

3. Hierarchical Logging

• Logged messages to the module-level logger get forwarded to handlers of loggers in higher-level modules, all the way up to the highest-level logger known as the root logger; this approach is known as hierarchical logging

4. Quickly Configuring Logger via basicConfig()

• For logging to be useful, it needs to be configured: setting the levels and destinations for each logger, potentially changing how specific modules log, often based on command-line arguments or application configuration.
• In most cases, like the one above, only the root logger needs to be so configured, since all the lower level loggers at module level eventually forward their messages to its handlers.
• basicConfig() provides a quick way to configure the root logger that handles many use cases.

logging.basicConfig(**kwargs)

 

• basicConfig()
  • does basic configuration for the logging system by creating a StreamHandler w/ a default Formatter and adding it to the root logger
  • The functions debug(), info(), warning(), error() and critical() will automatically call basicConfig() if no handlers are defined for the root logger
  • does nothing if the root logger already has handlers configured, unless the keyword argument force is set to True
  • Note: This function should be called from the main thread before other threads are started. In versions of Python prior to 2.7.1 and 3.2, if this function is called from multiple threads, it is possible (in rare circumstances) that a handler will be added to the root logger more than once, leading to unexpected results such as messages being duplicated in the log.
  • keyword arguments:

kwargs	설명
filename	Specifies that a FileHandler be created, using the specified filename, rather than a StreamHandler Note: this argument is incompatible stream (if both are present, a ValueError is raised)
filemode	If filename is specified, open the file in this mode. (기본값 = 'a')
	mode 종류:  'r' -reading, synonym of 'rt' 'w' - writing, truncating the file first, 'x' - exclusive creation, failing if the file already exists, 'a' - writing, appending to the end of file if it exists, 'b' - binary mode, 't' - text mode, '+' - updating (reading and writing) 'w+' and 'w+b' truncate the file 'r+' and 'r+b' no truncation
format	Use the specified format string for the handler.  Defaults to attributes levelname, name and message separated by colons.
datefmt	Use the specified date/time format, as accepted by time.strftime().
style	If format is specified, use this style for the format string.  One of '%', '{' or '$' for printf-style, str.format() or string.Template respectively.  (기본값 = '%')
level	Set the root logger level to the specified level.
stream	Use the specified stream to initialize the StreamHandler.  Note: this argument is incompatible with filename (if both are present, a ValueError is raised)
handlers	If specified, this should be an iterable of already created handlers to add to the root logger.  Any handlers which don’t already have a formatter set will be assigned the default formatter created in this function.  Note: this argument is incompatible with filename or stream (if both are present, a ValueError is raised)
force	If this keyword argument is specified as true, any existing handlers attached to the root logger are removed and closed, before carrying out the configuration as specified by the other arguments.
encoding	If this keyword argument is specified along with filename, its value is used when the FileHandler is created, and thus used when opening the output file.
errors	If this keyword argument is specified along with filename, its value is used when the FileHandler is created, and thus used when opening the output file. If not specified, the value ‘backslashreplace’ is used. Note that if None is specified, it will be passed as such to open(), which means that it will be treated the same as passing ‘errors’.

5. Logging의 클래스 / 객체

• The basic classes defined by the module, together with their attributes and methods:
  • Loggers expose the interface that application code directly uses
  • Handlers send the log records/messages (created by loggers) to the appropriate destination.
  • Filters provide a finer grained facility for determining which log records to output. 
  • Formatters specify the layout of log records in the final output. Formatter objects configure the final order, structure, and contents of the log message.
• Log event information is passed between loggers, handlers, filters and formatters in a LogRecord instance

1) Logger Objects

• Threefold job of a Logger:
  • First, they expose several methods to application code so that applications can log messages at runtime.
  • Second, logger objects determine which log messages to act upon based upon severity (the default filtering facility) or filter objects
  • Third, logger objects pass along relevant log messages to all interested log handlers
• Loggers should NEVER be instantiated directly, but always through the module-level function logging.getLogger(name)
• Multiple calls to getLogger() with the same name will always return a reference to the same Logger object
• The name is potentially a period-separated hierarchical value, like foo.bar.baz (though it could also be just plain foo, for example)
• Loggers that are further down in the hierarchical list are children of loggers higher up in the list.
  • e.g., given a logger with a name of foo, loggers with names of foo.bar, foo.bar.baz, and foo.bam are all descendants of foo
• All loggers are descendants of the root logger.
• The logger name hierarchy is analogous to the Python package hierarchy, and identical to it if you organise your loggers on a per-module basis using the recommended construction logging.getLogger(__name__).
  • 이유: in a module, __name__ is the module’s name in the Python package namespace

• Loggers have the following attributes and methods:

class logging.Logger

속성	설명	예시
name	logger’s name, and is the value that was passed to getLogger() to obtain the logger	>>> import logging >>> my_logger = logging.getLogger('Pearl') >>> my_logger <Logger Pearl (WARNING)> >>> my_logger.name 'Pearl'
level	threshold of this logger, as set by setLevel() method	>>> import logging  >>> my_logger = logging.getLogger('Pearl')  >>> my_logger.level  0  >>> my_logger.setLevel(logging.INFO)  >>> my_logger.level  20
parent	the parent logger of this logger. it may change bbased on later instantiation of loggers which are higher up in the namespace hierarchy
propagate	If this attribute evaluates to true, events logged to this logger will be passed to the handlers of higher level (ancestor) loggers, in addition to any handlers attached to this logger. Messages are passed directly to the ancestor loggers’ handlers - neither the level nor filters of the ancestor loggers in question are considered.  If this evaluates to false, logging messages are not passed to the handlers of ancestor loggers.  Spelling it out with an example: If the propagate attribute of the logger named A.B.C evaluates to true, any event logged to A.B.C via a method call such as logging.getLogger('A.B.C').error(...) will [subject to passing that logger’s level and filter settings] be passed in turn to any handlers attached to loggers named A.B, A and the root logger, after first being passed to any handlers attached to A.B.C. If any logger in the chain A.B.C, A.B, A has its propagate attribute set to false, then that is the last logger whose handlers are offered the event to handle, and propagation stops at that point.  The constructor sets this attribute to True.
handlers	list of handlers directly attached to this logger instance can be changed via the addHandler() and removeHandler() methods, which use locks to ensure thread-safe operation
disabled	This attribute disables handling of any events. It is set to False in the initializer, and only changed by logging configuration code.

메소드	설명	예시
setLevel(level)	- Sets the threshold for this logger to level. - Logging messages which are less severe than level will be ignored; logging messages which have severity level or higher will be emitted by whichever handler or handlers service this logger, unless a handler’s level has been set to a higher severity level than level. - (기본값 = NOTSET → causes all messages to be processed when the logger is the root logger, or delegation to the parent when the logger is a non-root logger) - Note that the root logger is created with level WARNING. - The term ‘delegation to the parent’ means that if a logger has a level of NOTSET, its chain of ancestor loggers is traversed until either an ancestor with a level other than NOTSET is found, or the root is reached. - If an ancestor is found with a level other than NOTSET, then that ancestor’s level is treated as the effective level of the logger where the ancestor search began, and is used to determine how a logging event is handled. - If the root is reached, and it has a level of NOTSET, then all messages will be processed. Otherwise, the root’s level will be used as the effective level.	>>> import logging  >>> my_logger = logging.getLogger('Pearl')  >>> my_logger.level  0  >>> my_logger.setLevel(logging.INFO)  >>> my_logger.level  20
debug(msg, *args, **kwargs)	- Logs a message with level DEBUG on this logger. - msg = message format string - args = arguments which are merged into msg using the string formatting operator. - Note: you can use keywords in the format string, together with a single dictionary argument.) - No % formatting operation is performed on msg when no args are supplied - kwargs: exc_info, stack_info, stacklevel and extra
info(msg, *args, **kwargs)	- Logs a message with level INFO on this logger. - The arguments are interpreted as for debug()
warning(msg, *args, **kwargs)	- Logs a message with level WARNING on this logger - The arguments are interpreted as for debug().
error(msg, *args, **kwargs)	- Logs a message with level ERROR on this logger. - The arguments are interpreted as for debug().
critical(msg, *args, **kwargs)	- Logs a message with level CRITICAL on this logger. - The arguments are interpreted as for debug().
log(level, msg, *args, **kwargs)	- Logs a message with integer level level on this logger. - The other arguments are interpreted as for debug().
exception(msg, *args, **kwargs)	- Logs a message with level ERROR on this logger. - The arguments are interpreted as for debug(). - Exception info is added to the logging message. - This method should only be called from an exception handler.
addFilter(filter)	- Adds the specified filter filter to this logger
removeFilter(filter)	- Removes the specified filter filter from this logger.
filter(record)	- Apply this logger’s filters to the record and return True if the record is to be processed - The filters are consulted in turn, until one of them returns a false value. - If none of them return a false value, the record will be processed (passed to handlers). - If one returns a false value, no further processing of the record occurs.
addHandler(hdlr)	Adds the specified handler hdlr to this logger.
removeHandler(hdlr)	Removes the specified handler hdlr from this logger.
handle(record)	- Handles a record by passing it to all handlers associated with this logger and its ancestors (until a false value of propagate is found). - This method is used for unpickled records received from a socket, as well as those created locally. - Logger-level filtering is applied using filter().
hasHandlers()	- Checks to see if this logger has any handlers configured. - This is done by looking for handlers in this logger and its parents in the logger hierarchy. - Returns True if a handler was found, else False. - The method stops searching up the hierarchy whenever a logger with the ‘propagate’ attribute set to false is found - that will be the last logger which is checked for the existence of handlers.

2) Handler Objects

• Handler is never instantiated directly; this class acts as a base for more useful subclasses (however, the __init__() method in subclasses needs to call Handler.__init__())
• Handler objects are responsible for dispatching the appropriate log messages (based on the log messages’ severity) to the handler’s specified destination. 

• Handlers have the following attributes and methods:

class logging.Handler

메소드	설명	예
__init__(level=NOTSET)	Initializes the Handler instance by setting its level, setting the list of filters to the empty list and creating a lock (using createLock()) for serializing access to an I/O mechanism
createLock()	Initializes a thread lock which can be used to serialize access to underlying I/O functionality which may not be threadsafe.
acquire()	Acquires the thread lock created with createLock().
release()	Releases the thread lock acquired with acquire().
setLevel(level)	Sets the threshold for this handler to level.  Logging messages which are less severe than level will be ignored.  When a handler is created, the level is set to NOTSET (which causes all messages to be processed).
setFormatter(fmt)	Sets the Formatter for this handler to fmt.
addFilter(filter)	Adds the specified filter filter to this handler.
remove(filter)	Removes the specified filter filter from this handler.
filter(record)	Apply this handler’s filters to the record and return True if the record is to be processed.  The filters are consulted in turn, until one of them returns a false value.  If none of them return a false value, the record will be emitted.  If one returns a false value, the handler will not emit the record.
flush()	Ensure all logging output has been flushed.  This version does nothing and is intended to be implemented by subclasses.
close()	Tidy up any resources used by the handler.  This version does no output but removes the handler from an internal list of handlers which is closed when shutdown() is called.  Subclasses should ensure that this gets called from overridden close() methods.
handle(record)	Conditionally emits the specified logging record, depending on filters which may have been added to the handler.  Wraps the actual emission of the record with acquisition/release of the I/O thread lock.
handleError(record)	This method should be called from handlers when an exception is encountered during an emit() call.  If the module-level attribute raiseExceptions is False, exceptions get silently ignored.  This is what is mostly wanted for a logging system - most users will not care about errors in the logging system, they are more interested in application errors.  You could, however, replace this with a custom handler if you wish.  The specified record is the one which was being processed when the exception occurred.  (The default value of raiseExceptions is True, as that is more useful during development).
format(record)	Do formatting for a record If a formatter is set, use it. Otherwise, use the default formatter for the module.
emit(record)	Do whatever it takes to actually log the specified logging record.  This version is intended to be implemented by subclasses and so raises a NotImplementedError.

• Logger objects can add zero or more handler objects to themselves with an addHandler() method
  • 예) an application may want to send all log messages to a log file, all log messages of error or higher to stdout, and all messages of critical to an email address. This scenario requires three individual handlers where each handler is responsible for sending messages of a specific severity to a specific location.
  • The following are the types of Handlers ( In addition to the base Handler class, many useful subclasses are provided):

subclass Handlers	설명
StreamHandler	StreamHandler instances send messages to streams (file-like objects).
FileHandler	FileHandler instances send messages to disk files.
BaseRotatingHandler	BaseRotatingHandler is the base class for handlers that rotate log files at a certain point.  It is not meant to be instantiated directly.  Instead, use RotatingFileHandler or TimedRotatingFileHandler.
RotatingFileHandler	RotatingFileHandler instances send messages to disk files, with support for maximum log file sizes and log file rotation.
TimedRotatingFileHandler	TimedRotatingFileHandler instances send messages to disk files, rotating the log file at certain timed intervals.
SocketHandler	SocketHandler instances send messages to TCP/IP sockets.  Since 3.4, Unix domain sockets are also supported.
DatagramHandler	DatagramHandler instances send messages to UDP sockets.  Since 3.4, Unix domain sockets are also supported.
SMTPHandler	SMTPHandler instances send messages to a designated email address.
SysLogHandler	SysLogHandler instances send messages to a Unix syslog daemon, possibly on a remote machine.
NTEventLogHandler	NTEventLogHandler instances send messages to a Windows NT/2000/XP event log.
MemoryHandler	MemoryHandler instances send messages to a buffer in memory, which is flushed whenever specific criteria are met.
HTTPHandler	HTTPHandler instances send messages to an HTTP server using either GET or POST semantics.
WatchedFileHandler	WatchedFileHandler instances watch the file they are logging to. If the file changes, it is closed and reopened using the file name. This handler is only useful on Unix-like systems; Windows does not support the underlying mechanism used.
QueueHandler	QueueHandler instances send messages to a queue, such as those implemented in the queue or multiprocessing modules.
NullHandler	NullHandler instances do nothing with error messages.  They are used by library developers who want to use logging, but want to avoid the ‘No handlers could be found for logger XXX’ message which can be displayed if the library user has not configured logging.

 

 

 

참조

1. (Logging이란) https://docs.python.org/3/howto/logging.html#logging-basic-tutorial
2. (Logging 모듈) https://docs.python.org/3/library/logging.html
3. (logging 소스코드) https://github.com/python/cpython/blob/3.12/Lib/logging/__init__.py
4. (Handlers) https://docs.python.org/3/library/logging.handlers.html#module-logging.handlers
5.  
6.