Python Logging Module Logging Timestamp To Include Microsecond

I am using python's logging module for logs, but needed the timestamp to include microsecond. It seems the timestamp can only get as precise as millisecond. Here's my test code imp

Solution 1:

According to the documentation, strftime() does not support %f. The logger provides milliseconds as a separate msecs attribute, so you could just add it yourself after the existing timestamp as follows:

logging.basicConfig(format='%(asctime)s.%(msecs)03d %(levelname)s {%(module)s} [%(funcName)s] %(message)s', datefmt='%Y-%m-%d,%H:%M:%S', level=logging.INFO)

This gave me the following output using your script:

2015-07-10,09:21:16.841 INFO {test script} [get_started2] Logged2 Here

Solution 2:

I just ran into this issue - and it can be solved. It just requires a little hacking on some of the logging infrastructure. See below example:

import logging
import time

try: # Python >= 3.7from time import time_ns
except: # Python <= 3.6from time import time as _time_
    time_ns = lambda: int(_time_() * 1e9)

classLogRecord_ns(logging.LogRecord):
    def__init__(self, *args, **kwargs):
        self.created_ns = time_ns() # Fetch precise timestampsuper().__init__(*args, **kwargs)

classFormatter_ns(logging.Formatter):
    default_nsec_format = '%s,%09d'defformatTime(self, record, datefmt=None):
        if datefmt isnotNone: # Do not handle custom formats here ...returnsuper().formatTime(record, datefmt) # ... leave to original implementation
        ct = self.converter(record.created_ns / 1e9)
        t = time.strftime(self.default_time_format, ct)
        s = self.default_nsec_format % (t, record.created_ns - (record.created_ns // 10**9) * 10**9)
        return s

logging.setLogRecordFactory(LogRecord_ns)

# +++++ DEMO +++++

log_formater = Formatter_ns('%(asctime)s (%(name)s) %(message)s')

logger = logging.getLogger('demo-log')
logger.setLevel(logging.DEBUG)

ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
ch.setFormatter(log_formater)
logger.addHandler(ch)

logger.info('foo bar')

This will happily print: 2019-04-10 14:08:28,819931368 (demo-log) foo bar

Key to this is a modified logging.Formatter class, which has a custom implementation of formatTime. Just to be on the safe side, I also recommend to use time.time_ns, which will return an integer in nano seconds in Python 3.7 and beyond. The original time.time returns a float in seconds, which therefore obviously has precision issues. Getting the more precise timestamp into a log record is achieved through a modified logging.LogRecord class, which simply fetches its created_ns field from time.time_ns in its extended constructor method.

Solution 3:

I didnt find a easy way to print out microsecond,but %(created).6f could be a temp solution, which will be the result of time.time(),like 1517080746.007748.

Didnt find a way to remove unnecessary part, so if you really need microsecond, but dont want to change your code too much, one easy way will be

logging.basicConfig(level=logging.INFO,format="%(asctime)s.%(msecs)03d[%(levelname)-8s]:%(created).6f %(message)s", datefmt="%Y-%m-%d %H:%M:%S")

It will give you below output,

2018-01-2804:19:06.807[INFO    ]:1517080746.807794buyorderissued2018-01-2804:19:07.007[INFO    ]:1517080747.007806buyorderissued2018-01-2804:19:07.207[INFO    ]:1517080747.207817buyorderissued2018-01-2804:19:07.407[INFO    ]:1517080747.407829buyorderissued2018-01-2804:19:07.607[INFO    ]:1517080747.607840buyorderissued

Solution 4:

Just completing @Luo_Hongshuai answer for python 3.7 :

format=%(asctime)s.%(msecs)06f
datefmt=%Y-%m-%d %H:%M:%S

Solution 5:

I have same question: Using logging.Formatter need timestamp in micro seconds with exact 6 digits. Something like: 2021-11-02 15:21:12.891531

After goin through the answers and checking out the other SO links mentioned here I couldn't find a way to get timestamp in this format. Does anyone know how to get timestamp in 2021-11-02 15:21:12.891531 format?

I have tried following and the comment next to each line in code is what that line prints as date.

1#!/bla/bla/bla/bin/python23 import logging
  45 logger = logging.getLogger(__name__)
  6 l_level = 'INFO'7 l_level = eval("logging." + l_level.upper())
  8 logger.setLevel(l_level)
  910 handler = logging.StreamHandler()
 11 handler.setLevel(l_level)
 1213#formatter = logging.Formatter('%(asctime)s|%(message)s', datefmt="%Y-%m-%d %H:%M:%S.%s") # 2021-11-02 15:12:59.163588037914#formatter = logging.Formatter('%(asctime)s.%(msecs)06d|%(message)s', datefmt="%Y-%m-%d %H:%M:%S") # 2021-11-02 15:11:50.00095215#formatter = logging.Formatter('%(asctime)s.%(msecs)03d|%(message)s', datefmt="%Y-%m-%d %H:%M:%S") # 2021-11-02 15:12:10.63316 formatter = logging.Formatter('%(asctime)s.%(msecs)06f', datefmt="%Y-%m-%d %H:%M:%S") # 2021-11-02 15:18:04.274.37210117 handler.setFormatter(formatter)
 18 logger.addHandler(handler)
 1920 logger.info("")

I'll add more ways here if I find new options.