Learn More
Resend Email
23
Root
6y

What features would you want in a logger?

Here's what I'm planning so far:
- Tagged entries for easy scanning of log file
- Support for indenting to group similar sequential entries
- Multiple entry types (normal, info, event, warning, error, fatal, debug, verbose)
- Meta entries, so the logger logging about itself, e.g. disk i/o failures.
- Ability to add custom entry types, including tag, log-level, etc.
- Customizable timestamp function
- Support for JS's async nature -- this equates to passing a unique key per 'thread'; the logger will re-write all the parent blocks for context, if necessary. if that sounds confusing, it's okay; just trust that it makes sense.
- Caching, retries, etc. in the event of disk i/o issues.
- Support for custom writers, allowing you to e.g. write logs to an API rather than console or disk.

How about these features?
- Multiple (named) logs with separate writers (console, disk, etc.)
- Ability to individually enable/disable writing of specific entry types. (want verbose but not info? sure thing, weirdo!)
- Multiple writers per log. Combined with the above, this would allow you to write specific entry types (e.g. error, warning, fatal) to stderr instead of stdout, or to different apis.
- Ability to write the same log entry to multiple logs simultaneously

What do you think of these features?
What other features would you want?
I'm open to suggestions!

question

root of the open source

logging library

sylvan
Favorite
Ranter
Comments
  • 6
    6y
    My custom logger I put into my C# apps does most of that. The format of a line is roughly:
    *) DateStamp
    *) Severity
    *) Scope
    *) Description
    *) Message

    Elements separated by tabs, written to disk, echoed to console (configurable per call, default True), rolled per app launch (.log -> .bak), thread safe but singleton pattern.
  • 2
    6y
    @bkwilliams are you trying to one up op?
  • 3
    6y
    Just wondering, why do you need to write a logger.

    So far I've never cared much... Either log4j or Winston.

    I'm actually more interested in using ELK cuz text logs are pretty useless... Especially when they are 200mb/API/day and there are N APIs on N machine.
  • 6
    6y
    @billgates Good logs are beautiful, useful things. Awful logs are giant blobs of barely-readable text.

    I want to write a library to allow others to make beautiful logs, too.
  • 0
    6y
    @Root but for usual corporate cases, you have multiple machines and the logs are so large the information is not very useful. Just like Big Data. The details maybe important, but you need to be able to locate them first?

    Most of the time after finding which machine it's at, I just grep for the line and look for the error stack trace. Then just go-to the code and see how it's possible that this error is thrown.
  • 3
    6y
    @billgates One of the aims is to increase contextual information, which would still help in your example. Regardless, this is kind of off topic, so I don't really want to discuss it here. I'd like more input on the features listed above instead!
  • 2
    6y
    Periodic logging.

    This way, if your code runs in a loop, everything can "write to the log", but actual IO only happens at the end of a loop.
  • 2
    6y
    @jesustricks nah, @Root tagged this as open source, I can’t do that with my current code base (is a CS file I add to my solution and use). I gave him the features and format I used because he asked.
    If he did a dotNetCore build it would be one less thing for me to port later or I could help with his effort.
  • 2
    6y
    @bkwilliams she 😋

    But no, I'm writing this in js, not .net
  • 3
    6y
    @Root that’s what I get for not having avatars turned on in the thread; sorry.
  • 2
    6y
    Compability with IE6 and IE7!

    Just kidding. I'll show myself out.
  • 0
    6y
    Mainly something that unifies the reading of the different types of application logs and presents it in a meaningful way, without all the cruft. Like with failed logins on an sshd I only care about how many times it happened, and maybe from where.

    Definitely needs a good API too. But one that isn't absolutely necessary for the program to function. Grafana + Prometheus appears to have a nice API but it's also way too much work to set up in smaller deployments. Netdata on the other hand isn't really that customizable and doesn't really give much more than system performance charts, but it's super easy to deploy. Execute one-line command and go pretty much.

    Goaccess is something I like because it's also possible to view in a TUI. But it doesn't support much more than webserver logs as far as I'm aware.. kinda sucks. But I like being able to view logs - truncated ones that do away with the cruft - from the comfort of my shell.
  • 1
    6y
    @Condor Looks like I have some research to do.

    Indentation and tagging helps with contextual info, but how free of cruft it is really is up to what you write to the log.

    As far as reading logs from shell... I haven't figured that one out yet. I could have a writer to generate XML logs, but who likes XML? Adding a block name to each line would break vertical aligning, so that won't work either.
  • 2
    6y
    @Root JSON > XML
  • 1
    6y
    @bkwilliams Absolutely! Doesn't really solve the problem, though. 🙁
  • 0
    6y
    @Root When considering any kind of API, definitely JSON over XML! Indentation and tagging-wise.. well JSON doesn't really have an issue with indentation so I wouldn't worry too much about that ¯\_(ツ)_/¯ but tagging could definitely be something as valuable as variables would be to your fellow and future project maintainer.

    As for free of cruft, that's kind of a hard question really.. it depends on each log and what I tend to look for. On sshd logs for example I tend to look at successful logins first, and then the unsuccessful ones grouped by IP. OpenVPN for example I like to see source public IP for and VPN profile name, also grouped in successful login attempts. I believe most of those are in the logs but yeah.. it depends on service log you're looking at, sysadmin you're talking to, so many things...

    If needed, I'm on Telegram at https://t.me/leCondor or on Signal as well. Or via email at something like devrant.2112888@nixmagic.com :)
  • 0
    6y
    @Condor Sounds like you want SQL-like querying of logfiles.
  • 0
    6y
    @Root I've got no experience whatsoever with databases 😅 so SQL, definitely not! But yeah something universal for the API anyway, SQL or JSON or anything like that.. well on SQL you of course have Postgres vs MariaDB inconsistencies, and having that as an API would bring it closer to the critical infrastructure than I'd like - a JSON API server provides its own layer. But yeah something universal like that, definitely!
devRant © 2021 Hexical Labs LLC
Privacy Policy  |  Terms of Service
Add Comment