lib: added logger api in node core#60468
Conversation
|
Review requested:
|
|
thats my first performance results: I think need improvement for child logger area, but ı'm so happy because other results it looks nice. node:logger vs pino ➜ node git:(mert/create-logger-api/node-core) ✗ ./node benchmark/logger/vs-pino.js n=100000
logger/vs-pino.js scenario="simple" logger="node-logger" n=100000: 5,240,540.823813018
logger/vs-pino.js scenario="child" logger="node-logger" n=100000: 2,635,847.7027229806
logger/vs-pino.js scenario="disabled" logger="node-logger" n=100000: 159,436,487.67795104
logger/vs-pino.js scenario="fields" logger="node-logger" n=100000: 3,619,336.304205216
logger/vs-pino.js scenario="simple" logger="pino" n=100000: 3,398,489.9761368227
logger/vs-pino.js scenario="child" logger="pino" n=100000: 4,489,799.803418606
logger/vs-pino.js scenario="disabled" logger="pino" n=100000: 119,772,384.56038144
logger/vs-pino.js scenario="fields" logger="pino" n=100000: 1,257,930.8609750536 |
|
I now learn this feat in Pino I will try add in node:logger |
|
This will require support for serializers |
I wonder, should we name them like Pino does (built-in serializers), or go with something like standardSerializers ? |
Follow pino and we’ll change it |
I tryed serializer implement for logger, and some bench result repaired and fields and simple are experiencing a decline; I will try to resolve these fields: 3.62M → 2.16M (-40%) previously, the results for the child logger were quite slow at around 70%, but the new results have dropped to 18% and have actually improved significantly. I continue to try new methods. |
I inspected pino and I learn some patterns, than I applied this commit and new results! 1ededc7 I used this patterns removed the cost of serializing bindings in each log for the child logger, simple: 6.06M vs 3.48M ops/s (+74% faster) |
|
hello @mcollina do you any comments or suggestions for this end commits, I'm curious 🙏 . |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #60468 +/- ##
==========================================
+ Coverage 89.70% 89.74% +0.03%
==========================================
Files 692 697 +5
Lines 214012 215131 +1119
Branches 41054 41157 +103
==========================================
+ Hits 191984 193067 +1083
- Misses 14091 14127 +36
Partials 7937 7937
🚀 New features to boost your workflow:
|
f5b0108 to
429359b
Compare
As Matteo said, Pino has baggage that we can't realistically remove. I have provided a few tips along the way of this PR, but I really don't have the energy to devote to an actual review of it. So I can't speak to the implementation as it is today. The last time I looked at it, it was going in the right direction and I assume that has continued. If we are at the "go or no-go" point, I can attempt to use it and provide feedback on if it works as I would expect or not. |
Co-authored-by: Yagiz Nizipli <yagiz@nizipli.com>
|
@mertcanaltin Can you update the description of this pull-request? Please provide a full description of all the changes you're bringing, the performance characteristics of this new module etc. |
I Updated PR description |
I don't think there's any need to rush into this must land immediately. We can provide people the time to review it properly if anyone wants to do so. Personally, I think it's in good shape at this point, but it's also a somewhat big and critical system so it makes sense that we might want to wait a bit for deeper review to ensure we're fully confident in the architecture. |
I don't understand this reply. I am not trying to rush anything. I am stating that the best I can do for a review is to try out the "done" product and provide feedback at that point. I have no idea if that point has been reached. |
|
@Qard as a month has gone by since your last commit, is this still an issue? I think anyone who might have wanted to review this would by now have had the time. |
| logs independently. | ||
|
|
||
| ```mjs | ||
| import { Logger, JSONConsumer } from 'node:logger'; |
There was a problem hiding this comment.
I'd actually prefer if this weren't a new top-level module, and don't really think it's needs to be. We can attach this to require('console') or require('util') ... or event require('console/logging')
There was a problem hiding this comment.
I think a separate module is better. IMHO it wouldn't make sense to have multiple things related to logger in node:util.
There was a problem hiding this comment.
require('console/logging') counts as a separate module. The key thing I'd rather us avoid is Yet Another Top-Level module.
There was a problem hiding this comment.
Do you think node:console/logging would work?
This keeps it as a separate module while avoiding the creation of a new top-level namespace, and we already have a precedent for this pattern with node:test/reporters.
There was a problem hiding this comment.
I don't really like console a the prefix here, because:
- This is not a whatwg console
- Having console in the name implies it logs to stdout, which is not the point of the new module
That said, I will not block on this, whatever's the chosen name
There was a problem hiding this comment.
Agreed with @atlowChemi. Putting this under node:console seems very strange. A logger does not necessarily target a tty, which is what I expect when I read "console". node:logging or node:logger make the most sense to me. What is the problem with a new module name?
There was a problem hiding this comment.
... What is the problem with a new module name?
A couple reasons: (1) new top-level modules have historically been required to be semver-major, which means no backporting to lts versions, and this is something I'd like to see backported. (2) There's an old logger module on npm and even though this would be node:logger that still makes things a bit confusing... My general preference is to hang new things off existing modules as the first choice and only falling back to a new top-level module when nothing existing fits. It doesn't need to be console if folks feel strongly about that.
There was a problem hiding this comment.
I think that is very backward. Adding a new module should be a minor change. Changing an existing module should be a major change. I can agree that adding a subspaced module off of an existing module can be a minor change change. But I'd be much more easily convinced that it is a major change than I would be that adding a new module altogether is a major change.
There was a problem hiding this comment.
Per the collaborator guide:
For pull requests introducing new core modules:
* Allow at least one week for review.
* Land only after sign-off from at least two TSC voting members.
* Land with a [Stability Index][] of Experimental. The module must remain
Experimental until a semver-major release.
Experimental stuff is not covered by semver. Graduating an experimental new core module is semver-major, which means while it can land in backports as experimental it can never be considered non-experimental in downlevel releases.
... Adding a new module should be a minor change
We've never treated it as minor. Adding new features to existing modules has always been treated as minor.
jasnell
left a comment
There was a problem hiding this comment.
I'd prefer require('console/logging') or require('console').Logging rather than a new top-level node:... module.
|
|
||
| // Add consumer fields | ||
| const consumerFields = this.#fields; | ||
| for (const key of ObjectKeys(consumerFields)) { |
There was a problem hiding this comment.
should we validate that the fields are JSON-stringifiable types on construction to avoid errors during handle call?
There was a problem hiding this comment.
I initially added shallow type validation (rejecting bigint/function/symbol) but removed it because it only checks one level deep, { nested: { val: 42n } } would still throw at log time from JSON.stringify. Partial validation felt inconsistent.
I documented the behavior in the "Non-JSON-serializable values" section instead. Happy to add it back if you prefer the early check even if shallow.
|
@jasnell Thanks for the reviews. I edited and answered your reviews. |
Description
Adds an experimental
node:loggermodule that provides a structured, high-performance logging API for Node.js. Available behind the--experimental-loggerflag.Refs: #49296
Architecture
diagnostics_channelas the dispatch mechanism, loggers publish log records to level-specific channels, consumers subscribe to receive themnoopfunctions, eliminating runtime level checksUtf8Stream(SonicBoom port) for high-throughput writesAPI Surface
Logger, Producer class withtrace,debug,info,warn,error,fatalmethodsLogConsumer, Base consumer class for custom log handlingJSONConsumer, Built-in consumer that outputs structured JSON viaUtf8Streamchild(bindings, options), Child loggers with inherited context and serializerslogger.<level>.enabled, Getter for conditional logging (typo-safe)stdSerializers, Built-in serializers forerr,req,resserializesymbol,Symbol.for('nodejs.logger.serialize'), custom object serialization hook (stacked with field serializers)Log levels follow RFC 5424 numerical ordering internally (
trace: 10, debug: 20, info: 30, warn: 40, error: 50, fatal: 60). The API accepts string level names.diagnostics_channelchannel names (log:trace,log:debug,log:info,log:warn,log:error,log:fatal) are used internally and can be subscribed to directly viadiagnostics_channelif needed.Performance
Benchmarked against Pino (output to
/dev/null, n=100000):New files
lib/logger.js, Main module (Logger,LogConsumer,JSONConsumer)lib/internal/logger/serializers.js, Standarderr/req/resserializersdoc/api/logger.md, Full API documentationbenchmark/logger/basic-json.js, Benchmarkstest/parallel/test-log-basic.js, Core logger teststest/parallel/test-logger-serializers.js, Serializer testsModified files
src/node_options.h/src/node_options.cc,--experimental-loggerflaglib/internal/process/pre_execution.js,setupLogger()to allow module loading when flag is setlib/internal/bootstrap/realm.js, Module registrationdoc/api/cli.md, Flag documentationdoc/api/index.md/doc/node.1, Index updatestest/parallel/test-code-cache.js/test-process-get-builtin.mjs/test-require-resolve.js, Test adjustments for new module