[WFLY-11031] Add proposal for JSON access logging.

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[WFLY-11031] Add proposal for JSON access logging.

James Perkins
Hello All,
I've created a proposal [1] for JSON access logging. Any feedback is welcome.

[1]: https://github.com/wildfly/wildfly-proposals/pull/176

--
James R. Perkins
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [WFLY-11031] Add proposal for JSON access logging.

James Perkins
Hello All,
It's been longer than I'd want to really get into this, but now that I've got a start on it I'd like some feedback on the best way to deal with the pattern attribute. For the file it currently just uses a pattern like %h %l %u %t %r %s %b %{i,Referer} %{i,User-Agent}. The problem is that doesn't really provide a key for the value. What I'm currently using is that same pattern and using the ExchangeAttributeBuilder.getName() as the key. However if the value of getName() changes at some point the data would end up with a different key. This approach doesn't seem ideal.

One option would be to switch to some kind of property format type of thing like:

attributes={"remoteHost"="%h", "identityUsername"="%l"}

I'm not sure how user friendly that is though.

The other option would be to limit it to known exchange attributes, but that would require a change every time a new attribute is added to Undertow itself or we'd have to allow for some kind of custom attribute.

Any opinions are welcome. Ideally we want something user friendly that's going to produce consistent keys as that would be part of the structured data contract.

On Mon, Feb 18, 2019 at 8:10 AM James Perkins <[hidden email]> wrote:
Hello All,
I've created a proposal [1] for JSON access logging. Any feedback is welcome.

[1]: https://github.com/wildfly/wildfly-proposals/pull/176

--
James R. Perkins
JBoss by Red Hat


--
James R. Perkins
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [WFLY-11031] Add proposal for JSON access logging.

Brian Stansberry


On Thu, Apr 18, 2019 at 1:20 PM James Perkins <[hidden email]> wrote:
Hello All,
It's been longer than I'd want to really get into this, but now that I've got a start on it I'd like some feedback on the best way to deal with the pattern attribute. For the file it currently just uses a pattern like %h %l %u %t %r %s %b %{i,Referer} %{i,User-Agent}. The problem is that doesn't really provide a key for the value. What I'm currently using is that same pattern and using the ExchangeAttributeBuilder.getName() as the key. However if the value of getName() changes at some point the data would end up with a different key. This approach doesn't seem ideal.

One option would be to switch to some kind of property format type of thing like:

attributes={"remoteHost"="%h", "identityUsername"="%l"}

I'm not sure how user friendly that is though.

The other option would be to limit it to known exchange attributes, but that would require a change every time a new attribute is added to Undertow itself or we'd have to allow for some kind of custom attribute.

Any opinions are welcome. Ideally we want something user friendly that's going to produce consistent keys as that would be part of the structured data contract.

That pattern string also provides order. I suppose that's not as important with structured data though.

Perhaps two alternative attributes, one that's a pattern String and then one that's complex like your example?  The pattern string is documented with the values it supports and the names of the keys it will use for those values. (Those keys are set in the WF code, not pulled from ExchangeAttributeBuilder.) If you want other key names or other values you have to use the complex attribute.

That's kind of kludgy but it makes the hopefully common case simple.


On Mon, Feb 18, 2019 at 8:10 AM James Perkins <[hidden email]> wrote:
Hello All,
I've created a proposal [1] for JSON access logging. Any feedback is welcome.

[1]: https://github.com/wildfly/wildfly-proposals/pull/176

--
James R. Perkins
JBoss by Red Hat


--
James R. Perkins
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


--
Brian Stansberry
Manager, Senior Principal Software Engineer
Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [WFLY-11031] Add proposal for JSON access logging.

James Perkins


On Thu, Apr 18, 2019 at 12:41 PM Brian Stansberry <[hidden email]> wrote:


On Thu, Apr 18, 2019 at 1:20 PM James Perkins <[hidden email]> wrote:
Hello All,
It's been longer than I'd want to really get into this, but now that I've got a start on it I'd like some feedback on the best way to deal with the pattern attribute. For the file it currently just uses a pattern like %h %l %u %t %r %s %b %{i,Referer} %{i,User-Agent}. The problem is that doesn't really provide a key for the value. What I'm currently using is that same pattern and using the ExchangeAttributeBuilder.getName() as the key. However if the value of getName() changes at some point the data would end up with a different key. This approach doesn't seem ideal.

One option would be to switch to some kind of property format type of thing like:

attributes={"remoteHost"="%h", "identityUsername"="%l"}

I'm not sure how user friendly that is though.

The other option would be to limit it to known exchange attributes, but that would require a change every time a new attribute is added to Undertow itself or we'd have to allow for some kind of custom attribute.

Any opinions are welcome. Ideally we want something user friendly that's going to produce consistent keys as that would be part of the structured data contract.

That pattern string also provides order. I suppose that's not as important with structured data though.

Yeah order doesn't seem all that important, however I do my best to keep it in the order defined.
 

Perhaps two alternative attributes, one that's a pattern String and then one that's complex like your example?  The pattern string is documented with the values it supports and the names of the keys it will use for those values. (Those keys are set in the WF code, not pulled from ExchangeAttributeBuilder.) If you want other key names or other values you have to use the complex attribute.

My main concern there is just that there are currently 38 possible attribute builders in Undertow. I'm not sure if it's possible or not for a user to provide there own attributes, but if it is we'd have to find a way to allow for that as well. Having a limit may not necessarily be a bad thing though.

Another kind of oddity in the pattern I forgot to mention is it allows for constants which seems a bit weird to support in JSON. I was thinking of just add a "constants" array or some similar name.

I suppose another benefit to not using a pattern would be the parser wouldn't need to be copied from Undertow. However the ExchangeAttributeBuilder does include a priority that wouldn't be respected.

For one more kludgy option we could just use the pattern and allow for something like "remoteUser=%u" where remoteUser would be a key override and we just have a defined set of default keys. If it's an unknown attribute to WildFly, then a key is required. Again a bit kludgy I guess.
 

That's kind of kludgy but it makes the hopefully common case simple.


On Mon, Feb 18, 2019 at 8:10 AM James Perkins <[hidden email]> wrote:
Hello All,
I've created a proposal [1] for JSON access logging. Any feedback is welcome.

[1]: https://github.com/wildfly/wildfly-proposals/pull/176

--
James R. Perkins
JBoss by Red Hat


--
James R. Perkins
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


--
Brian Stansberry
Manager, Senior Principal Software Engineer
Red Hat


--
James R. Perkins
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [WFLY-11031] Add proposal for JSON access logging.

James Perkins
Hello All,
I've now got a draft PR open for this feature [1] and any reviews and feed back would be welcome. Please note I went with a complex "attributes" management model attribute instead of the standard pattern which the current access log uses.

Note too that this currently requires a SNAPSHOT build of WildFly Core which is why this is currently a draft.


On Thu, Apr 18, 2019 at 1:19 PM James Perkins <[hidden email]> wrote:


On Thu, Apr 18, 2019 at 12:41 PM Brian Stansberry <[hidden email]> wrote:


On Thu, Apr 18, 2019 at 1:20 PM James Perkins <[hidden email]> wrote:
Hello All,
It's been longer than I'd want to really get into this, but now that I've got a start on it I'd like some feedback on the best way to deal with the pattern attribute. For the file it currently just uses a pattern like %h %l %u %t %r %s %b %{i,Referer} %{i,User-Agent}. The problem is that doesn't really provide a key for the value. What I'm currently using is that same pattern and using the ExchangeAttributeBuilder.getName() as the key. However if the value of getName() changes at some point the data would end up with a different key. This approach doesn't seem ideal.

One option would be to switch to some kind of property format type of thing like:

attributes={"remoteHost"="%h", "identityUsername"="%l"}

I'm not sure how user friendly that is though.

The other option would be to limit it to known exchange attributes, but that would require a change every time a new attribute is added to Undertow itself or we'd have to allow for some kind of custom attribute.

Any opinions are welcome. Ideally we want something user friendly that's going to produce consistent keys as that would be part of the structured data contract.

That pattern string also provides order. I suppose that's not as important with structured data though.

Yeah order doesn't seem all that important, however I do my best to keep it in the order defined.
 

Perhaps two alternative attributes, one that's a pattern String and then one that's complex like your example?  The pattern string is documented with the values it supports and the names of the keys it will use for those values. (Those keys are set in the WF code, not pulled from ExchangeAttributeBuilder.) If you want other key names or other values you have to use the complex attribute.

My main concern there is just that there are currently 38 possible attribute builders in Undertow. I'm not sure if it's possible or not for a user to provide there own attributes, but if it is we'd have to find a way to allow for that as well. Having a limit may not necessarily be a bad thing though.

Another kind of oddity in the pattern I forgot to mention is it allows for constants which seems a bit weird to support in JSON. I was thinking of just add a "constants" array or some similar name.

I suppose another benefit to not using a pattern would be the parser wouldn't need to be copied from Undertow. However the ExchangeAttributeBuilder does include a priority that wouldn't be respected.

For one more kludgy option we could just use the pattern and allow for something like "remoteUser=%u" where remoteUser would be a key override and we just have a defined set of default keys. If it's an unknown attribute to WildFly, then a key is required. Again a bit kludgy I guess.
 

That's kind of kludgy but it makes the hopefully common case simple.


On Mon, Feb 18, 2019 at 8:10 AM James Perkins <[hidden email]> wrote:
Hello All,
I've created a proposal [1] for JSON access logging. Any feedback is welcome.

[1]: https://github.com/wildfly/wildfly-proposals/pull/176

--
James R. Perkins
JBoss by Red Hat


--
James R. Perkins
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


--
Brian Stansberry
Manager, Senior Principal Software Engineer
Red Hat


--
James R. Perkins
JBoss by Red Hat


--
James R. Perkins
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev