[wildfly-dev] Configuration of statistics gathering

classic Classic list List threaded Threaded
32 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[wildfly-dev] Configuration of statistics gathering

Brian Stansberry
I'd like to get some info from the component leads responsible for WF
subsystems.

1) What statistical data does your subsystem capture (including the
underlying libraries it integrates).

2) What configuration options do you support for enabling/disabling
statistics gathering. What's the resource address and attribute name of
the config option?

3) Is the statistic gathering enabled by default?

Paul Ferraro's pull request [1] fixing JIRA [2] prompts this question,
although it's long overdue. WF should be consistent about how we handle
configuration of statistics gathering, so I'd like to take the
opportunity of Paul's PR to move in that direction.

Thanks for your help.

--
Brian Stansberry
Principal Software Engineer
JBoss by Red Hat

[1] https://github.com/wildfly/wildfly/pull/5415
[2] https://issues.jboss.org/browse/WFLY-2415
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [wildfly-dev] Configuration of statistics gathering

kkhan
For JMX there are no statistics
On 6 Nov 2013, at 16:26, Brian Stansberry <[hidden email]> wrote:

> I'd like to get some info from the component leads responsible for WF
> subsystems.
>
> 1) What statistical data does your subsystem capture (including the
> underlying libraries it integrates).
>
> 2) What configuration options do you support for enabling/disabling
> statistics gathering. What's the resource address and attribute name of
> the config option?
>
> 3) Is the statistic gathering enabled by default?
>
> Paul Ferraro's pull request [1] fixing JIRA [2] prompts this question,
> although it's long overdue. WF should be consistent about how we handle
> configuration of statistics gathering, so I'd like to take the
> opportunity of Paul's PR to move in that direction.
>
> Thanks for your help.
>
> --
> Brian Stansberry
> Principal Software Engineer
> JBoss by Red Hat
>
> [1] https://github.com/wildfly/wildfly/pull/5415
> [2] https://issues.jboss.org/browse/WFLY-2415
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [wildfly-dev] Configuration of statistics gathering

Andrig Miller
Just to add to what Brian is saying.  I presented a proposal for runtime statistics at the AS 8 meeting (before the name change), and I would like to see all the subsystems have a configuration option for their runtime statistics.  I would also like it to be off by default.

The reason for the later, is our benchmarking efforts.  We have recently discovered two areas where the runtime statistics were causing problems with performance/scalability.  This is one area, hence Paul's pull request, and the other is the JCA container's statistics.

We may discover other areas too, as we continue scaling, but this is of vital importance for our efforts.

Thanks.

Andy


Brian Stansberry <[hidden email]> wrote:

>I'd like to get some info from the component leads responsible for WF
>subsystems.
>
>1) What statistical data does your subsystem capture (including the
>underlying libraries it integrates).
>
>2) What configuration options do you support for enabling/disabling
>statistics gathering. What's the resource address and attribute name of
>the config option?
>
>3) Is the statistic gathering enabled by default?
>
>Paul Ferraro's pull request [1] fixing JIRA [2] prompts this question,
>although it's long overdue. WF should be consistent about how we handle
>configuration of statistics gathering, so I'd like to take the
>opportunity of Paul's PR to move in that direction.
>
>Thanks for your help.
>
>--
>Brian Stansberry
>Principal Software Engineer
>JBoss by Red Hat
>
>[1] https://github.com/wildfly/wildfly/pull/5415
>[2] https://issues.jboss.org/browse/WFLY-2415
>_______________________________________________
>wildfly-dev mailing list
>[hidden email]
>https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [wildfly-dev] Configuration of statistics gathering

James Perkins
In reply to this post by Brian Stansberry
Logging does not have any statistics and batch does not at this point.
I'm not sure it will in the future, but I guess it doesn't really matter
at this point.

On 11/06/2013 08:26 AM, Brian Stansberry wrote:

> I'd like to get some info from the component leads responsible for WF
> subsystems.
>
> 1) What statistical data does your subsystem capture (including the
> underlying libraries it integrates).
>
> 2) What configuration options do you support for enabling/disabling
> statistics gathering. What's the resource address and attribute name of
> the config option?
>
> 3) Is the statistic gathering enabled by default?
>
> Paul Ferraro's pull request [1] fixing JIRA [2] prompts this question,
> although it's long overdue. WF should be consistent about how we handle
> configuration of statistics gathering, so I'd like to take the
> opportunity of Paul's PR to move in that direction.
>
> Thanks for your help.
>

--
James R. Perkins
Red Hat JBoss Middleware

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [wildfly-dev] Configuration of statistics gathering

Jeff Mesnil
In reply to this post by Brian Stansberry
> 1) What statistical data does your subsystem capture (including the
> underlying libraries it integrates).
> 2) What configuration options do you support for enabling/disabling
> statistics gathering. What's the resource address and attribute name of
> the config option?


For the messaging subsystem, we can capture message counters for queues[1].

Capturing the counters is controlled by the /subsystem=messaging/hornetq-server=* resources

* message-counter-enabled attribute (default to false) controls whether the hornetq-server resource capture the counters
* message-counter-sample-period attributecontrols the interval to capture the counters (default to 10s, only relevant if message-counter-enabled is true)
* message-counter-max-day-history attributes controls how many days to keep the message counter history (default to 10 days, only relevant if message-counter-enabled is true)
=> if message-counter-enabled is true, message-counter-sample-period is short, and message-counter-max-day-history is big, the counters can eat up a significant chunk of memory.

The hornetq-server resource has 2 operations to reset the counters and their histories:

* reset-all-message-counter-histories
* reset-all-message-counters

The counters themselves can be fetched from the queue and jms-queue resources (children of the /subsystem=messaging/hornetq-server=* resource) with the operations:

* list-message-counter-as-json
* list-message-counter-as-html
* list-message-counter-history-as-json
* list-message-counter-history-as-html

> 3) Is the statistic gathering enabled by default?


By default, nothing is enabled, nothing is gathered and there should be no performance or memory impact.

[1] http://docs.jboss.org/hornetq/2.4.0.beta1/docs/user-manual/html_single/index.html#management.message-counters

--
Jeff Mesnil
JBoss, a division of Red Hat
http://jmesnil.net/


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [wildfly-dev] Configuration of statistics gathering

Tomaž Cerar-2
We have plans to add support for statistics to undertow subsystem.
but for now we don't have anything.

Plan is to support as much as web subsystem did.
Statistics will have a switch to be enabled/disabled.

--
tomaz


On Wed, Nov 6, 2013 at 6:51 PM, Jeff Mesnil <[hidden email]> wrote:
> 1) What statistical data does your subsystem capture (including the
> underlying libraries it integrates).
> 2) What configuration options do you support for enabling/disabling
> statistics gathering. What's the resource address and attribute name of
> the config option?


For the messaging subsystem, we can capture message counters for queues[1].

Capturing the counters is controlled by the /subsystem=messaging/hornetq-server=* resources

* message-counter-enabled attribute (default to false) controls whether the hornetq-server resource capture the counters
* message-counter-sample-period attributecontrols the interval to capture the counters (default to 10s, only relevant if message-counter-enabled is true)
* message-counter-max-day-history attributes controls how many days to keep the message counter history (default to 10 days, only relevant if message-counter-enabled is true)
=> if message-counter-enabled is true, message-counter-sample-period is short, and message-counter-max-day-history is big, the counters can eat up a significant chunk of memory.

The hornetq-server resource has 2 operations to reset the counters and their histories:

* reset-all-message-counter-histories
* reset-all-message-counters

The counters themselves can be fetched from the queue and jms-queue resources (children of the /subsystem=messaging/hornetq-server=* resource) with the operations:

* list-message-counter-as-json
* list-message-counter-as-html
* list-message-counter-history-as-json
* list-message-counter-history-as-html

> 3) Is the statistic gathering enabled by default?


By default, nothing is enabled, nothing is gathered and there should be no performance or memory impact.

[1] http://docs.jboss.org/hornetq/2.4.0.beta1/docs/user-manual/html_single/index.html#management.message-counters

--
Jeff Mesnil
JBoss, a division of Red Hat
http://jmesnil.net/


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [wildfly-dev] Configuration of statistics gathering

Paul Ferraro
In reply to this post by Brian Stansberry
On Wed, 2013-11-06 at 10:26 -0600, Brian Stansberry wrote:
> I'd like to get some info from the component leads responsible for WF
> subsystems.
>
> 1) What statistical data does your subsystem capture (including the
> underlying libraries it integrates).

Infinispan collects a bazillion statistics [1] per cache and cache
container instance.

In JGroups, statistics are collected by each protocol in the protocol
stack.  I don't have a complete list, but, for example, the transport
protocols collect the following:
num_msgs_received
num_bytes_received
num_msgs_sent
num_bytes_sent

> 2) What configuration options do you support for enabling/disabling
> statistics gathering. What's the resource address and attribute name of
> the config option?

For Infinispan, [2] adds an option to enable/disable statistics per
cache and per cache-container.
e.g.
/subsystem=infinispan/cache-container=foo:read-attribute(name=statistics)
/subsystem=infinispan/cache-container=foo/local-cache=bar:read-attribute(name=statistics)
/subsystem=infinispan/cache-container=foo/invalidation-cache=bar:read-attribute(name=statistics)
/subsystem=infinispan/cache-container=foo/distributed-cache=bar:read-attribute(name=statistics)
/subsystem=infinispan/cache-container=foo/replicated-cache=bar:read-attribute(name=statistics)

For JGroups, a mechanism to disable statistics gathering is exposed, but
it's a bit cumbersome.  [3] seeks to fix this.
e.g.
/subsystem=jgroups/stack=udp/transport=TRANSPORT/property=stats:add(value=true)
/subsystem=jgroups/stack=udp/protocol=FOO/property=stats:add(value=true)

> 3) Is the statistic gathering enabled by default?

Infinispan, currently yes, but [2] will disable statistics by default.

JGroups, yes.

> Paul Ferraro's pull request [1] fixing JIRA [2] prompts this question,
> although it's long overdue. WF should be consistent about how we handle
> configuration of statistics gathering, so I'd like to take the
> opportunity of Paul's PR to move in that direction.
>
> Thanks for your help.
>

[1] http://docs.jboss.org/infinispan/6.0/apidocs/jmxComponents.html
[2] https://github.com/wildfly/wildfly/pull/5415
[2] https://issues.jboss.org/browse/WFLY-2453

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [wildfly-dev] Configuration of statistics gathering

Scott Marlow
In reply to this post by Brian Stansberry
On 11/06/2013 11:26 AM, Brian Stansberry wrote:
> I'd like to get some info from the component leads responsible for WF
> subsystems.
>
> 1) What statistical data does your subsystem capture (including the
> underlying libraries it integrates).

The JPA subsystem captures the Hibernate statistics, only if Hibernate
statistics are enabled (each persistence unit has its own statistics).

>
> 2) What configuration options do you support for enabling/disabling
> statistics gathering. What's the resource address and attribute name of
> the config option?

You can enable the statistics (off by default) with the following or you
can include a (application) persistence unit property to enable the
Hibernate statistics:

:write-attribute(name=enabled,value=true)

Address is
/deployment=DEPLOYMENTDeterminedAddress/subsystem=jpa/hibernate-persistence-unit=ApplicationScopedPersistenceUnitName

See http://fpaste.org/52218 for example of
/deployment=2lc.jar/subsystem=jpa/hibernate-persistence-unit=2lc.jar#TEST_PU

>
> 3) Is the statistic gathering enabled by default?

No, disabled by default.

>
> Paul Ferraro's pull request [1] fixing JIRA [2] prompts this question,
> although it's long overdue. WF should be consistent about how we handle
> configuration of statistics gathering, so I'd like to take the
> opportunity of Paul's PR to move in that direction.
>
> Thanks for your help.
>

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [wildfly-dev] Configuration of statistics gathering

Jozef Hartinger
In reply to this post by Brian Stansberry
The Weld subsystem does not gather any statistics.

On 11/06/2013 05:26 PM, Brian Stansberry wrote:

> I'd like to get some info from the component leads responsible for WF
> subsystems.
>
> 1) What statistical data does your subsystem capture (including the
> underlying libraries it integrates).
>
> 2) What configuration options do you support for enabling/disabling
> statistics gathering. What's the resource address and attribute name of
> the config option?
>
> 3) Is the statistic gathering enabled by default?
>
> Paul Ferraro's pull request [1] fixing JIRA [2] prompts this question,
> although it's long overdue. WF should be consistent about how we handle
> configuration of statistics gathering, so I'd like to take the
> opportunity of Paul's PR to move in that direction.
>
> Thanks for your help.
>

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Configuration of statistics gathering

Michael Musgrove
In reply to this post by Brian Stansberry
Apologies for the delay, the requested data is:

> I'd like to get some info from the component leads responsible for WF
> subsystems.
>
> 1) What statistical data does your subsystem capture (including the
> underlying libraries it integrates).

number-of-aborted-transactions
number-of-application-rollbacks
number-of-committed-transactions
number-of-heuristics
number-of-inflight-transactions
number-of-nested-transactions
number-of-resource-rollbacks
number-of-timed-out-transactions
number-of-transactions


2) What configuration options do you support for enabling/disabling
statistics gathering. What's the resource address and attribute name of
the config option?


/profile=default/subsystem=transactions/:write-attribute(name=enable-statistics,value=true)

>
> 3) Is the statistic gathering enabled by default?

No it is off by default

Mike
>
> Paul Ferraro's pull request [1] fixing JIRA [2] prompts this question,
> although it's long overdue. WF should be consistent about how we handle
> configuration of statistics gathering, so I'd like to take the
> opportunity of Paul's PR to move in that direction.
>
> Thanks for your help.
>


--
Michael Musgrove
Transactions Team
e: [hidden email]
t: +44 191 243 0870

Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham (US), Paul Hickey (Ireland), Matt Parson
(US), Charles Peters (US)

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Configuration of statistics gathering

Alessio Soldano
In reply to this post by Brian Stansberry
Hi Brian,

On 06/11/13 17:26, Brian Stansberry wrote:
> I'd like to get some info from the component leads responsible for WF
> subsystems.
>
> 1) What statistical data does your subsystem capture (including the
> underlying libraries it integrates).
WS endpoint metrics are collected by default: request-count,
response-count, fault-count, total-processing-time, max-processing-time,
min-processing-time, average-processing-time.
You can have a look at them in e.g.
/deployment=foo.war/subsystem=webservices/endpoint=fooWSService/

> 2) What configuration options do you support for enabling/disabling
> statistics gathering. What's the resource address and attribute name of
> the config option?
>
> 3) Is the statistic gathering enabled by default?
The metrics are enabled by default; I've just created
https://issues.jboss.org/browse/JBWS-3733 .

Cheers
Alessio

--
Alessio Soldano
Web Service Lead, JBoss

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Configuration of statistics gathering

Brian Stansberry
In reply to this post by Brian Stansberry
Thanks, everyone, for the input on this.

I'd like to do the following about getting configuration of these
consistent for WF 8.0:

1) For all resources that currently expose an attribute to
enable/disable statistics, I'll add an attribute that will be named
"statistics-enabled". Why that?

a) it's an attribute, so it's name should reflect a state, not an action
like, for example, "enable-statistics"
b) simply using "statistics" is shorter, but in some places that key is
used as a child type (e.g. statistics=pool) and a resource can't have a
child type and an attribute with the same name.

2) For compatibility, the existing attribute will be retained as an
alias to the new one. I'll add deprecation metadata. I'll also set up
the appropriate transformation stuff for mixed domain usage.

3) Any new stuff people do (e.g. [1], [2]), please use "statistics-enabled".

4) My intent is to make the default values for these attributes be
"false". I won't change any default though unless the relative compoent
lead has indicated agreement in this thread or elsewhere.

But, if you disagree with "false" as the default, please let's use this
thread to sort that out, get input from the performance team, etc, so we
can have at least a clear understanding of why some uses have 'true'.

Jesper, ^^^. ;-)


That's for WF 8. For the next release I recommend that we also create
subsystem-level attributes to hold the default value for the several
cases where this setting is only configurable on a per-deployment level.
An example would be the jpa subsystem could have a statistics-enabled
attribute that drives the setting for all persistence units unless the
PU descriptor overrides or the admin overrides using the existing
per-deployment attribute. Besides being an aid to usability, having an
attribute that ends up in the persistent configuration will also help
deal with issues where, for example, the desired default for WildFly is
different than the desired default for some future EAP release. The
standard configs for the EAP release can just use a different value.


[1] https://issues.jboss.org/browse/JBWS-3733
[2] https://issues.jboss.org/browse/WFLY-2453

On 11/6/13 10:26 AM, Brian Stansberry wrote:

> I'd like to get some info from the component leads responsible for WF
> subsystems.
>
> 1) What statistical data does your subsystem capture (including the
> underlying libraries it integrates).
>
> 2) What configuration options do you support for enabling/disabling
> statistics gathering. What's the resource address and attribute name of
> the config option?
>
> 3) Is the statistic gathering enabled by default?
>
> Paul Ferraro's pull request [1] fixing JIRA [2] prompts this question,
> although it's long overdue. WF should be consistent about how we handle
> configuration of statistics gathering, so I'd like to take the
> opportunity of Paul's PR to move in that direction.
>
> Thanks for your help.
>


--
Brian Stansberry
Principal Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Configuration of statistics gathering

Jesper Pedersen
On 12/12/2013 12:38 PM, Brian Stansberry wrote:
> 1) For all resources that currently expose an attribute to
> enable/disable statistics, I'll add an attribute that will be named
> "statistics-enabled". Why that?
>
> a) it's an attribute, so it's name should reflect a state, not an action
> like, for example, "enable-statistics"

Ok with that.

> b) simply using "statistics" is shorter, but in some places that key is
> used as a child type (e.g. statistics=pool) and a resource can't have a
> child type and an attribute with the same name.
>

Yeah, lets not change this.

> 2) For compatibility, the existing attribute will be retained as an
> alias to the new one. I'll add deprecation metadata. I'll also set up
> the appropriate transformation stuff for mixed domain usage.
>

Ok.

> 3) Any new stuff people do (e.g. [1], [2]), please use "statistics-enabled".
>

Agreed.

> 4) My intent is to make the default values for these attributes be
> "false". I won't change any default though unless the relative compoent
> lead has indicated agreement in this thread or elsewhere.
>
> But, if you disagree with "false" as the default, please let's use this
> thread to sort that out, get input from the performance team, etc, so we
> can have at least a clear understanding of why some uses have 'true'.
>

IronJacamar uses statistics internally in order to display run-time
information of the pools.

IronJacamar 1.1.2+ has a change which will allow to turn-off statistics,
but that results in nothing is displayed to the users in that case.

If we turn-off statistics by default we will have to be careful on what
to communicate to the users, since they will expect see basic
information out of the box. Maybe IronJacamar can display more
information than the core pool settings, but requires further investigation.

I think in order to turn it off we need proof that there is a huge
benefit for standard work loads. In the end we are talking about an
extra section in the guide that explains turning off statistics will
increase performance. Nothing new there. Or ship a "performance" profile.

Just so we are on the same page, the IronJacamar statistics is done by 2
x System.currentTimeMillis() + a call to Atomic(Long|Integer|...) - let
the flame war begin :) As always, patches welcome.

> Jesper, ^^^. ;-)
>

Why do you single me out, dude ;)

>
> That's for WF 8. For the next release I recommend that we also create
> subsystem-level attributes to hold the default value for the several
> cases where this setting is only configurable on a per-deployment level.
> An example would be the jpa subsystem could have a statistics-enabled
> attribute that drives the setting for all persistence units unless the
> PU descriptor overrides or the admin overrides using the existing
> per-deployment attribute. Besides being an aid to usability, having an
> attribute that ends up in the persistent configuration will also help
> deal with issues where, for example, the desired default for WildFly is
> different than the desired default for some future EAP release. The
> standard configs for the EAP release can just use a different value.
>
>

Sounds good, although having it as an option in each deployment
descriptor for each component type maybe overkill. I think an option in
wildfly.xml would be enough, and then override through
statistics-enabled via CLI.

Best regards,
  Jesper

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Configuration of statistics gathering

Brian Stansberry
On 12/12/13 12:03 PM, Jesper Pedersen wrote:
> On 12/12/2013 12:38 PM, Brian Stansberry wrote:

>> 4) My intent is to make the default values for these attributes be
>> "false". I won't change any default though unless the relative compoent
>> lead has indicated agreement in this thread or elsewhere.
>>
>> But, if you disagree with "false" as the default, please let's use this
>> thread to sort that out, get input from the performance team, etc, so we
>> can have at least a clear understanding of why some uses have 'true'.
>>
>
> IronJacamar uses statistics internally in order to display run-time
> information of the pools.
>
> IronJacamar 1.1.2+ has a change which will allow to turn-off statistics,
> but that results in nothing is displayed to the users in that case.
>
> If we turn-off statistics by default we will have to be careful on what
> to communicate to the users, since they will expect see basic
> information out of the box. Maybe IronJacamar can display more
> information than the core pool settings, but requires further investigation.
>
> I think in order to turn it off we need proof that there is a huge
> benefit for standard work loads. In the end we are talking about an
> extra section in the guide that explains turning off statistics will
> increase performance. Nothing new there. Or ship a "performance" profile.
>
> Just so we are on the same page, the IronJacamar statistics is done by 2
> x System.currentTimeMillis() + a call to Atomic(Long|Integer|...) - let
> the flame war begin :) As always, patches welcome.
>
>> Jesper, ^^^. ;-)
>>
>
> Why do you single me out, dude ;)
>

You're the guy who argued for 'true'!

>>
>> That's for WF 8. For the next release I recommend that we also create
>> subsystem-level attributes to hold the default value for the several
>> cases where this setting is only configurable on a per-deployment level.
>> An example would be the jpa subsystem could have a statistics-enabled
>> attribute that drives the setting for all persistence units unless the
>> PU descriptor overrides or the admin overrides using the existing
>> per-deployment attribute. Besides being an aid to usability, having an
>> attribute that ends up in the persistent configuration will also help
>> deal with issues where, for example, the desired default for WildFly is
>> different than the desired default for some future EAP release. The
>> standard configs for the EAP release can just use a different value.
>>
>>
>
> Sounds good, although having it as an option in each deployment
> descriptor for each component type maybe overkill. I think an option in
> wildfly.xml would be enough, and then override through
> statistics-enabled via CLI.
>

My previous post wasn't meant to propose making this configurable in
deployment descriptors (although maybe that's a good idea; I haven't
really thought about it.)  I mentioned the PU descriptor just because I
was using JPA as an example and IIRC config in the DD is part of that
particular example. Hibernate lets the user enable/disable statistics
via a property in persistence.xml.


--
Brian Stansberry
Principal Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Configuration of statistics gathering

Jesper Pedersen
On 12/12/2013 01:26 PM, Brian Stansberry wrote:
>> Why do you single me out, dude ;)
>
> You're the guy who argued for 'true'!
>

I know :)

>> Sounds good, although having it as an option in each deployment
>> descriptor for each component type maybe overkill. I think an option in
>> wildfly.xml would be enough, and then override through
>> statistics-enabled via CLI.
>>
>
> My previous post wasn't meant to propose making this configurable in
> deployment descriptors (although maybe that's a good idea; I haven't
> really thought about it.)  I mentioned the PU descriptor just because I
> was using JPA as an example and IIRC config in the DD is part of that
> particular example. Hibernate lets the user enable/disable statistics
> via a property in persistence.xml.
>

Sure, if it can be done in standard deployment descriptors I have no
problem with that. But doing it in all sub-projects, and related
subsystems, is a different kettle of fish.

Overall (wildfly.xml) + run-time management commands should cover most
use-cases. People will enable/disable during the deployment life time
anyway.

Best regards,
  Jesper

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Configuration of statistics gathering

Andrig Miller
In reply to this post by Jesper Pedersen


----- Original Message -----

> From: "Jesper Pedersen" <[hidden email]>
> To: [hidden email]
> Sent: Thursday, December 12, 2013 11:03:54 AM
> Subject: Re: [wildfly-dev] Configuration of statistics gathering
>
> On 12/12/2013 12:38 PM, Brian Stansberry wrote:
> > 1) For all resources that currently expose an attribute to
> > enable/disable statistics, I'll add an attribute that will be named
> > "statistics-enabled". Why that?
> >
> > a) it's an attribute, so it's name should reflect a state, not an
> > action
> > like, for example, "enable-statistics"
>
> Ok with that.
>
> > b) simply using "statistics" is shorter, but in some places that
> > key is
> > used as a child type (e.g. statistics=pool) and a resource can't
> > have a
> > child type and an attribute with the same name.
> >
>
> Yeah, lets not change this.
>
> > 2) For compatibility, the existing attribute will be retained as an
> > alias to the new one. I'll add deprecation metadata. I'll also set
> > up
> > the appropriate transformation stuff for mixed domain usage.
> >
>
> Ok.
>
> > 3) Any new stuff people do (e.g. [1], [2]), please use
> > "statistics-enabled".
> >
>
> Agreed.
>
> > 4) My intent is to make the default values for these attributes be
> > "false". I won't change any default though unless the relative
> > compoent
> > lead has indicated agreement in this thread or elsewhere.
> >
> > But, if you disagree with "false" as the default, please let's use
> > this
> > thread to sort that out, get input from the performance team, etc,
> > so we
> > can have at least a clear understanding of why some uses have
> > 'true'.
> >
>
> IronJacamar uses statistics internally in order to display run-time
> information of the pools.
>
> IronJacamar 1.1.2+ has a change which will allow to turn-off
> statistics,
> but that results in nothing is displayed to the users in that case.
>
> If we turn-off statistics by default we will have to be careful on
> what
> to communicate to the users, since they will expect see basic
> information out of the box. Maybe IronJacamar can display more
> information than the core pool settings, but requires further
> investigation.
>
> I think in order to turn it off we need proof that there is a huge
> benefit for standard work loads. In the end we are talking about an
> extra section in the guide that explains turning off statistics will
> increase performance. Nothing new there. Or ship a "performance"
> profile.
>

There is nothing special about the workload that drove this change, in terms of how its using IJ.  So, the change benefits any application that uses a data source.

I'm not really sure why any additional proof is required.  If somehow the workload was special in some way, then I could see that, but its not.  Its an EE 5 based application using EJB 3 entities with container managed persistence mapped to data sources.

By definition, if that application improves its response times signficantly, which it did, then any application will experience similar results as its scaled.  You probably wouldn't be able to measure it on an application with one user, but as you scale up the user count, and concurrency increases, the larger the impact.

Andy

> Just so we are on the same page, the IronJacamar statistics is done
> by 2
> x System.currentTimeMillis() + a call to Atomic(Long|Integer|...) -
> let
> the flame war begin :) As always, patches welcome.
>
> > Jesper, ^^^. ;-)
> >
>
> Why do you single me out, dude ;)
>
> >
> > That's for WF 8. For the next release I recommend that we also
> > create
> > subsystem-level attributes to hold the default value for the
> > several
> > cases where this setting is only configurable on a per-deployment
> > level.
> > An example would be the jpa subsystem could have a
> > statistics-enabled
> > attribute that drives the setting for all persistence units unless
> > the
> > PU descriptor overrides or the admin overrides using the existing
> > per-deployment attribute. Besides being an aid to usability, having
> > an
> > attribute that ends up in the persistent configuration will also
> > help
> > deal with issues where, for example, the desired default for
> > WildFly is
> > different than the desired default for some future EAP release. The
> > standard configs for the EAP release can just use a different
> > value.
> >
> >
>
> Sounds good, although having it as an option in each deployment
> descriptor for each component type maybe overkill. I think an option
> in
> wildfly.xml would be enough, and then override through
> statistics-enabled via CLI.
>
> Best regards,
>   Jesper
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Configuration of statistics gathering

Scott Marlow
In reply to this post by Brian Stansberry
On 12/12/2013 12:38 PM, Brian Stansberry wrote:

> Thanks, everyone, for the input on this.
>
> I'd like to do the following about getting configuration of these
> consistent for WF 8.0:
>
> 1) For all resources that currently expose an attribute to
> enable/disable statistics, I'll add an attribute that will be named
> "statistics-enabled". Why that?
>
> a) it's an attribute, so it's name should reflect a state, not an action
> like, for example, "enable-statistics"
> b) simply using "statistics" is shorter, but in some places that key is
> used as a child type (e.g. statistics=pool) and a resource can't have a
> child type and an attribute with the same name.
>
> 2) For compatibility, the existing attribute will be retained as an
> alias to the new one. I'll add deprecation metadata. I'll also set up
> the appropriate transformation stuff for mixed domain usage.
>
> 3) Any new stuff people do (e.g. [1], [2]), please use "statistics-enabled".
>
> 4) My intent is to make the default values for these attributes be
> "false". I won't change any default though unless the relative compoent
> lead has indicated agreement in this thread or elsewhere.
>
> But, if you disagree with "false" as the default, please let's use this
> thread to sort that out, get input from the performance team, etc, so we
> can have at least a clear understanding of why some uses have 'true'.
>
> Jesper, ^^^. ;-)
>
>
> That's for WF 8. For the next release I recommend that we also create
> subsystem-level attributes to hold the default value for the several
> cases where this setting is only configurable on a per-deployment level.
> An example would be the jpa subsystem could have a statistics-enabled
> attribute that drives the setting for all persistence units unless the
> PU descriptor overrides or the admin overrides using the existing
> per-deployment attribute. Besides being an aid to usability, having an
> attribute that ends up in the persistent configuration will also help
> deal with issues where, for example, the desired default for WildFly is
> different than the desired default for some future EAP release. The
> standard configs for the EAP release can just use a different value.

Should the deployment level settings always take precedence?  If yes,
then deployments that specify a statistics setting (e.g. persistence.xml
includes a hint to set statistics to some state), are not be controlled
by the subsystem level setting.  Having deployment settings override the
subsystem settings sounds fine to me but I wanted to point out that we
could let the subsystem settings override the deployment settings.

I wonder if "statistics-enabled" should have three states, so that the
subsystem level settings always overrides the deployment level?  For the
JPA case (with Hibernate):

true - statistics are enabled (even if the persistence.xml includes a
hint to disable statistics).

false - statistics are disabled (even if the persistence.xml includes a
hint to enable statistics).

ignore - statistics are determined by property hints in the persistence
unit (in persistence.xml).

IMO, I like having the deployment settings override the subsystem level
settings.  I would find it frustrating if setting a deployment level
setting is ignored (one of the challenges of allowing multiple levels to
set the same property).

>
>
> [1] https://issues.jboss.org/browse/JBWS-3733
> [2] https://issues.jboss.org/browse/WFLY-2453
>
> On 11/6/13 10:26 AM, Brian Stansberry wrote:
>> I'd like to get some info from the component leads responsible for WF
>> subsystems.
>>
>> 1) What statistical data does your subsystem capture (including the
>> underlying libraries it integrates).
>>
>> 2) What configuration options do you support for enabling/disabling
>> statistics gathering. What's the resource address and attribute name of
>> the config option?
>>
>> 3) Is the statistic gathering enabled by default?
>>
>> Paul Ferraro's pull request [1] fixing JIRA [2] prompts this question,
>> although it's long overdue. WF should be consistent about how we handle
>> configuration of statistics gathering, so I'd like to take the
>> opportunity of Paul's PR to move in that direction.
>>
>> Thanks for your help.
>>
>
>

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Configuration of statistics gathering

Brian Stansberry
On 12/12/13 3:15 PM, Scott Marlow wrote:

> On 12/12/2013 12:38 PM, Brian Stansberry wrote:
>> Thanks, everyone, for the input on this.
>>
>> I'd like to do the following about getting configuration of these
>> consistent for WF 8.0:
>>
>> 1) For all resources that currently expose an attribute to
>> enable/disable statistics, I'll add an attribute that will be named
>> "statistics-enabled". Why that?
>>
>> a) it's an attribute, so it's name should reflect a state, not an action
>> like, for example, "enable-statistics"
>> b) simply using "statistics" is shorter, but in some places that key is
>> used as a child type (e.g. statistics=pool) and a resource can't have a
>> child type and an attribute with the same name.
>>
>> 2) For compatibility, the existing attribute will be retained as an
>> alias to the new one. I'll add deprecation metadata. I'll also set up
>> the appropriate transformation stuff for mixed domain usage.
>>
>> 3) Any new stuff people do (e.g. [1], [2]), please use
>> "statistics-enabled".
>>
>> 4) My intent is to make the default values for these attributes be
>> "false". I won't change any default though unless the relative compoent
>> lead has indicated agreement in this thread or elsewhere.
>>
>> But, if you disagree with "false" as the default, please let's use this
>> thread to sort that out, get input from the performance team, etc, so we
>> can have at least a clear understanding of why some uses have 'true'.
>>
>> Jesper, ^^^. ;-)
>>
>>
>> That's for WF 8. For the next release I recommend that we also create
>> subsystem-level attributes to hold the default value for the several
>> cases where this setting is only configurable on a per-deployment level.
>> An example would be the jpa subsystem could have a statistics-enabled
>> attribute that drives the setting for all persistence units unless the
>> PU descriptor overrides or the admin overrides using the existing
>> per-deployment attribute. Besides being an aid to usability, having an
>> attribute that ends up in the persistent configuration will also help
>> deal with issues where, for example, the desired default for WildFly is
>> different than the desired default for some future EAP release. The
>> standard configs for the EAP release can just use a different value.
>
> Should the deployment level settings always take precedence?  If yes,
> then deployments that specify a statistics setting (e.g. persistence.xml
> includes a hint to set statistics to some state), are not be controlled
> by the subsystem level setting.  Having deployment settings override the
> subsystem settings sounds fine to me but I wanted to point out that we
> could let the subsystem settings override the deployment settings.
>

True.

> I wonder if "statistics-enabled" should have three states, so that the
> subsystem level settings always overrides the deployment level?  For the
> JPA case (with Hibernate):
>
> true - statistics are enabled (even if the persistence.xml includes a
> hint to disable statistics).
>
> false - statistics are disabled (even if the persistence.xml includes a
> hint to enable statistics).
>
> ignore - statistics are determined by property hints in the persistence
> unit (in persistence.xml).
>

In the end there would have to be a default though. That is, if there is
no deployment-level setting, "ignore" would have to be statically
defined as meaning true or false.

> IMO, I like having the deployment settings override the subsystem level
> settings.  I would find it frustrating if setting a deployment level
> setting is ignored (one of the challenges of allowing multiple levels to
> set the same property).
>

I prefer deployments overriding too. I think the opposite would be
pretty unintiutive, and requires complex stuff like the "ignore" value.
In general, things work in our management on a
more-specific-overrides-less-specific basis (e.g. system property
settings at various levels in the domain/host config) and I think it's
good to be consistent about that. Also, we have another mechanism for
allowing admins to override the settings in their deployments:
deployment overlays.

Do we have other cases where subsystem configs override some aspect of
deployment config?

Side note on JPA: a subsystem-level setting presumes that the different
providers have some mechanism for enabling/disabling statistics. If they
don't a subsystem level setting becomes meaningless. Which isn't the end
of the world, since our standard provider support this.


>>
>>
>> [1] https://issues.jboss.org/browse/JBWS-3733
>> [2] https://issues.jboss.org/browse/WFLY-2453
>>
>> On 11/6/13 10:26 AM, Brian Stansberry wrote:
>>> I'd like to get some info from the component leads responsible for WF
>>> subsystems.
>>>
>>> 1) What statistical data does your subsystem capture (including the
>>> underlying libraries it integrates).
>>>
>>> 2) What configuration options do you support for enabling/disabling
>>> statistics gathering. What's the resource address and attribute name of
>>> the config option?
>>>
>>> 3) Is the statistic gathering enabled by default?
>>>
>>> Paul Ferraro's pull request [1] fixing JIRA [2] prompts this question,
>>> although it's long overdue. WF should be consistent about how we handle
>>> configuration of statistics gathering, so I'd like to take the
>>> opportunity of Paul's PR to move in that direction.
>>>
>>> Thanks for your help.
>>>
>>
>>
>


--
Brian Stansberry
Principal Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Configuration of statistics gathering

Sanne Grinovero
In reply to this post by Jesper Pedersen
On 12 December 2013 18:03, Jesper Pedersen <[hidden email]> wrote:

> On 12/12/2013 12:38 PM, Brian Stansberry wrote:
>> 1) For all resources that currently expose an attribute to
>> enable/disable statistics, I'll add an attribute that will be named
>> "statistics-enabled". Why that?
>>
>> a) it's an attribute, so it's name should reflect a state, not an action
>> like, for example, "enable-statistics"
>
> Ok with that.
>
>> b) simply using "statistics" is shorter, but in some places that key is
>> used as a child type (e.g. statistics=pool) and a resource can't have a
>> child type and an attribute with the same name.
>>
>
> Yeah, lets not change this.
>
>> 2) For compatibility, the existing attribute will be retained as an
>> alias to the new one. I'll add deprecation metadata. I'll also set up
>> the appropriate transformation stuff for mixed domain usage.
>>
>
> Ok.
>
>> 3) Any new stuff people do (e.g. [1], [2]), please use "statistics-enabled".
>>
>
> Agreed.
>
>> 4) My intent is to make the default values for these attributes be
>> "false". I won't change any default though unless the relative compoent
>> lead has indicated agreement in this thread or elsewhere.
>>
>> But, if you disagree with "false" as the default, please let's use this
>> thread to sort that out, get input from the performance team, etc, so we
>> can have at least a clear understanding of why some uses have 'true'.
>>
>
> IronJacamar uses statistics internally in order to display run-time
> information of the pools.
>
> IronJacamar 1.1.2+ has a change which will allow to turn-off statistics,
> but that results in nothing is displayed to the users in that case.
>
> If we turn-off statistics by default we will have to be careful on what
> to communicate to the users, since they will expect see basic
> information out of the box. Maybe IronJacamar can display more
> information than the core pool settings, but requires further investigation.
>
> I think in order to turn it off we need proof that there is a huge
> benefit for standard work loads. In the end we are talking about an
> extra section in the guide that explains turning off statistics will
> increase performance. Nothing new there. Or ship a "performance" profile.
>
> Just so we are on the same page, the IronJacamar statistics is done by 2
> x System.currentTimeMillis() + a call to Atomic(Long|Integer|...) - let
> the flame war begin :) As always, patches welcome.

The following blog post highly recommends using nanoTime() rather than
currentTimeMillis() when it comes to measuring time intervals:

   https://blogs.oracle.com/dholmes/entry/inside_the_hotspot_vm_clocks

In the case of Infinispan, the contention on the AtomicLong(s) was
actually significant, we resorted using a LongAdder, backported from
Java8.

Still I doubt the contention was being caused by an insanely high
throughput: Infinispan might be fast but isn't that fast, so we
suspect the fix is being effective not because of the lower contention
on the CAS operation but more likely it has helped preventing false
sharing, as several statistics where being recorded as different
Atomic fields in the same class.

Best Regards,
Sanne


>
>> Jesper, ^^^. ;-)
>>
>
> Why do you single me out, dude ;)
>
>>
>> That's for WF 8. For the next release I recommend that we also create
>> subsystem-level attributes to hold the default value for the several
>> cases where this setting is only configurable on a per-deployment level.
>> An example would be the jpa subsystem could have a statistics-enabled
>> attribute that drives the setting for all persistence units unless the
>> PU descriptor overrides or the admin overrides using the existing
>> per-deployment attribute. Besides being an aid to usability, having an
>> attribute that ends up in the persistent configuration will also help
>> deal with issues where, for example, the desired default for WildFly is
>> different than the desired default for some future EAP release. The
>> standard configs for the EAP release can just use a different value.
>>
>>
>
> Sounds good, although having it as an option in each deployment
> descriptor for each component type maybe overkill. I think an option in
> wildfly.xml would be enough, and then override through
> statistics-enabled via CLI.
>
> Best regards,
>   Jesper
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Configuration of statistics gathering

Brian Stansberry
In reply to this post by Brian Stansberry
I've sent in a PR [1] that makes the changes I described for the
messaging, transactions, resource-adapters and infinispan subsystems,
one commit per subsystems. If the respective leads could have a look I'd
appreciate it.

Those are the WF subsystems that have a specific attribute for this. JPA
has an attribute, but it's part of jipijapa, so that will need to be
corrected there [2]. JGroups has a particular property key for this
instead of an attribute, and there is already a JIRA for updating that [3].

[1] https://github.com/wildfly/wildfly/pull/5609
[2] https://issues.jboss.org/browse/JIPI-28
[3] https://issues.jboss.org/browse/WFLY-2453

On 12/12/13 11:38 AM, Brian Stansberry wrote:

> Thanks, everyone, for the input on this.
>
> I'd like to do the following about getting configuration of these
> consistent for WF 8.0:
>
> 1) For all resources that currently expose an attribute to
> enable/disable statistics, I'll add an attribute that will be named
> "statistics-enabled". Why that?
>
> a) it's an attribute, so it's name should reflect a state, not an action
> like, for example, "enable-statistics"
> b) simply using "statistics" is shorter, but in some places that key is
> used as a child type (e.g. statistics=pool) and a resource can't have a
> child type and an attribute with the same name.
>
> 2) For compatibility, the existing attribute will be retained as an
> alias to the new one. I'll add deprecation metadata. I'll also set up
> the appropriate transformation stuff for mixed domain usage.
>
> 3) Any new stuff people do (e.g. [1], [2]), please use "statistics-enabled".
>
> 4) My intent is to make the default values for these attributes be
> "false". I won't change any default though unless the relative compoent
> lead has indicated agreement in this thread or elsewhere.
>
> But, if you disagree with "false" as the default, please let's use this
> thread to sort that out, get input from the performance team, etc, so we
> can have at least a clear understanding of why some uses have 'true'.
>
> Jesper, ^^^. ;-)
>
>
> That's for WF 8. For the next release I recommend that we also create
> subsystem-level attributes to hold the default value for the several
> cases where this setting is only configurable on a per-deployment level.
> An example would be the jpa subsystem could have a statistics-enabled
> attribute that drives the setting for all persistence units unless the
> PU descriptor overrides or the admin overrides using the existing
> per-deployment attribute. Besides being an aid to usability, having an
> attribute that ends up in the persistent configuration will also help
> deal with issues where, for example, the desired default for WildFly is
> different than the desired default for some future EAP release. The
> standard configs for the EAP release can just use a different value.
>
>
> [1] https://issues.jboss.org/browse/JBWS-3733
> [2] https://issues.jboss.org/browse/WFLY-2453
>
> On 11/6/13 10:26 AM, Brian Stansberry wrote:
>> I'd like to get some info from the component leads responsible for WF
>> subsystems.
>>
>> 1) What statistical data does your subsystem capture (including the
>> underlying libraries it integrates).
>>
>> 2) What configuration options do you support for enabling/disabling
>> statistics gathering. What's the resource address and attribute name of
>> the config option?
>>
>> 3) Is the statistic gathering enabled by default?
>>
>> Paul Ferraro's pull request [1] fixing JIRA [2] prompts this question,
>> although it's long overdue. WF should be consistent about how we handle
>> configuration of statistics gathering, so I'd like to take the
>> opportunity of Paul's PR to move in that direction.
>>
>> Thanks for your help.
>>
>
>


--
Brian Stansberry
Principal Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
12