Introducing a new thread pool implementation

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Introducing a new thread pool implementation

David Lloyd-2
For WildFly 12 I intend to introduce a new thread pool implementation
[1] [2] that will replace most, if not all, uses of ThreadPoolExecutor
in the main codebases of WildFly and WildFly Core, and also in several
support projects such as JBoss MSC and XNIO.

The thread pool has the following desirable characteristics:

• Idle threads are always reused before queueing tasks or starting new threads
• The most recently used idle thread is always preferred, which allows
the pool to shrink when it is not fully utilized
• Useful core/max thread pool size distinction
• API-equivalent to ThreadPoolExecutor
• Performance parity with common ThreadPoolExecutor configurations
• Suitable as a replacement for other jboss-threads thread pool types as well

In addition, it presents the following features:

• Configure via a builder API
• Configurable exception handler
• Integrated task queue with optional size limit
• Full complement of standard metrics, which can be disabled
• Configurable rejection handler in the form of a handoff executor
• An optional growth-resistance algorithm which can be applied to
larger pools to create growth resistance between the core and maximum
pool size
• An optional post-terminate task
• A few system property based adjustments that can be applied for
experimental purposes
• A global flag which can be set via system property to recommend that
the new thread pool should be disabled, allowing an emergency fallback
in case of an unexpected problem

Design-wise, the thread pool is based on a special lock-free/wait-free
combination FIFO/LIFO queue algorithm.  See [3] for a more complete
explanation of internal operation, if you're curious about the gritty
details.

This change has a few implications with regards to the application server:
• Thread pools with a core- and max-size which were locked together
may now have these variables decoupled in a safe and useful manner
• Thread pool configurations in the management model which previously
featured only a max size can now have a core size attribute introduced

Most of the thread pools we use are built and configured within the
wildfly-core code base; that is where the first part of this change
will be done.  Later on there will be a follow-up change for WildFly
proper, but this of course can not happen until the core change is
merged and a core release done, so expect that part somewhat later.
The initial WIP change to wildfly-core can be found here [4].  Note
that no pull request will be opened until all the components are
Final.  I have done a good deal of testing already, but I will be
doing more CI testing on other platforms and configurations before
submitting the PR.

I've made an attempt to link up all the JIRAs that are related to this
effort from [1] (either directly or indirectly).  If you find another
JIRA that you think is related, or if you have questions or concerns,
ask on this list, or you can ping me directly on HipChat or IRC.

Thanks!

[1] https://issues.jboss.org/browse/JBTHR-38 - Introduce new thread
pool implementation
[2] https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128
[3] https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128#diff-f5807a689506af0b70791d93ae81cc1cR67
[4] https://github.com/dmlloyd/wildfly-core/tree/threadpool

--
- DML

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Introducing a new thread pool implementation

Andrig Miller
I think it would be wise for the performance team to test this out at scale using our lab, as I'm sure you would agree.

If you could point us at a particular build, or point us at what to build, we can do that testing in parallel and give feedback on how this is working.

Thanks.

Andy

On Fri, Nov 3, 2017 at 8:03 AM, David Lloyd <[hidden email]> wrote:
For WildFly 12 I intend to introduce a new thread pool implementation
[1] [2] that will replace most, if not all, uses of ThreadPoolExecutor
in the main codebases of WildFly and WildFly Core, and also in several
support projects such as JBoss MSC and XNIO.

The thread pool has the following desirable characteristics:

• Idle threads are always reused before queueing tasks or starting new threads
• The most recently used idle thread is always preferred, which allows
the pool to shrink when it is not fully utilized
• Useful core/max thread pool size distinction
• API-equivalent to ThreadPoolExecutor
• Performance parity with common ThreadPoolExecutor configurations
• Suitable as a replacement for other jboss-threads thread pool types as well

In addition, it presents the following features:

• Configure via a builder API
• Configurable exception handler
• Integrated task queue with optional size limit
• Full complement of standard metrics, which can be disabled
• Configurable rejection handler in the form of a handoff executor
• An optional growth-resistance algorithm which can be applied to
larger pools to create growth resistance between the core and maximum
pool size
• An optional post-terminate task
• A few system property based adjustments that can be applied for
experimental purposes
• A global flag which can be set via system property to recommend that
the new thread pool should be disabled, allowing an emergency fallback
in case of an unexpected problem

Design-wise, the thread pool is based on a special lock-free/wait-free
combination FIFO/LIFO queue algorithm.  See [3] for a more complete
explanation of internal operation, if you're curious about the gritty
details.

This change has a few implications with regards to the application server:
• Thread pools with a core- and max-size which were locked together
may now have these variables decoupled in a safe and useful manner
• Thread pool configurations in the management model which previously
featured only a max size can now have a core size attribute introduced

Most of the thread pools we use are built and configured within the
wildfly-core code base; that is where the first part of this change
will be done.  Later on there will be a follow-up change for WildFly
proper, but this of course can not happen until the core change is
merged and a core release done, so expect that part somewhat later.
The initial WIP change to wildfly-core can be found here [4].  Note
that no pull request will be opened until all the components are
Final.  I have done a good deal of testing already, but I will be
doing more CI testing on other platforms and configurations before
submitting the PR.

I've made an attempt to link up all the JIRAs that are related to this
effort from [1] (either directly or indirectly).  If you find another
JIRA that you think is related, or if you have questions or concerns,
ask on this list, or you can ping me directly on HipChat or IRC.

Thanks!

[1] https://issues.jboss.org/browse/JBTHR-38 - Introduce new thread
pool implementation
[2] https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128
[3] https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128#diff-f5807a689506af0b70791d93ae81cc1cR67
[4] https://github.com/dmlloyd/wildfly-core/tree/threadpool

--
- DML

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev



--
Andrig (Andy) T. Miller
Global Platform Director, Middleware
Red Hat, Inc.

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Introducing a new thread pool implementation

David Lloyd-2
The proposed change can be found here:
https://github.com/dmlloyd/wildfly-core/tree/threadpool

You can build that wildfly-core and then build a wildfly which uses
the produced build.  The whole process with -DskipTests should take
under 10 minutes.  I would be very happy to know what the performance
team finds, particularly in high-contention scenarios.  Thanks!

On Fri, Nov 3, 2017 at 10:17 AM, Andrig Miller <[hidden email]> wrote:

> I think it would be wise for the performance team to test this out at scale
> using our lab, as I'm sure you would agree.
>
> If you could point us at a particular build, or point us at what to build,
> we can do that testing in parallel and give feedback on how this is working.
>
> Thanks.
>
> Andy
>
> On Fri, Nov 3, 2017 at 8:03 AM, David Lloyd <[hidden email]> wrote:
>>
>> For WildFly 12 I intend to introduce a new thread pool implementation
>> [1] [2] that will replace most, if not all, uses of ThreadPoolExecutor
>> in the main codebases of WildFly and WildFly Core, and also in several
>> support projects such as JBoss MSC and XNIO.
>>
>> The thread pool has the following desirable characteristics:
>>
>> • Idle threads are always reused before queueing tasks or starting new
>> threads
>> • The most recently used idle thread is always preferred, which allows
>> the pool to shrink when it is not fully utilized
>> • Useful core/max thread pool size distinction
>> • API-equivalent to ThreadPoolExecutor
>> • Performance parity with common ThreadPoolExecutor configurations
>> • Suitable as a replacement for other jboss-threads thread pool types as
>> well
>>
>> In addition, it presents the following features:
>>
>> • Configure via a builder API
>> • Configurable exception handler
>> • Integrated task queue with optional size limit
>> • Full complement of standard metrics, which can be disabled
>> • Configurable rejection handler in the form of a handoff executor
>> • An optional growth-resistance algorithm which can be applied to
>> larger pools to create growth resistance between the core and maximum
>> pool size
>> • An optional post-terminate task
>> • A few system property based adjustments that can be applied for
>> experimental purposes
>> • A global flag which can be set via system property to recommend that
>> the new thread pool should be disabled, allowing an emergency fallback
>> in case of an unexpected problem
>>
>> Design-wise, the thread pool is based on a special lock-free/wait-free
>> combination FIFO/LIFO queue algorithm.  See [3] for a more complete
>> explanation of internal operation, if you're curious about the gritty
>> details.
>>
>> This change has a few implications with regards to the application server:
>> • Thread pools with a core- and max-size which were locked together
>> may now have these variables decoupled in a safe and useful manner
>> • Thread pool configurations in the management model which previously
>> featured only a max size can now have a core size attribute introduced
>>
>> Most of the thread pools we use are built and configured within the
>> wildfly-core code base; that is where the first part of this change
>> will be done.  Later on there will be a follow-up change for WildFly
>> proper, but this of course can not happen until the core change is
>> merged and a core release done, so expect that part somewhat later.
>> The initial WIP change to wildfly-core can be found here [4].  Note
>> that no pull request will be opened until all the components are
>> Final.  I have done a good deal of testing already, but I will be
>> doing more CI testing on other platforms and configurations before
>> submitting the PR.
>>
>> I've made an attempt to link up all the JIRAs that are related to this
>> effort from [1] (either directly or indirectly).  If you find another
>> JIRA that you think is related, or if you have questions or concerns,
>> ask on this list, or you can ping me directly on HipChat or IRC.
>>
>> Thanks!
>>
>> [1] https://issues.jboss.org/browse/JBTHR-38 - Introduce new thread
>> pool implementation
>> [2] https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128
>> [3]
>> https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128#diff-f5807a689506af0b70791d93ae81cc1cR67
>> [4] https://github.com/dmlloyd/wildfly-core/tree/threadpool
>>
>> --
>> - DML
>>
>> _______________________________________________
>> wildfly-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
>
>
>
> --
> Andrig (Andy) T. Miller
> Global Platform Director, Middleware
> Red Hat, Inc.



--
- DML

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Introducing a new thread pool implementation

Andrig Miller
Sounds good.  If we have time this next week, we will see about getting a first test done during our face to face week.  It's a very busy week though, so we may not be able to until after next week.

Thanks for the pointers.

Andy

On Fri, Nov 3, 2017 at 9:33 AM, David Lloyd <[hidden email]> wrote:
The proposed change can be found here:
https://github.com/dmlloyd/wildfly-core/tree/threadpool

You can build that wildfly-core and then build a wildfly which uses
the produced build.  The whole process with -DskipTests should take
under 10 minutes.  I would be very happy to know what the performance
team finds, particularly in high-contention scenarios.  Thanks!

On Fri, Nov 3, 2017 at 10:17 AM, Andrig Miller <[hidden email]> wrote:
> I think it would be wise for the performance team to test this out at scale
> using our lab, as I'm sure you would agree.
>
> If you could point us at a particular build, or point us at what to build,
> we can do that testing in parallel and give feedback on how this is working.
>
> Thanks.
>
> Andy
>
> On Fri, Nov 3, 2017 at 8:03 AM, David Lloyd <[hidden email]> wrote:
>>
>> For WildFly 12 I intend to introduce a new thread pool implementation
>> [1] [2] that will replace most, if not all, uses of ThreadPoolExecutor
>> in the main codebases of WildFly and WildFly Core, and also in several
>> support projects such as JBoss MSC and XNIO.
>>
>> The thread pool has the following desirable characteristics:
>>
>> • Idle threads are always reused before queueing tasks or starting new
>> threads
>> • The most recently used idle thread is always preferred, which allows
>> the pool to shrink when it is not fully utilized
>> • Useful core/max thread pool size distinction
>> • API-equivalent to ThreadPoolExecutor
>> • Performance parity with common ThreadPoolExecutor configurations
>> • Suitable as a replacement for other jboss-threads thread pool types as
>> well
>>
>> In addition, it presents the following features:
>>
>> • Configure via a builder API
>> • Configurable exception handler
>> • Integrated task queue with optional size limit
>> • Full complement of standard metrics, which can be disabled
>> • Configurable rejection handler in the form of a handoff executor
>> • An optional growth-resistance algorithm which can be applied to
>> larger pools to create growth resistance between the core and maximum
>> pool size
>> • An optional post-terminate task
>> • A few system property based adjustments that can be applied for
>> experimental purposes
>> • A global flag which can be set via system property to recommend that
>> the new thread pool should be disabled, allowing an emergency fallback
>> in case of an unexpected problem
>>
>> Design-wise, the thread pool is based on a special lock-free/wait-free
>> combination FIFO/LIFO queue algorithm.  See [3] for a more complete
>> explanation of internal operation, if you're curious about the gritty
>> details.
>>
>> This change has a few implications with regards to the application server:
>> • Thread pools with a core- and max-size which were locked together
>> may now have these variables decoupled in a safe and useful manner
>> • Thread pool configurations in the management model which previously
>> featured only a max size can now have a core size attribute introduced
>>
>> Most of the thread pools we use are built and configured within the
>> wildfly-core code base; that is where the first part of this change
>> will be done.  Later on there will be a follow-up change for WildFly
>> proper, but this of course can not happen until the core change is
>> merged and a core release done, so expect that part somewhat later.
>> The initial WIP change to wildfly-core can be found here [4].  Note
>> that no pull request will be opened until all the components are
>> Final.  I have done a good deal of testing already, but I will be
>> doing more CI testing on other platforms and configurations before
>> submitting the PR.
>>
>> I've made an attempt to link up all the JIRAs that are related to this
>> effort from [1] (either directly or indirectly).  If you find another
>> JIRA that you think is related, or if you have questions or concerns,
>> ask on this list, or you can ping me directly on HipChat or IRC.
>>
>> Thanks!
>>
>> [1] https://issues.jboss.org/browse/JBTHR-38 - Introduce new thread
>> pool implementation
>> [2] https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128
>> [3]
>> https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128#diff-f5807a689506af0b70791d93ae81cc1cR67
>> [4] https://github.com/dmlloyd/wildfly-core/tree/threadpool
>>
>> --
>> - DML
>>
>> _______________________________________________
>> wildfly-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
>
>
>
> --
> Andrig (Andy) T. Miller
> Global Platform Director, Middleware
> Red Hat, Inc.



--
- DML



--
Andrig (Andy) T. Miller
Global Platform Director, Middleware
Red Hat, Inc.

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Introducing a new thread pool implementation

Dimitris Andreadis
In reply to this post by David Lloyd-2
Every time you re-design a new thread pool, a kitten dies.

/me ducks :)

On 03/11/2017 15:03, David Lloyd wrote:

> For WildFly 12 I intend to introduce a new thread pool implementation
> [1] [2] that will replace most, if not all, uses of ThreadPoolExecutor
> in the main codebases of WildFly and WildFly Core, and also in several
> support projects such as JBoss MSC and XNIO.
>
> The thread pool has the following desirable characteristics:
>
> • Idle threads are always reused before queueing tasks or starting new threads
> • The most recently used idle thread is always preferred, which allows
> the pool to shrink when it is not fully utilized
> • Useful core/max thread pool size distinction
> • API-equivalent to ThreadPoolExecutor
> • Performance parity with common ThreadPoolExecutor configurations
> • Suitable as a replacement for other jboss-threads thread pool types as well
>
> In addition, it presents the following features:
>
> • Configure via a builder API
> • Configurable exception handler
> • Integrated task queue with optional size limit
> • Full complement of standard metrics, which can be disabled
> • Configurable rejection handler in the form of a handoff executor
> • An optional growth-resistance algorithm which can be applied to
> larger pools to create growth resistance between the core and maximum
> pool size
> • An optional post-terminate task
> • A few system property based adjustments that can be applied for
> experimental purposes
> • A global flag which can be set via system property to recommend that
> the new thread pool should be disabled, allowing an emergency fallback
> in case of an unexpected problem
>
> Design-wise, the thread pool is based on a special lock-free/wait-free
> combination FIFO/LIFO queue algorithm.  See [3] for a more complete
> explanation of internal operation, if you're curious about the gritty
> details.
>
> This change has a few implications with regards to the application server:
> • Thread pools with a core- and max-size which were locked together
> may now have these variables decoupled in a safe and useful manner
> • Thread pool configurations in the management model which previously
> featured only a max size can now have a core size attribute introduced
>
> Most of the thread pools we use are built and configured within the
> wildfly-core code base; that is where the first part of this change
> will be done.  Later on there will be a follow-up change for WildFly
> proper, but this of course can not happen until the core change is
> merged and a core release done, so expect that part somewhat later.
> The initial WIP change to wildfly-core can be found here [4].  Note
> that no pull request will be opened until all the components are
> Final.  I have done a good deal of testing already, but I will be
> doing more CI testing on other platforms and configurations before
> submitting the PR.
>
> I've made an attempt to link up all the JIRAs that are related to this
> effort from [1] (either directly or indirectly).  If you find another
> JIRA that you think is related, or if you have questions or concerns,
> ask on this list, or you can ping me directly on HipChat or IRC.
>
> Thanks!
>
> [1] https://issues.jboss.org/browse/JBTHR-38 - Introduce new thread
> pool implementation
> [2] https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128
> [3] https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128#diff-f5807a689506af0b70791d93ae81cc1cR67
> [4] https://github.com/dmlloyd/wildfly-core/tree/threadpool
>

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Introducing a new thread pool implementation

Anil Saldanha
Any thoughts on getting developers not to use Classes and interfaces that are there since JDK 1.0? :-)

/me ducks behind Dimitris.

> On Nov 3, 2017, at 12:08 PM, Dimitris Andreadis <[hidden email]> wrote:
>
> Every time you re-design a new thread pool, a kitten dies.
>
> /me ducks :)
>
>> On 03/11/2017 15:03, David Lloyd wrote:
>> For WildFly 12 I intend to introduce a new thread pool implementation
>> [1] [2] that will replace most, if not all, uses of ThreadPoolExecutor
>> in the main codebases of WildFly and WildFly Core, and also in several
>> support projects such as JBoss MSC and XNIO.
>>
>> The thread pool has the following desirable characteristics:
>>
>> • Idle threads are always reused before queueing tasks or starting new threads
>> • The most recently used idle thread is always preferred, which allows
>> the pool to shrink when it is not fully utilized
>> • Useful core/max thread pool size distinction
>> • API-equivalent to ThreadPoolExecutor
>> • Performance parity with common ThreadPoolExecutor configurations
>> • Suitable as a replacement for other jboss-threads thread pool types as well
>>
>> In addition, it presents the following features:
>>
>> • Configure via a builder API
>> • Configurable exception handler
>> • Integrated task queue with optional size limit
>> • Full complement of standard metrics, which can be disabled
>> • Configurable rejection handler in the form of a handoff executor
>> • An optional growth-resistance algorithm which can be applied to
>> larger pools to create growth resistance between the core and maximum
>> pool size
>> • An optional post-terminate task
>> • A few system property based adjustments that can be applied for
>> experimental purposes
>> • A global flag which can be set via system property to recommend that
>> the new thread pool should be disabled, allowing an emergency fallback
>> in case of an unexpected problem
>>
>> Design-wise, the thread pool is based on a special lock-free/wait-free
>> combination FIFO/LIFO queue algorithm.  See [3] for a more complete
>> explanation of internal operation, if you're curious about the gritty
>> details.
>>
>> This change has a few implications with regards to the application server:
>> • Thread pools with a core- and max-size which were locked together
>> may now have these variables decoupled in a safe and useful manner
>> • Thread pool configurations in the management model which previously
>> featured only a max size can now have a core size attribute introduced
>>
>> Most of the thread pools we use are built and configured within the
>> wildfly-core code base; that is where the first part of this change
>> will be done.  Later on there will be a follow-up change for WildFly
>> proper, but this of course can not happen until the core change is
>> merged and a core release done, so expect that part somewhat later.
>> The initial WIP change to wildfly-core can be found here [4].  Note
>> that no pull request will be opened until all the components are
>> Final.  I have done a good deal of testing already, but I will be
>> doing more CI testing on other platforms and configurations before
>> submitting the PR.
>>
>> I've made an attempt to link up all the JIRAs that are related to this
>> effort from [1] (either directly or indirectly).  If you find another
>> JIRA that you think is related, or if you have questions or concerns,
>> ask on this list, or you can ping me directly on HipChat or IRC.
>>
>> Thanks!
>>
>> [1] https://issues.jboss.org/browse/JBTHR-38 - Introduce new thread
>> pool implementation
>> [2] https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128
>> [3] https://github.com/jbossas/jboss-threads/commit/be95b8d6b42128#diff-f5807a689506af0b70791d93ae81cc1cR67
>> [4] https://github.com/dmlloyd/wildfly-core/tree/threadpool
>>
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev