New openshift resource issue

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

New openshift resource issue

Scott Stark
One problem people are running into in testing as7 openshift express
cartridge is running out of threads. I need a configuration that has all
of the subsystem thread pools/usage explicitly declared so that one
knows that the upper bound on the thread is. The reason for this is that
the server runs in a constrained environment that has an upper limit on
the number of process(maps to native threads) the server can create.
Right now there are no subsystems referring to a thread pool in the
threads subsystem.

At a minimum, there needs to be a sample standalone.xml configuration
that is tested that illustrates how to do this if we are not going to do
this by default.

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: New openshift resource issue

Heiko Braun

Good point.
I was thinking about this today as well.
Basically you cannot tell what thread pool each subsystem relies on.
Ideally they would be referencing thread pools by it's logical name through the configuration.

Ike

On Jul 19, 2011, at 10:20 PM, Scott Stark wrote:

> Right now there are no subsystems referring to a thread pool in the
> threads subsystem.
>
> At a minimum, there needs to be a sample standalone.xml configuration
> that is tested that illustrates how to do this if we are not going to do
> this by default.


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: New openshift resource issue

Emanuel Muckenhuber
I'm afraid that the current thread-pool configuration is not really
consistent. We plan to remove the threads subsystem and move the thread
pool configuration directly in each subsystem where they are used. At
least that's the last i heard of.

I assume we would also need add a thread pool configuration for 'system'
(non subsystem specific) usages as well in order to be able to address
the resource issue Scott described.

On 07/20/2011 09:03 AM, Heiko Braun wrote:

>
> Good point.
> I was thinking about this today as well.
> Basically you cannot tell what thread pool each subsystem relies on.
> Ideally they would be referencing thread pools by it's logical name through the configuration.
>
> Ike
>
> On Jul 19, 2011, at 10:20 PM, Scott Stark wrote:
>
>> Right now there are no subsystems referring to a thread pool in the
>> threads subsystem.
>>
>> At a minimum, there needs to be a sample standalone.xml configuration
>> that is tested that illustrates how to do this if we are not going to do
>> this by default.
>
>
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: New openshift resource issue

Rémy Maucherat
In reply to this post by Scott Stark
On Tue, 2011-07-19 at 13:20 -0700, Scott Stark wrote:

> One problem people are running into in testing as7 openshift express
> cartridge is running out of threads. I need a configuration that has all
> of the subsystem thread pools/usage explicitly declared so that one
> knows that the upper bound on the thread is. The reason for this is that
> the server runs in a constrained environment that has an upper limit on
> the number of process(maps to native threads) the server can create.
> Right now there are no subsystems referring to a thread pool in the
> threads subsystem.
>
> At a minimum, there needs to be a sample standalone.xml configuration
> that is tested that illustrates how to do this if we are not going to do
> this by default.

Can you give details on the max amount of threads ?

The default config uses 46 threads after server startup, so if the
constraint is too low, it will simply kill the ability to process any
requests.

--
Remy Maucherat <[hidden email]>
Red Hat Inc

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: New openshift resource issue

Scott Stark
The rhel instance uses:
http://www.kernel.org/pub/linux/libs/pam/Linux-PAM-html/sag-pam_limits.html

with the following process limit:
limits_nproc=50        # max number of processes

On 7/20/11 4:38 AM, Remy Maucherat wrote:

> On Tue, 2011-07-19 at 13:20 -0700, Scott Stark wrote:
>> One problem people are running into in testing as7 openshift express
>> cartridge is running out of threads. I need a configuration that has all
>> of the subsystem thread pools/usage explicitly declared so that one
>> knows that the upper bound on the thread is. The reason for this is that
>> the server runs in a constrained environment that has an upper limit on
>> the number of process(maps to native threads) the server can create.
>> Right now there are no subsystems referring to a thread pool in the
>> threads subsystem.
>>
>> At a minimum, there needs to be a sample standalone.xml configuration
>> that is tested that illustrates how to do this if we are not going to do
>> this by default.
> Can you give details on the max amount of threads ?
>
> The default config uses 46 threads after server startup, so if the
> constraint is too low, it will simply kill the ability to process any
> requests.
>

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: New openshift resource issue

Rémy Maucherat
On Wed, 2011-07-20 at 06:59 -0700, Scott Stark wrote:
> The rhel instance uses:
> http://www.kernel.org/pub/linux/libs/pam/Linux-PAM-html/sag-pam_limits.html
>
> with the following process limit:
> limits_nproc=50        # max number of processes

Ok, so this is not going to do anything then. The server will have only
5 request processsing threads for web (assuming no other component
spawns any thread, which is obviously not going to happen).

Unless I missed something, we have a problem, and some thread count
optimization is needed. My internal thread pool defaults are also too
high right now.

--
Remy Maucherat <[hidden email]>
Red Hat Inc

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: New openshift resource issue

Scott Stark
In reply to this post by Emanuel Muckenhuber
During startup, it seems that deployment of multiple apps is using a lot
of threads due to processing of the associated services. I'm working on
identifying where all the uses are coming from.

If you know there is a hard limit to the number of threads that can be
created, it would be good to be able to know the server configuration
will not exceed that.

On 7/20/11 3:51 AM, Emanuel Muckenhuber wrote:

> I'm afraid that the current thread-pool configuration is not really
> consistent. We plan to remove the threads subsystem and move the thread
> pool configuration directly in each subsystem where they are used. At
> least that's the last i heard of.
>
> I assume we would also need add a thread pool configuration for 'system'
> (non subsystem specific) usages as well in order to be able to address
> the resource issue Scott described.
>
> On 07/20/2011 09:03 AM, Heiko Braun wrote:
>> Good point.
>> I was thinking about this today as well.
>> Basically you cannot tell what thread pool each subsystem relies on.
>> Ideally they would be referencing thread pools by it's logical name through the configuration.
>>
>> Ike
>>
>> On Jul 19, 2011, at 10:20 PM, Scott Stark wrote:
>>
>>> Right now there are no subsystems referring to a thread pool in the
>>> threads subsystem.
>>>
>>> At a minimum, there needs to be a sample standalone.xml configuration
>>> that is tested that illustrates how to do this if we are not going to do
>>> this by default.
>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: New openshift resource issue

Emanuel Muckenhuber
Hmm, that should be the MSC thread pool then. You can try to limit that
in org.jboss.as.server.BootstrapImpl - when creating the
ServiceContainer there.

On 07/20/2011 04:15 PM, Scott Stark wrote:

> During startup, it seems that deployment of multiple apps is using a lot
> of threads due to processing of the associated services. I'm working on
> identifying where all the uses are coming from.
>
> If you know there is a hard limit to the number of threads that can be
> created, it would be good to be able to know the server configuration
> will not exceed that.
>
> On 7/20/11 3:51 AM, Emanuel Muckenhuber wrote:
>> I'm afraid that the current thread-pool configuration is not really
>> consistent. We plan to remove the threads subsystem and move the thread
>> pool configuration directly in each subsystem where they are used. At
>> least that's the last i heard of.
>>
>> I assume we would also need add a thread pool configuration for 'system'
>> (non subsystem specific) usages as well in order to be able to address
>> the resource issue Scott described.
>>
>> On 07/20/2011 09:03 AM, Heiko Braun wrote:
>>> Good point.
>>> I was thinking about this today as well.
>>> Basically you cannot tell what thread pool each subsystem relies on.
>>> Ideally they would be referencing thread pools by it's logical name through the configuration.
>>>
>>> Ike
>>>
>>> On Jul 19, 2011, at 10:20 PM, Scott Stark wrote:
>>>
>>>> Right now there are no subsystems referring to a thread pool in the
>>>> threads subsystem.
>>>>
>>>> At a minimum, there needs to be a sample standalone.xml configuration
>>>> that is tested that illustrates how to do this if we are not going to do
>>>> this by default.
>>>
>>> _______________________________________________
>>> jboss-as7-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev