Another Failing Tests E-Mail

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Another Failing Tests E-Mail

Darran Lofthouse
For the past couple of days I have been unable to perform an all tests
build, I have had some failures reported but the most common issue is
the testsuite hanging.

If I generate a thread dump for the surefire process I regularly see
threads stuck as in the trace at the bottom of this e-mail.

To me it seems like there may actually be two issues here: -
  1 - The hang itself.
  2 - The lack of any timeout.

Is anyone else experiencing this hang?

Regards,
Darran Lofthouse.

"main" prio=10 tid=0xb6a04c00 nid=0x7c0e in Object.wait() [0xb6bd8000]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x7ed2b430> (a
org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
        at java.lang.Object.wait(Object.java:485)
        at org.jboss.threads.AsyncFutureTask.await(AsyncFutureTask.java:192)
        - locked <0x7ed2b430> (a
org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
        at org.jboss.threads.AsyncFutureTask.get(AsyncFutureTask.java:266)
        - locked <0x7ed2b430> (a
org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
        at
org.jboss.as.protocol.mgmt.ManagementRequest.executeForResult(ManagementRequest.java:131)
        at
org.jboss.as.controller.client.impl.AbstractModelControllerClient.executeSynch(AbstractModelControllerClient.java:127)
        at
org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:92)
        at
org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:82)
        at
org.jboss.as.arquillian.container.ManagementClient.isServerInRunningState(ManagementClient.java:134)
        at
org.jboss.as.arquillian.container.managed.ManagedDeployableContainer.startInternal(ManagedDeployableContainer.java:140)
        at
org.jboss.as.arquillian.container.CommonDeployableContainer.start(CommonDeployableContainer.java:75)



_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Another Failing Tests E-Mail

Ales Justin
Yeah, i've not been able to do a full testsuite build as well for the last few days.
Thought it was just my branch, as i have some subsystem changes,
But it was the same yday when catching that ExampleDS bug on master.
I'll need to check thread dump to see if it's the same isssue.

Sent from my iPad

On Oct 29, 2011, at 12:06, Darran Lofthouse <[hidden email]> wrote:

> For the past couple of days I have been unable to perform an all tests
> build, I have had some failures reported but the most common issue is
> the testsuite hanging.
>
> If I generate a thread dump for the surefire process I regularly see
> threads stuck as in the trace at the bottom of this e-mail.
>
> To me it seems like there may actually be two issues here: -
>  1 - The hang itself.
>  2 - The lack of any timeout.
>
> Is anyone else experiencing this hang?
>
> Regards,
> Darran Lofthouse.
>
> "main" prio=10 tid=0xb6a04c00 nid=0x7c0e in Object.wait() [0xb6bd8000]
>    java.lang.Thread.State: WAITING (on object monitor)
>    at java.lang.Object.wait(Native Method)
>    - waiting on <0x7ed2b430> (a
> org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
>    at java.lang.Object.wait(Object.java:485)
>    at org.jboss.threads.AsyncFutureTask.await(AsyncFutureTask.java:192)
>    - locked <0x7ed2b430> (a
> org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
>    at org.jboss.threads.AsyncFutureTask.get(AsyncFutureTask.java:266)
>    - locked <0x7ed2b430> (a
> org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
>    at
> org.jboss.as.protocol.mgmt.ManagementRequest.executeForResult(ManagementRequest.java:131)
>    at
> org.jboss.as.controller.client.impl.AbstractModelControllerClient.executeSynch(AbstractModelControllerClient.java:127)
>    at
> org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:92)
>    at
> org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:82)
>    at
> org.jboss.as.arquillian.container.ManagementClient.isServerInRunningState(ManagementClient.java:134)
>    at
> org.jboss.as.arquillian.container.managed.ManagedDeployableContainer.startInternal(ManagedDeployableContainer.java:140)
>    at
> org.jboss.as.arquillian.container.CommonDeployableContainer.start(CommonDeployableContainer.java:75)
>
>
>
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Another Failing Tests E-Mail

Jaikiran Pai
In reply to this post by Darran Lofthouse
I have been seeing the hangs too. Kabir filed a JIRA for the timeout
issue https://issues.jboss.org/browse/AS7-2396

-Jaikiran
On Saturday 29 October 2011 03:36 PM, Darran Lofthouse wrote:

> For the past couple of days I have been unable to perform an all tests
> build, I have had some failures reported but the most common issue is
> the testsuite hanging.
>
> If I generate a thread dump for the surefire process I regularly see
> threads stuck as in the trace at the bottom of this e-mail.
>
> To me it seems like there may actually be two issues here: -
>    1 - The hang itself.
>    2 - The lack of any timeout.
>
> Is anyone else experiencing this hang?
>
> Regards,
> Darran Lofthouse.
>
> "main" prio=10 tid=0xb6a04c00 nid=0x7c0e in Object.wait() [0xb6bd8000]
>      java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on<0x7ed2b430>  (a
> org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
> at java.lang.Object.wait(Object.java:485)
> at org.jboss.threads.AsyncFutureTask.await(AsyncFutureTask.java:192)
> - locked<0x7ed2b430>  (a
> org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
> at org.jboss.threads.AsyncFutureTask.get(AsyncFutureTask.java:266)
> - locked<0x7ed2b430>  (a
> org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
> at
> org.jboss.as.protocol.mgmt.ManagementRequest.executeForResult(ManagementRequest.java:131)
> at
> org.jboss.as.controller.client.impl.AbstractModelControllerClient.executeSynch(AbstractModelControllerClient.java:127)
> at
> org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:92)
> at
> org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:82)
> at
> org.jboss.as.arquillian.container.ManagementClient.isServerInRunningState(ManagementClient.java:134)
> at
> org.jboss.as.arquillian.container.managed.ManagedDeployableContainer.startInternal(ManagedDeployableContainer.java:140)
> at
> org.jboss.as.arquillian.container.CommonDeployableContainer.start(CommonDeployableContainer.java:75)
>
>
>
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Another Failing Tests E-Mail

kkhan
Sorry, I've had this sitting in a branch for a few days without opening a pull request. Here it is: https://github.com/jbossas/jboss-as/pull/533

Note that I still see hangs in this branch.
On 29 Oct 2011, at 14:25, Jaikiran Pai wrote:

> I have been seeing the hangs too. Kabir filed a JIRA for the timeout
> issue https://issues.jboss.org/browse/AS7-2396
>
> -Jaikiran
> On Saturday 29 October 2011 03:36 PM, Darran Lofthouse wrote:
>> For the past couple of days I have been unable to perform an all tests
>> build, I have had some failures reported but the most common issue is
>> the testsuite hanging.
>>
>> If I generate a thread dump for the surefire process I regularly see
>> threads stuck as in the trace at the bottom of this e-mail.
>>
>> To me it seems like there may actually be two issues here: -
>>   1 - The hang itself.
>>   2 - The lack of any timeout.
>>
>> Is anyone else experiencing this hang?
>>
>> Regards,
>> Darran Lofthouse.
>>
>> "main" prio=10 tid=0xb6a04c00 nid=0x7c0e in Object.wait() [0xb6bd8000]
>>     java.lang.Thread.State: WAITING (on object monitor)
>> at java.lang.Object.wait(Native Method)
>> - waiting on<0x7ed2b430>  (a
>> org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
>> at java.lang.Object.wait(Object.java:485)
>> at org.jboss.threads.AsyncFutureTask.await(AsyncFutureTask.java:192)
>> - locked<0x7ed2b430>  (a
>> org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
>> at org.jboss.threads.AsyncFutureTask.get(AsyncFutureTask.java:266)
>> - locked<0x7ed2b430>  (a
>> org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
>> at
>> org.jboss.as.protocol.mgmt.ManagementRequest.executeForResult(ManagementRequest.java:131)
>> at
>> org.jboss.as.controller.client.impl.AbstractModelControllerClient.executeSynch(AbstractModelControllerClient.java:127)
>> at
>> org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:92)
>> at
>> org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:82)
>> at
>> org.jboss.as.arquillian.container.ManagementClient.isServerInRunningState(ManagementClient.java:134)
>> at
>> org.jboss.as.arquillian.container.managed.ManagedDeployableContainer.startInternal(ManagedDeployableContainer.java:140)
>> at
>> org.jboss.as.arquillian.container.CommonDeployableContainer.start(CommonDeployableContainer.java:75)
>>
>>
>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Another Failing Tests E-Mail

Brian Stansberry
I still see the hangs though. Thread dump attached. (This is with
Kabir's 533 pull branch; I see the same thing with master although the
line numbers may vary.)

The key one is:

"pool-1-thread-1" prio=5 tid=1019c4000 nid=0x10be76000 in Object.wait()
[10be75000]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <7f380de28> (a java.lang.Object)
        at java.lang.Object.wait(Object.java:485)
        at org.xnio.AbstractIoFuture.await(AbstractIoFuture.java:73)
        - locked <7f380de28> (a java.lang.Object)
        at org.xnio.AbstractIoFuture.get(AbstractIoFuture.java:156)
        - locked <7f380de28> (a java.lang.Object)
        at
org.jboss.as.protocol.ProtocolChannelClient.connect(ProtocolChannelClient.java:109)
        at
org.jboss.as.protocol.mgmt.ManagementClientChannelStrategy$Establishing.getChannel(ManagementClientChannelStrategy.java:123)
        at
org.jboss.as.protocol.mgmt.ManagementRequest$1.run(ManagementRequest.java:101)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)

The XNIO threads don't
On 10/30/11 6:46 AM, Kabir Khan wrote:

> Sorry, I've had this sitting in a branch for a few days without opening a pull request. Here it is: https://github.com/jbossas/jboss-as/pull/533
>
> Note that I still see hangs in this branch.
> On 29 Oct 2011, at 14:25, Jaikiran Pai wrote:
>
>> I have been seeing the hangs too. Kabir filed a JIRA for the timeout
>> issue https://issues.jboss.org/browse/AS7-2396
>>
>> -Jaikiran
>> On Saturday 29 October 2011 03:36 PM, Darran Lofthouse wrote:
>>> For the past couple of days I have been unable to perform an all tests
>>> build, I have had some failures reported but the most common issue is
>>> the testsuite hanging.
>>>
>>> If I generate a thread dump for the surefire process I regularly see
>>> threads stuck as in the trace at the bottom of this e-mail.
>>>
>>> To me it seems like there may actually be two issues here: -
>>>    1 - The hang itself.
>>>    2 - The lack of any timeout.
>>>
>>> Is anyone else experiencing this hang?
>>>
>>> Regards,
>>> Darran Lofthouse.
>>>
>>> "main" prio=10 tid=0xb6a04c00 nid=0x7c0e in Object.wait() [0xb6bd8000]
>>>      java.lang.Thread.State: WAITING (on object monitor)
>>> at java.lang.Object.wait(Native Method)
>>> - waiting on<0x7ed2b430>   (a
>>> org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
>>> at java.lang.Object.wait(Object.java:485)
>>> at org.jboss.threads.AsyncFutureTask.await(AsyncFutureTask.java:192)
>>> - locked<0x7ed2b430>   (a
>>> org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
>>> at org.jboss.threads.AsyncFutureTask.get(AsyncFutureTask.java:266)
>>> - locked<0x7ed2b430>   (a
>>> org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
>>> at
>>> org.jboss.as.protocol.mgmt.ManagementRequest.executeForResult(ManagementRequest.java:131)
>>> at
>>> org.jboss.as.controller.client.impl.AbstractModelControllerClient.executeSynch(AbstractModelControllerClient.java:127)
>>> at
>>> org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:92)
>>> at
>>> org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:82)
>>> at
>>> org.jboss.as.arquillian.container.ManagementClient.isServerInRunningState(ManagementClient.java:134)
>>> at
>>> org.jboss.as.arquillian.container.managed.ManagedDeployableContainer.startInternal(ManagedDeployableContainer.java:140)
>>> at
>>> org.jboss.as.arquillian.container.CommonDeployableContainer.start(CommonDeployableContainer.java:75)
>>>
>>>
>>>
>>> _______________________________________________
>>> jboss-as7-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>
>
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

--
Brian Stansberry
Principal Software Engineer
JBoss by Red Hat

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

thread-dump.txt (14K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Another Failing Tests E-Mail

Brian Stansberry
While the test client is hung, if I start the CLI and try to connect to
the server I get the same problem. A thread dump of the CLI shows the
same pattern.

Note that the patch in pull 533 causes the CLI to hang indefinitely. In
master it fails after 5 secs reporting it can't connect. So removing the
timeout makes the situation worse here.

On 10/31/11 12:48 AM, Brian Stansberry wrote:

> I still see the hangs though. Thread dump attached. (This is with
> Kabir's 533 pull branch; I see the same thing with master although the
> line numbers may vary.)
>
> The key one is:
>
> "pool-1-thread-1" prio=5 tid=1019c4000 nid=0x10be76000 in Object.wait()
> [10be75000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <7f380de28> (a java.lang.Object)
> at java.lang.Object.wait(Object.java:485)
> at org.xnio.AbstractIoFuture.await(AbstractIoFuture.java:73)
> - locked <7f380de28> (a java.lang.Object)
> at org.xnio.AbstractIoFuture.get(AbstractIoFuture.java:156)
> - locked <7f380de28> (a java.lang.Object)
> at
> org.jboss.as.protocol.ProtocolChannelClient.connect(ProtocolChannelClient.java:109)
>
> at
> org.jboss.as.protocol.mgmt.ManagementClientChannelStrategy$Establishing.getChannel(ManagementClientChannelStrategy.java:123)
>
> at
> org.jboss.as.protocol.mgmt.ManagementRequest$1.run(ManagementRequest.java:101)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> at java.lang.Thread.run(Thread.java:680)
>
> The XNIO threads don't
> On 10/30/11 6:46 AM, Kabir Khan wrote:
>> Sorry, I've had this sitting in a branch for a few days without
>> opening a pull request. Here it is:
>> https://github.com/jbossas/jboss-as/pull/533
>>
>> Note that I still see hangs in this branch.
>> On 29 Oct 2011, at 14:25, Jaikiran Pai wrote:
>>
>>> I have been seeing the hangs too. Kabir filed a JIRA for the timeout
>>> issue https://issues.jboss.org/browse/AS7-2396
>>>
>>> -Jaikiran
>>> On Saturday 29 October 2011 03:36 PM, Darran Lofthouse wrote:
>>>> For the past couple of days I have been unable to perform an all tests
>>>> build, I have had some failures reported but the most common issue is
>>>> the testsuite hanging.
>>>>
>>>> If I generate a thread dump for the surefire process I regularly see
>>>> threads stuck as in the trace at the bottom of this e-mail.
>>>>
>>>> To me it seems like there may actually be two issues here: -
>>>> 1 - The hang itself.
>>>> 2 - The lack of any timeout.
>>>>
>>>> Is anyone else experiencing this hang?
>>>>
>>>> Regards,
>>>> Darran Lofthouse.
>>>>
>>>> "main" prio=10 tid=0xb6a04c00 nid=0x7c0e in Object.wait() [0xb6bd8000]
>>>> java.lang.Thread.State: WAITING (on object monitor)
>>>> at java.lang.Object.wait(Native Method)
>>>> - waiting on<0x7ed2b430> (a
>>>> org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
>>>> at java.lang.Object.wait(Object.java:485)
>>>> at org.jboss.threads.AsyncFutureTask.await(AsyncFutureTask.java:192)
>>>> - locked<0x7ed2b430> (a
>>>> org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
>>>> at org.jboss.threads.AsyncFutureTask.get(AsyncFutureTask.java:266)
>>>> - locked<0x7ed2b430> (a
>>>> org.jboss.as.protocol.mgmt.ManagementRequest$ManagementFuture)
>>>> at
>>>> org.jboss.as.protocol.mgmt.ManagementRequest.executeForResult(ManagementRequest.java:131)
>>>>
>>>> at
>>>> org.jboss.as.controller.client.impl.AbstractModelControllerClient.executeSynch(AbstractModelControllerClient.java:127)
>>>>
>>>> at
>>>> org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:92)
>>>>
>>>> at
>>>> org.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:82)
>>>>
>>>> at
>>>> org.jboss.as.arquillian.container.ManagementClient.isServerInRunningState(ManagementClient.java:134)
>>>>
>>>> at
>>>> org.jboss.as.arquillian.container.managed.ManagedDeployableContainer.startInternal(ManagedDeployableContainer.java:140)
>>>>
>>>> at
>>>> org.jboss.as.arquillian.container.CommonDeployableContainer.start(CommonDeployableContainer.java:75)
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> jboss-as7-dev mailing list
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>
>>> _______________________________________________
>>> jboss-as7-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>
>
>
>
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


--
Brian Stansberry
Principal Software Engineer
JBoss by Red Hat
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev