Testsuite changes

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Testsuite changes

kkhan
I have pushed Ondrej's work, and here is how to run it for developers (summarized from another mail from a few days ago):

>
> mvn clean install
> -> runs smoke tests as expected
> --
>
> mvn clean install -DallTests
> -> runs all tests as expected
> --
>
> mvn clean install -DallTests=true
> -> runs all tests as expected
> --
>
> mvn install -pl testsuite/integration/group-smoke/ -Dtest=org.jboss.as.test.smoke.embedded.demos.client.jms.JmsClientTestCase
> -> runs the one test as expected
> (I did -pl testsute/integration/group-smoke on the assumption that we will be moving to modules)
> --
>
> cd testsuite/integration/group-basic
> mvn install -Dtest=org.jboss.as.test.integration.deployment.classloading.ear.EarClassPath2TestCase
> -> runs the one test as expected
> --
>
> cd testsuite/integration/group-basic
> mvn install
> -> Runs those tests as expected
> --

There will be some slight changes coming up, such as moving the test classes into individual modules. They currently all live in testsuite/integration/ and are compiled by the individual sub-modules.
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Testsuite changes

Darran Lofthouse
Have we lost -DskipTests again?

On 11/25/2011 01:56 PM, Kabir Khan wrote:

> I have pushed Ondrej's work, and here is how to run it for developers (summarized from another mail from a few days ago):
>>
>> mvn clean install
>> ->  runs smoke tests as expected
>> --
>>
>> mvn clean install -DallTests
>> ->  runs all tests as expected
>> --
>>
>> mvn clean install -DallTests=true
>> ->  runs all tests as expected
>> --
>>
>> mvn install -pl testsuite/integration/group-smoke/ -Dtest=org.jboss.as.test.smoke.embedded.demos.client.jms.JmsClientTestCase
>> ->  runs the one test as expected
>> (I did -pl testsute/integration/group-smoke on the assumption that we will be moving to modules)
>> --
>>
>> cd testsuite/integration/group-basic
>> mvn install -Dtest=org.jboss.as.test.integration.deployment.classloading.ear.EarClassPath2TestCase
>> ->  runs the one test as expected
>> --
>>
>> cd testsuite/integration/group-basic
>> mvn install
>> ->  Runs those tests as expected
>> --
>
> There will be some slight changes coming up, such as moving the test classes into individual modules. They currently all live in testsuite/integration/ and are compiled by the individual sub-modules.
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Testsuite changes

Thomas Diesler
Here is a document that describes the testsuite requirements

http://community.jboss.org/wiki/ASTestsuiteRequirements

Ondřej, could you pls check that these requirements are met and document
how this currently works.

cheers
-thomas

On 11/25/2011 03:31 PM, Darran Lofthouse wrote:

> Have we lost -DskipTests again?
>
> On 11/25/2011 01:56 PM, Kabir Khan wrote:
>> I have pushed Ondrej's work, and here is how to run it for developers (summarized from another mail from a few days ago):
>>> mvn clean install
>>> ->   runs smoke tests as expected
>>> --
>>>
>>> mvn clean install -DallTests
>>> ->   runs all tests as expected
>>> --
>>>
>>> mvn clean install -DallTests=true
>>> ->   runs all tests as expected
>>> --
>>>
>>> mvn install -pl testsuite/integration/group-smoke/ -Dtest=org.jboss.as.test.smoke.embedded.demos.client.jms.JmsClientTestCase
>>> ->   runs the one test as expected
>>> (I did -pl testsute/integration/group-smoke on the assumption that we will be moving to modules)
>>> --
>>>
>>> cd testsuite/integration/group-basic
>>> mvn install -Dtest=org.jboss.as.test.integration.deployment.classloading.ear.EarClassPath2TestCase
>>> ->   runs the one test as expected
>>> --
>>>
>>> cd testsuite/integration/group-basic
>>> mvn install
>>> ->   Runs those tests as expected
>>> --
>> There will be some slight changes coming up, such as moving the test classes into individual modules. They currently all live in testsuite/integration/ and are compiled by the individual sub-modules.
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Testsuite changes

Thomas Diesler
In reply to this post by kkhan
So how do I actually run most integration tests together

https://github.com/tdiesler/jboss-as/blob/master/testsuite/integration/group-basic/pom.xml

The basic group executes

<includes>
<include>org/jboss/as/test/integration/**/*SecondTestCase.java</include>
<include>org/jboss/as/test/integration/ejb/iiop/**/*TestCase*.java</include>
<include>org/jboss/as/test/integration/ejb/mdb/**/*TestCase*.java</include>
<include>org/jboss/as/test/integration/ejb/entity/cmp/**/*TestCase*.java</include>
<include>org/jboss/as/test/integration/ejb/remote/entity/cmp/**/*TestCase*.java</include>
<include>org/jboss/as/test/integration/ejb/management/deployments/EjbJarInEarRuntimeResourcesTestCase.java</include>
<include>org/jboss/as/test/integration/ejb/management/deployments/EjbJarRuntimeResourcesTestCase.java</include>
<include>org/jboss/as/test/integration/messaging/**/*TestCase*.java</include>
<include>org/jboss/as/test/integration/management/cli/HelpTestCase.java</include>
<include>org/jboss/as/test/integration/management/cli/JmsTestCase.java</include>
</includes> We have however many more [tdiesler@tdvaio jboss-as]$ ls -1
testsuite/integration/src/test/java/org/jboss/as/test/integration/
beanvalidation common deployment ee ejb jaxb jaxr jaxrs jmx jpa jsf
management messaging osgi pojo rar sar security transaction web weld ws
xerces
Basically I need to run the OSGi tests together with the ejb, web, etc
tests -thomas



On 11/25/2011 02:56 PM, Kabir Khan wrote:

> I have pushed Ondrej's work, and here is how to run it for developers (summarized from another mail from a few days ago):
>> mvn clean install
>> ->  runs smoke tests as expected
>> --
>>
>> mvn clean install -DallTests
>> ->  runs all tests as expected
>> --
>>
>> mvn clean install -DallTests=true
>> ->  runs all tests as expected
>> --
>>
>> mvn install -pl testsuite/integration/group-smoke/ -Dtest=org.jboss.as.test.smoke.embedded.demos.client.jms.JmsClientTestCase
>> ->  runs the one test as expected
>> (I did -pl testsute/integration/group-smoke on the assumption that we will be moving to modules)
>> --
>>
>> cd testsuite/integration/group-basic
>> mvn install -Dtest=org.jboss.as.test.integration.deployment.classloading.ear.EarClassPath2TestCase
>> ->  runs the one test as expected
>> --
>>
>> cd testsuite/integration/group-basic
>> mvn install
>> ->  Runs those tests as expected
>> --
> There will be some slight changes coming up, such as moving the test classes into individual modules. They currently all live in testsuite/integration/ and are compiled by the individual sub-modules.
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Testsuite changes

kkhan
Was this functionality available before?
On 25 Nov 2011, at 16:58, Thomas Diesler wrote:

> So how do I actually run most integration tests together
>
> https://github.com/tdiesler/jboss-as/blob/master/testsuite/integration/group-basic/pom.xml
>
> The basic group executes
>
> <includes>
> <include>org/jboss/as/test/integration/**/*SecondTestCase.java</include>
> <include>org/jboss/as/test/integration/ejb/iiop/**/*TestCase*.java</include>
> <include>org/jboss/as/test/integration/ejb/mdb/**/*TestCase*.java</include>
> <include>org/jboss/as/test/integration/ejb/entity/cmp/**/*TestCase*.java</include>
> <include>org/jboss/as/test/integration/ejb/remote/entity/cmp/**/*TestCase*.java</include>
> <include>org/jboss/as/test/integration/ejb/management/deployments/EjbJarInEarRuntimeResourcesTestCase.java</include>
> <include>org/jboss/as/test/integration/ejb/management/deployments/EjbJarRuntimeResourcesTestCase.java</include>
> <include>org/jboss/as/test/integration/messaging/**/*TestCase*.java</include>
> <include>org/jboss/as/test/integration/management/cli/HelpTestCase.java</include>
> <include>org/jboss/as/test/integration/management/cli/JmsTestCase.java</include>
> </includes> We have however many more [tdiesler@tdvaio jboss-as]$ ls -1 testsuite/integration/src/test/java/org/jboss/as/test/integration/ beanvalidation common deployment ee ejb jaxb jaxr jaxrs jmx jpa jsf management messaging osgi pojo rar sar security transaction web weld ws xerces
> Basically I need to run the OSGi tests together with the ejb, web, etc tests -thomas
>
>
>
> On 11/25/2011 02:56 PM, Kabir Khan wrote:
>> I have pushed Ondrej's work, and here is how to run it for developers (summarized from another mail from a few days ago):
>>> mvn clean install
>>> ->  runs smoke tests as expected
>>> --
>>>
>>> mvn clean install -DallTests
>>> ->  runs all tests as expected
>>> --
>>>
>>> mvn clean install -DallTests=true
>>> ->  runs all tests as expected
>>> --
>>>
>>> mvn install -pl testsuite/integration/group-smoke/ -Dtest=org.jboss.as.test.smoke.embedded.demos.client.jms.JmsClientTestCase
>>> ->  runs the one test as expected
>>> (I did -pl testsute/integration/group-smoke on the assumption that we will be moving to modules)
>>> --
>>>
>>> cd testsuite/integration/group-basic
>>> mvn install -Dtest=org.jboss.as.test.integration.deployment.classloading.ear.EarClassPath2TestCase
>>> ->  runs the one test as expected
>>> --
>>>
>>> cd testsuite/integration/group-basic
>>> mvn install
>>> ->  Runs those tests as expected
>>> --
>> There will be some slight changes coming up, such as moving the test classes into individual modules. They currently all live in testsuite/integration/ and are compiled by the individual sub-modules.
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>
> --
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Thomas Diesler
> JBoss OSGi Lead
> JBoss, a division of Red Hat
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Testsuite changes

kkhan
In reply to this post by kkhan
I have some work in progress in my branch at: https://github.com/kabir/jboss-as/commits/rework-testsuite

I have pushed all the testsuite/integration classes into sub modules, and renamed group-smoke -> smoke etc.
There are still a few scripts and xslt hanging around in the testsuite/integration aggregator. I have not investigated their usage.
>From what I can see the pom2.xml files are not being used, so I've made no attempt to update those.
There are some TODOs in the pom's where I am not sure about stuff. I did not change the pom2.xml files, I'm not sure they are being used?

Broken stuff:
It is impossible to run a single test from Eclipse, the arquillian.xml file is not found in Eclipse. Probably because this from testsuite/pom.xml does not get added to Eclipse's classpath:
                    <additionalClasspathElements combine.children="append">
                        <additionalClasspathElement>${basedir}/src/test/config/arq</additionalClasspathElement>
                    </additionalClasspathElements>
so I think we should move these back to the resources directory.
Thomas does not like -DallTests running benchmark and stress, what would a good system property name be to enable running those?
We need to reenable the -DskipTests=true we had before
Rather than using xslt to modify the port offsets, we should change the socket bindings to use port-offset="${jboss.port.offset:0}" so that we can just pass a system property.

I won't be around much this weekend, so there is minimum chance of me crashing with you if any of you decide to do anything.  However, please make your changes public somewhere so I can pick them up if I feel the urge.


On 25 Nov 2011, at 13:56, Kabir Khan wrote:

> I have pushed Ondrej's work, and here is how to run it for developers (summarized from another mail from a few days ago):
>>
>> mvn clean install
>> -> runs smoke tests as expected
>> --
>>
>> mvn clean install -DallTests
>> -> runs all tests as expected
>> --
>>
>> mvn clean install -DallTests=true
>> -> runs all tests as expected
>> --
>>
>> mvn install -pl testsuite/integration/group-smoke/ -Dtest=org.jboss.as.test.smoke.embedded.demos.client.jms.JmsClientTestCase
>> -> runs the one test as expected
>> (I did -pl testsute/integration/group-smoke on the assumption that we will be moving to modules)
>> --
>>
>> cd testsuite/integration/group-basic
>> mvn install -Dtest=org.jboss.as.test.integration.deployment.classloading.ear.EarClassPath2TestCase
>> -> runs the one test as expected
>> --
>>
>> cd testsuite/integration/group-basic
>> mvn install
>> -> Runs those tests as expected
>> --
>
> There will be some slight changes coming up, such as moving the test classes into individual modules. They currently all live in testsuite/integration/ and are compiled by the individual sub-modules.
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Testsuite changes

Ondrej Zizka
Kabir Khan píše v Pá 25. 11. 2011 v 18:29 +0000:
I have some work in progress in my branch at: https://github.com/kabir/jboss-as/commits/rework-testsuite

I have pushed all the testsuite/integration classes into sub modules, and renamed group-smoke -> smoke etc.
That's basically what I planned as a next step after merging.

There are still a few scripts and xslt hanging around in the testsuite/integration aggregator. I have not investigated their usage.
Ant scripts and XSLT serve to copy AS around and to change configuration. Can be shared by multiple sub-modules, so I'd recommend keeping it in integration/ .

>From what I can see the pom2.xml files are not being used, so I've made no attempt to update those.
Yes, that's something to remove, I had it there to compare old & new harness - regressions and sets of tests run.

There are some TODOs in the pom's where I am not sure about stuff. I
I'll do / clean up the TODOs next week.
did not change the pom2.xml files, I'm not sure they are being used?
Nope.



Broken stuff:
It is impossible to run a single test from Eclipse, the arquillian.xml file is not found in Eclipse. Probably because this from testsuite/pom.xml does not get added to Eclipse's classpath:
                    <additionalClasspathElements combine.children="append">
                        <additionalClasspathElement>${basedir}/src/test/config/arq</additionalClasspathElement>
                    </additionalClasspathElements>
I'll check that.

so I think we should move these back to the resources directory.
Thomas does not like -DallTests running benchmark and stress, what would a good system property name be to enable running those?
We need to reenable the -DskipTests=true we had before
Rather than using xslt to modify the port offsets, we should change the socket bindings to use port-offset="${jboss.port.offset:0}" so that we can just pass a system property.

I won't be around much this weekend, so there is minimum chance of me crashing with you if any of you decide to do anything.  However, please make your changes public somewhere so I can pick them up if I feel the urge.


On 25 Nov 2011, at 13:56, Kabir Khan wrote:

> I have pushed Ondrej's work, and here is how to run it for developers (summarized from another mail from a few days ago):
>> 
>> mvn clean install 
>> -> runs smoke tests as expected
>> --
>> 
>> mvn clean install -DallTests
>> -> runs all tests as expected
>> --
>> 
>> mvn clean install -DallTests=true
>> -> runs all tests as expected
>> --
>> 
>> mvn install -pl testsuite/integration/group-smoke/ -Dtest=org.jboss.as.test.smoke.embedded.demos.client.jms.JmsClientTestCase
>> -> runs the one test as expected
>> (I did -pl testsute/integration/group-smoke on the assumption that we will be moving to modules)
>> --
>> 
>> cd testsuite/integration/group-basic 
>> mvn install -Dtest=org.jboss.as.test.integration.deployment.classloading.ear.EarClassPath2TestCase
>> -> runs the one test as expected
>> --
>> 
>> cd testsuite/integration/group-basic 
>> mvn install
>> -> Runs those tests as expected
>> --
> 
> There will be some slight changes coming up, such as moving the test classes into individual modules. They currently all live in testsuite/integration/ and are compiled by the individual sub-modules.
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev