Test Suite Changes

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Test Suite Changes

Stuart Douglas
I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly. 

Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:

./build.sh install -DallTests  - To do a full build with all tests
./integration-tests.sh -DallTests - To just run all integration tests 

Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.

Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.

Stuart


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

Thomas Diesler
Currently, the smoke tests don't run as part of the build. Is this really what you want?

https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/

-thomas

On 10/27/2011 06:17 AM, Stuart Douglas wrote:
I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly. 

Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:

./build.sh install -DallTests  - To do a full build with all tests
./integration-tests.sh -DallTests - To just run all integration tests 

Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.

Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.

Stuart



_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

-- 
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx 

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

kkhan
The smoke tests must be run as part of a standard build. Please fix.

If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
On 27 Oct 2011, at 09:48, Thomas Diesler wrote:

> Currently, the smoke tests don't run as part of the build. Is this really what you want?
>
> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
>
> -thomas
>
> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly.
>>
>> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>>
>> ./build.sh install -DallTests  - To do a full build with all tests
>> ./integration-tests.sh -DallTests - To just run all integration tests
>>
>> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>>
>> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>>
>> Stuart
>>
>> *https://docspace.corp.redhat.com/docs/DOC-74146
>> *https://docspace.corp.redhat.com/docs/DOC-69049 
>> *https://github.com/jbossas/jboss-as/pull/491
>>
>>
>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>>
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>
> --
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Thomas Diesler
> JBoss OSGi Lead
> JBoss, a division of Red Hat
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

Ondrej Zizka
Ok, seems like QA will have to carry the burden of disabling smoke tests whenever they want to run anything else - because Maven's profile activation logic really sucks.
Fixing.

Ondra



On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
The smoke tests must be run as part of a standard build. Please fix.

If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
On 27 Oct 2011, at 09:48, Thomas Diesler wrote:

> Currently, the smoke tests don't run as part of the build. Is this really what you want?
> 
> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
> 
> -thomas
> 
> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly. 
>> 
>> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>> 
>> ./build.sh install -DallTests  - To do a full build with all tests
>> ./integration-tests.sh -DallTests - To just run all integration tests 
>> 
>> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>> 
>> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>> 
>> Stuart
>> 
>> *https://docspace.corp.redhat.com/docs/DOC-74146
>> *https://docspace.corp.redhat.com/docs/DOC-69049 
>> *https://github.com/jbossas/jboss-as/pull/491
>> 
>> 
>> 
>> _______________________________________________
>> jboss-as7-dev mailing list
>> 
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> 
> -- 
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Thomas Diesler
> JBoss OSGi Lead
> JBoss, a division of Red Hat
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx 
> 
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

kkhan
One thing we talked about internally is that lthough not much thought has gone into what makes up the smoke tests, at least they are good for picking up problems in parsing and management layer which is very useful as a pre-pull request requirement.

We had a look around and found: http://maven.apache.org/guides/introduction/introduction-to-profiles.html which mentions: "As of Maven 3.0, profiles in the POM can also be activated based on properties from active profiles from the setttings.xml.".

Here is the rest of the discussion:

[27/10/2011 14:29:01] Stuart Douglas: but that requires a specific settings.xml
[27/10/2011 14:29:06] Kabir Khan: Oh
[27/10/2011 14:29:11] Stuart Douglas: although
[27/10/2011 14:29:31] Stuart Douglas: we coud just require all the QA people to have a specific settings.xml
[27/10/2011 14:31:43] Kabir Khan: Yeah, I've not really looked at the current setup so I don't know if it would work or not but if it does I'm all for it
[27/10/2011 14:36:42] Jason T. Greene: we bundle maven with as though
[27/10/2011 14:36:58] Jason T. Greene: and there is a settings.xml in there
[27/10/2011 14:37:41] Jason T. Greene: so we could put it all in there
[27/10/2011 14:38:07] Stuart Douglas: but really we need to settings.xml
[27/10/2011 14:38:16] Stuart Douglas: the the default developer one
[27/10/2011 14:38:21] Stuart Douglas: and the QE one
[27/10/2011 14:38:25] Jason T. Greene: sure
[27/10/2011 14:38:30] Jason T. Greene: thats reaosnable imo
[27/10/2011 14:38:32] Kabir Khan: qe-build.sh :-)
[27/10/2011 14:38:45] Stuart Douglas: and hopefully that will work better

On 27 Oct 2011, at 17:20, Ondrej Zizka wrote:

> Ok, seems like QA will have to carry the burden of disabling smoke tests whenever they want to run anything else - because Maven's profile activation logic really sucks.
> Fixing.
>
> Ondra
>
>
>
> On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
>> The smoke tests must be run as part of a standard build. Please fix.
>>
>> If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
>> On 27 Oct 2011, at 09:48, Thomas Diesler wrote:
>>
>> > Currently, the smoke tests don't run as part of the build. Is this really what you want?
>> >
>> >
>> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
>>
>> >
>> > -thomas
>> >
>> > On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>> >> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly.
>> >>
>> >> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>> >>
>> >> ./build.sh install -DallTests  - To do a full build with all tests
>> >> ./integration-tests.sh -DallTests - To just run all integration tests
>> >>
>> >> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>> >>
>> >> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>> >>
>> >> Stuart
>> >>
>> >> *
>> https://docspace.corp.redhat.com/docs/DOC-74146
>>
>> >> *
>> https://docspace.corp.redhat.com/docs/DOC-69049
>>  
>> >> *
>> https://github.com/jbossas/jboss-as/pull/491
>>
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> jboss-as7-dev mailing list
>> >>
>> >>
>> [hidden email]
>>
>> >>
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>> >
>> > --
>> > xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>> > Thomas Diesler
>> > JBoss OSGi Lead
>> > JBoss, a division of Red Hat
>> > xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>> >
>> > _______________________________________________
>> > jboss-as7-dev mailing list
>> >
>> [hidden email]
>>
>> >
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>>
>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>>
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

Shelly McGowan


The discussions surrounding the AS 7 test suite have been on-going since early August.  While these are considered QE requirements[1] the changes
benefit the test suite overall - or are intended to.  As Stuart mentioned below, Ondra's primary task it to ensure the usability of the test suite satisfies the development community and other users of the test suite.   Be sure to discuss with him directory, on this list, or you can file issues here:

https://issues.jboss.org/browse/AS7-2007

Documentation has been started:
https://docs.jboss.org/author/display/AS71/AS+7+Integration+Test+Suite+User+Guide
https://docs.jboss.org/author/display/AS71/AS+7+Testsuite+Developer+Guide

Shelly McGowan
JBoss, by Red Hat




----- Original Message -----
From: "Kabir Khan" <[hidden email]>
To: "Ondrej Zizka" <[hidden email]>
Cc: "[hidden email] Development" <[hidden email]>
Sent: Thursday, October 27, 2011 12:24:53 PM
Subject: Re: [jboss-as7-dev] Test Suite Changes

One thing we talked about internally is that lthough not much thought has gone into what makes up the smoke tests, at least they are good for picking up problems in parsing and management layer which is very useful as a pre-pull request requirement.

We had a look around and found: http://maven.apache.org/guides/introduction/introduction-to-profiles.html which mentions: "As of Maven 3.0, profiles in the POM can also be activated based on properties from active profiles from the setttings.xml.".

Here is the rest of the discussion:

[27/10/2011 14:29:01] Stuart Douglas: but that requires a specific settings.xml
[27/10/2011 14:29:06] Kabir Khan: Oh
[27/10/2011 14:29:11] Stuart Douglas: although
[27/10/2011 14:29:31] Stuart Douglas: we coud just require all the QA people to have a specific settings.xml
[27/10/2011 14:31:43] Kabir Khan: Yeah, I've not really looked at the current setup so I don't know if it would work or not but if it does I'm all for it
[27/10/2011 14:36:42] Jason T. Greene: we bundle maven with as though
[27/10/2011 14:36:58] Jason T. Greene: and there is a settings.xml in there
[27/10/2011 14:37:41] Jason T. Greene: so we could put it all in there
[27/10/2011 14:38:07] Stuart Douglas: but really we need to settings.xml
[27/10/2011 14:38:16] Stuart Douglas: the the default developer one
[27/10/2011 14:38:21] Stuart Douglas: and the QE one
[27/10/2011 14:38:25] Jason T. Greene: sure
[27/10/2011 14:38:30] Jason T. Greene: thats reaosnable imo
[27/10/2011 14:38:32] Kabir Khan: qe-build.sh :-)
[27/10/2011 14:38:45] Stuart Douglas: and hopefully that will work better

On 27 Oct 2011, at 17:20, Ondrej Zizka wrote:

> Ok, seems like QA will have to carry the burden of disabling smoke tests whenever they want to run anything else - because Maven's profile activation logic really sucks.
> Fixing.
>
> Ondra
>
>
>
> On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
>> The smoke tests must be run as part of a standard build. Please fix.
>>
>> If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
>> On 27 Oct 2011, at 09:48, Thomas Diesler wrote:
>>
>> > Currently, the smoke tests don't run as part of the build. Is this really what you want?
>> >
>> >
>> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
>>
>> >
>> > -thomas
>> >
>> > On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>> >> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly.
>> >>
>> >> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>> >>
>> >> ./build.sh install -DallTests  - To do a full build with all tests
>> >> ./integration-tests.sh -DallTests - To just run all integration tests
>> >>
>> >> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>> >>
>> >> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>> >>
>> >> Stuart
>> >>
>> >> *
>> https://docspace.corp.redhat.com/docs/DOC-74146
>>
>> >> *
>> https://docspace.corp.redhat.com/docs/DOC-69049
>>  
>> >> *
>> https://github.com/jbossas/jboss-as/pull/491
>>
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> jboss-as7-dev mailing list
>> >>
>> >>
>> [hidden email]
>>
>> >>
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>> >
>> > --
>> > xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>> > Thomas Diesler
>> > JBoss OSGi Lead
>> > JBoss, a division of Red Hat
>> > xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>> >
>> > _______________________________________________
>> > jboss-as7-dev mailing list
>> >
>> [hidden email]
>>
>> >
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>>
>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>>
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

Thomas Diesler
In reply to this post by Ondrej Zizka
Is it really that hard? Lets assume you have a set of mutually exclusive test suites. They could be organized like this:

-Dsuite=foo
-Dsuite=bar
-Dsuite=baz
-Dsuite=...

in maven this would look like

      <activation>
        <property>
          <name>suite</name>
          <value>foo</value>
        </property>
      </activation>

If 'foo' is the default (i.e. the smoke tests). You would also have profile that activates like this

      <activation>
        <property>
          <name>!suite</name>
        </property>
      </activation>

If the suites are organized my maven modules. You can AND them together with

mvn -pl testsuite-foo,
testsuite-bar,testsuite-baz,...

It should always be possible to navigate to a particular module and run a subset of tests like this

mvn -Dtest=osgi/*/*TestCase test

cheers
-thomas



On 10/27/2011 06:20 PM, Ondrej Zizka wrote:
Ok, seems like QA will have to carry the burden of disabling smoke tests whenever they want to run anything else - because Maven's profile activation logic really sucks.
Fixing.

Ondra



On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
The smoke tests must be run as part of a standard build. Please fix.

If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
On 27 Oct 2011, at 09:48, Thomas Diesler wrote:

> Currently, the smoke tests don't run as part of the build. Is this really what you want?
> 
> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
> 
> -thomas
> 
> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly. 
>> 
>> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>> 
>> ./build.sh install -DallTests  - To do a full build with all tests
>> ./integration-tests.sh -DallTests - To just run all integration tests 
>> 
>> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>> 
>> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>> 
>> Stuart
>> 
>> *https://docspace.corp.redhat.com/docs/DOC-74146
>> *https://docspace.corp.redhat.com/docs/DOC-69049 
>> *https://github.com/jbossas/jboss-as/pull/491
>> 
>> 
>> 
>> _______________________________________________
>> jboss-as7-dev mailing list
>> 
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> 
> -- 
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Thomas Diesler
> JBoss OSGi Lead
> JBoss, a division of Red Hat
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx 
> 
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


-- 
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx 

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

Ondrej Zizka
The thing is, the groups are not mutually exclusive.
Second, they are not (all) organized as maven modules.

But, afterall, it will be solved by QEs using the integration-tests.sh script which will hide these irregularities.

Ondra



Thomas Diesler píše v Pá 28. 10. 2011 v 10:03 +0200:
Is it really that hard? Lets assume you have a set of mutually exclusive test suites. They could be organized like this:

-Dsuite=foo
-Dsuite=bar
-Dsuite=baz
-Dsuite=...

in maven this would look like

      <activation>
        <property>
          <name>suite</name>
          <value>foo</value>
        </property>
      </activation>

If 'foo' is the default (i.e. the smoke tests). You would also have profile that activates like this

      <activation>
        <property>
          <name>!suite</name>
        </property>
      </activation>

If the suites are organized my maven modules. You can AND them together with

mvn -pl testsuite-foo,testsuite-bar,testsuite-baz,...

It should always be possible to navigate to a particular module and run a subset of tests like this

mvn -Dtest=osgi/*/*TestCase test

cheers
-thomas



On 10/27/2011 06:20 PM, Ondrej Zizka wrote:
Ok, seems like QA will have to carry the burden of disabling smoke tests whenever they want to run anything else - because Maven's profile activation logic really sucks.
Fixing.

Ondra



On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
The smoke tests must be run as part of a standard build. Please fix.

If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
On 27 Oct 2011, at 09:48, Thomas Diesler wrote:

> Currently, the smoke tests don't run as part of the build. Is this really what you want?
> 
> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
> 
> -thomas
> 
> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly. 
>> 
>> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>> 
>> ./build.sh install -DallTests  - To do a full build with all tests
>> ./integration-tests.sh -DallTests - To just run all integration tests 
>> 
>> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>> 
>> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>> 
>> Stuart
>> 
>> *https://docspace.corp.redhat.com/docs/DOC-74146
>> *https://docspace.corp.redhat.com/docs/DOC-69049 
>> *https://github.com/jbossas/jboss-as/pull/491
>> 
>> 
>> 
>> _______________________________________________
>> jboss-as7-dev mailing list
>> 
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> 
> -- 
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Thomas Diesler
> JBoss OSGi Lead
> JBoss, a division of Red Hat
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx 
> 
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


-- 
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx 

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

Ondrej Zizka
In reply to this post by kkhan
There's a pull request for this:

https://github.com/jbossas/jboss-as/pull/534

Ondra


On Thu, 2011-10-27 at 17:24 +0100, Kabir Khan wrote:
One thing we talked about internally is that lthough not much thought has gone into what makes up the smoke tests, at least they are good for picking up problems in parsing and management layer which is very useful as a pre-pull request requirement.

We had a look around and found: http://maven.apache.org/guides/introduction/introduction-to-profiles.html which mentions: "As of Maven 3.0, profiles in the POM can also be activated based on properties from active profiles from the setttings.xml.".

Here is the rest of the discussion:

[27/10/2011 14:29:01] Stuart Douglas: but that requires a specific settings.xml
[27/10/2011 14:29:06] Kabir Khan: Oh
[27/10/2011 14:29:11] Stuart Douglas: although
[27/10/2011 14:29:31] Stuart Douglas: we coud just require all the QA people to have a specific settings.xml
[27/10/2011 14:31:43] Kabir Khan: Yeah, I've not really looked at the current setup so I don't know if it would work or not but if it does I'm all for it
[27/10/2011 14:36:42] Jason T. Greene: we bundle maven with as though
[27/10/2011 14:36:58] Jason T. Greene: and there is a settings.xml in there
[27/10/2011 14:37:41] Jason T. Greene: so we could put it all in there
[27/10/2011 14:38:07] Stuart Douglas: but really we need to settings.xml
[27/10/2011 14:38:16] Stuart Douglas: the the default developer one
[27/10/2011 14:38:21] Stuart Douglas: and the QE one
[27/10/2011 14:38:25] Jason T. Greene: sure
[27/10/2011 14:38:30] Jason T. Greene: thats reaosnable imo
[27/10/2011 14:38:32] Kabir Khan: qe-build.sh :-)
[27/10/2011 14:38:45] Stuart Douglas: and hopefully that will work better

On 27 Oct 2011, at 17:20, Ondrej Zizka wrote:

> Ok, seems like QA will have to carry the burden of disabling smoke tests whenever they want to run anything else - because Maven's profile activation logic really sucks.
> Fixing.
> 
> Ondra
> 
> 
> 
> On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
>> The smoke tests must be run as part of a standard build. Please fix.
>> 
>> If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
>> On 27 Oct 2011, at 09:48, Thomas Diesler wrote:
>> 
>> > Currently, the smoke tests don't run as part of the build. Is this really what you want?
>> > 
>> > 
>> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
>> 
>> > 
>> > -thomas
>> > 
>> > On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>> >> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly. 
>> >> 
>> >> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>> >> 
>> >> ./build.sh install -DallTests  - To do a full build with all tests
>> >> ./integration-tests.sh -DallTests - To just run all integration tests 
>> >> 
>> >> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>> >> 
>> >> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>> >> 
>> >> Stuart
>> >> 
>> >> *
>> https://docspace.corp.redhat.com/docs/DOC-74146
>> 
>> >> *
>> https://docspace.corp.redhat.com/docs/DOC-69049
>>  
>> >> *
>> https://github.com/jbossas/jboss-as/pull/491
>> 
>> >> 
>> >> 
>> >> 
>> >> _______________________________________________
>> >> jboss-as7-dev mailing list
>> >> 
>> >> 
>> [hidden email]
>> 
>> >> 
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>> 
>> > 
>> > -- 
>> > xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>> > Thomas Diesler
>> > JBoss OSGi Lead
>> > JBoss, a division of Red Hat
>> > xxxxxxxxxxxxxxxxxxxxxxxxxxxx 
>> > 
>> > _______________________________________________
>> > jboss-as7-dev mailing list
>> > 
>> [hidden email]
>> 
>> > 
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>> 
>> 
>> 
>> _______________________________________________
>> jboss-as7-dev mailing list
>> 
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> 



_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

Thomas Diesler
In reply to this post by Ondrej Zizka
> will be solved by QEs using the integration-tests.sh script

The AS7 community project should not need to have a dedicated script to run test suites. What are these "irregularities" specifically? Perhaps you could present the problem and possible solution in a document that we can then discuss. 

On 10/31/2011 11:04 PM, Ondřej Žižka wrote:
The thing is, the groups are not mutually exclusive.
Second, they are not (all) organized as maven modules.

But, afterall, it will be solved by QEs using the integration-tests.sh script which will hide these irregularities.

Ondra




Thomas Diesler píše v Pá 28. 10. 2011 v 10:03 +0200:

Is it really that hard? Lets assume you have a set of mutually exclusive test suites. They could be organized like this:

-Dsuite=foo
-Dsuite=bar
-Dsuite=baz
-Dsuite=...

in maven this would look like

      <activation>
        <property>
          <name>suite</name>
          <value>foo</value>
        </property>
      </activation>

If 'foo' is the default (i.e. the smoke tests). You would also have profile that activates like this

      <activation>
        <property>
          <name>!suite</name>
        </property>
      </activation>

If the suites are organized my maven modules. You can AND them together with

mvn -pl testsuite-foo,testsuite-bar,testsuite-baz,...

It should always be possible to navigate to a particular module and run a subset of tests like this

mvn -Dtest=osgi/*/*TestCase test

cheers
-thomas



On 10/27/2011 06:20 PM, Ondrej Zizka wrote:

Ok, seems like QA will have to carry the burden of disabling smoke tests whenever they want to run anything else - because Maven's profile activation logic really sucks.
Fixing.

Ondra




On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
The smoke tests must be run as part of a standard build. Please fix.

If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
On 27 Oct 2011, at 09:48, Thomas Diesler wrote:

> Currently, the smoke tests don't run as part of the build. Is this really what you want?
> 
> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
> 
> -thomas
> 
> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly. 
>> 
>> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>> 
>> ./build.sh install -DallTests  - To do a full build with all tests
>> ./integration-tests.sh -DallTests - To just run all integration tests 
>> 
>> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>> 
>> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>> 
>> Stuart
>> 
>> *https://docspace.corp.redhat.com/docs/DOC-74146
>> *https://docspace.corp.redhat.com/docs/DOC-69049 
>> *https://github.com/jbossas/jboss-as/pull/491
>> 
>> 
>> 
>> _______________________________________________
>> jboss-as7-dev mailing list
>> 
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> 
> -- 
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Thomas Diesler
> JBoss OSGi Lead
> JBoss, a division of Red Hat
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx 
> 
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


-- 
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx 

-- 
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx 

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

Ondrej Zizka
Thomas Diesler píše v Út 01. 11. 2011 v 18:29 +0000:
> will be solved by QEs using the integration-tests.sh script

The AS7 community project should not need to have a dedicated script to run test suites.
It doesn't need - but the script makes it easier to use.

What are these "irregularities" specifically?
That some test group runs by default and and needs to be disabled, while others do not run by default and need to be enabled.
With a script, one would simly specify which group of tests should be run, instead of passing a handful of options to mvn.

Perhaps you could present the problem and possible solution in a document that we can then discuss. 
I use jira for that usually, but this issue was discussed in a mail thread within a closed group. I may fwd it here, or to you, if you like.

Ondra



On 10/31/2011 11:04 PM, Ondřej Žižka wrote:
The thing is, the groups are not mutually exclusive.
Second, they are not (all) organized as maven modules.

But, afterall, it will be solved by QEs using the integration-tests.sh script which will hide these irregularities.

Ondra



Thomas Diesler píše v Pá 28. 10. 2011 v 10:03 +0200:
Is it really that hard? Lets assume you have a set of mutually exclusive test suites. They could be organized like this:

-Dsuite=foo
-Dsuite=bar
-Dsuite=baz
-Dsuite=...

in maven this would look like

      <activation>
        <property>
          <name>suite</name>
          <value>foo</value>
        </property>
      </activation>

If 'foo' is the default (i.e. the smoke tests). You would also have profile that activates like this

      <activation>
        <property>
          <name>!suite</name>
        </property>
      </activation>

If the suites are organized my maven modules. You can AND them together with

mvn -pl testsuite-foo,testsuite-bar,testsuite-baz,...

It should always be possible to navigate to a particular module and run a subset of tests like this

mvn -Dtest=osgi/*/*TestCase test

cheers
-thomas



On 10/27/2011 06:20 PM, Ondrej Zizka wrote:
Ok, seems like QA will have to carry the burden of disabling smoke tests whenever they want to run anything else - because Maven's profile activation logic really sucks.
Fixing.

Ondra



On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
The smoke tests must be run as part of a standard build. Please fix.

If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
On 27 Oct 2011, at 09:48, Thomas Diesler wrote:

> Currently, the smoke tests don't run as part of the build. Is this really what you want?
> 
> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
> 
> -thomas
> 
> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly. 
>> 
>> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>> 
>> ./build.sh install -DallTests  - To do a full build with all tests
>> ./integration-tests.sh -DallTests - To just run all integration tests 
>> 
>> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>> 
>> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>> 
>> Stuart
>> 
>> *https://docspace.corp.redhat.com/docs/DOC-74146
>> *https://docspace.corp.redhat.com/docs/DOC-69049 
>> *https://github.com/jbossas/jboss-as/pull/491
>> 
>> 
>> 
>> _______________________________________________
>> jboss-as7-dev mailing list
>> 
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> 
> -- 
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Thomas Diesler
> JBoss OSGi Lead
> JBoss, a division of Red Hat
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx 
> 
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


-- 
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx 

-- 
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx 

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

kkhan
In reply to this post by Ondrej Zizka
Smoke tests are running by default now which is nice. However, a './build.sh clean install -DallTests=true' no longer runs all the tests, while they did last week. This is starting to get very confusing, all I want to do is to process some pull requests and run the tests.

If this really needs to keep changing all the time, a this-is-how-we-run-the-tests-today.txt file in the root folder would help :-)

On 27 Oct 2011, at 17:20, Ondrej Zizka wrote:

> Ok, seems like QA will have to carry the burden of disabling smoke tests whenever they want to run anything else - because Maven's profile activation logic really sucks.
> Fixing.
>
> Ondra
>
>
>
> On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
>> The smoke tests must be run as part of a standard build. Please fix.
>>
>> If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
>> On 27 Oct 2011, at 09:48, Thomas Diesler wrote:
>>
>> > Currently, the smoke tests don't run as part of the build. Is this really what you want?
>> >
>> >
>> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
>>
>> >
>> > -thomas
>> >
>> > On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>> >> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly.
>> >>
>> >> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>> >>
>> >> ./build.sh install -DallTests  - To do a full build with all tests
>> >> ./integration-tests.sh -DallTests - To just run all integration tests
>> >>
>> >> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>> >>
>> >> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>> >>
>> >> Stuart
>> >>
>> >> *
>> https://docspace.corp.redhat.com/docs/DOC-74146
>>
>> >> *
>> https://docspace.corp.redhat.com/docs/DOC-69049
>>  
>> >> *
>> https://github.com/jbossas/jboss-as/pull/491
>>
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> jboss-as7-dev mailing list
>> >>
>> >>
>> [hidden email]
>>
>> >>
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>> >
>> > --
>> > xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>> > Thomas Diesler
>> > JBoss OSGi Lead
>> > JBoss, a division of Red Hat
>> > xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>> >
>> > _______________________________________________
>> > jboss-as7-dev mailing list
>> >
>> [hidden email]
>>
>> >
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>>
>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>>
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

Darran Lofthouse
It would be nice if we could agree which parameters should actually work
and what they mean - the one that caught me out recently was when
-DskipTests=true was broken.

Regards,
Darran Lofthouse.



On 11/03/2011 11:34 AM, Kabir Khan wrote:

> Smoke tests are running by default now which is nice. However, a './build.sh clean install -DallTests=true' no longer runs all the tests, while they did last week. This is starting to get very confusing, all I want to do is to process some pull requests and run the tests.
>
> If this really needs to keep changing all the time, a this-is-how-we-run-the-tests-today.txt file in the root folder would help :-)
>
> On 27 Oct 2011, at 17:20, Ondrej Zizka wrote:
>
>> Ok, seems like QA will have to carry the burden of disabling smoke tests whenever they want to run anything else - because Maven's profile activation logic really sucks.
>> Fixing.
>>
>> Ondra
>>
>>
>>
>> On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
>>> The smoke tests must be run as part of a standard build. Please fix.
>>>
>>> If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
>>> On 27 Oct 2011, at 09:48, Thomas Diesler wrote:
>>>
>>>> Currently, the smoke tests don't run as part of the build. Is this really what you want?
>>>>
>>>>
>>> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
>>>
>>>>
>>>> -thomas
>>>>
>>>> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>>>>> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly.
>>>>>
>>>>> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>>>>>
>>>>> ./build.sh install -DallTests  - To do a full build with all tests
>>>>> ./integration-tests.sh -DallTests - To just run all integration tests
>>>>>
>>>>> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>>>>>
>>>>> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>>>>>
>>>>> Stuart
>>>>>
>>>>> *
>>> https://docspace.corp.redhat.com/docs/DOC-74146
>>>
>>>>> *
>>> https://docspace.corp.redhat.com/docs/DOC-69049
>>>
>>>>> *
>>> https://github.com/jbossas/jboss-as/pull/491
>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> jboss-as7-dev mailing list
>>>>>
>>>>>
>>> [hidden email]
>>>
>>>>>
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>
>>>>
>>>> --
>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>> Thomas Diesler
>>>> JBoss OSGi Lead
>>>> JBoss, a division of Red Hat
>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>
>>>> _______________________________________________
>>>> jboss-as7-dev mailing list
>>>>
>>> [hidden email]
>>>
>>>>
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> jboss-as7-dev mailing list
>>>
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>
>
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

Thomas Diesler
Yes, I would also like to see a document that describes how the
testsuite is (more importantly should be) organised and how to execute
individual test sections. The document should start with a list of
requirements.

The argument of "integration-tests.sh makes it easier" is not valid
IMHO. It is a diversion of an established standard and quite the
contrary from "makes it easier" IMHO. In short, start with requirements
and show whether and how they are met and where work still needs to be
done.

Refactoring the test suite into a clean structure, such that standard
mvn calls can be used to execute what we need would be the preferred
solution IMHO.

-thomas

On 11/03/2011 12:40 PM, Darran Lofthouse wrote:

> It would be nice if we could agree which parameters should actually work
> and what they mean - the one that caught me out recently was when
> -DskipTests=true was broken.
>
> Regards,
> Darran Lofthouse.
>
>
>
> On 11/03/2011 11:34 AM, Kabir Khan wrote:
>> Smoke tests are running by default now which is nice. However, a './build.sh clean install -DallTests=true' no longer runs all the tests, while they did last week. This is starting to get very confusing, all I want to do is to process some pull requests and run the tests.
>>
>> If this really needs to keep changing all the time, a this-is-how-we-run-the-tests-today.txt file in the root folder would help :-)
>>
>> On 27 Oct 2011, at 17:20, Ondrej Zizka wrote:
>>
>>> Ok, seems like QA will have to carry the burden of disabling smoke tests whenever they want to run anything else - because Maven's profile activation logic really sucks.
>>> Fixing.
>>>
>>> Ondra
>>>
>>>
>>>
>>> On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
>>>> The smoke tests must be run as part of a standard build. Please fix.
>>>>
>>>> If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
>>>> On 27 Oct 2011, at 09:48, Thomas Diesler wrote:
>>>>
>>>>> Currently, the smoke tests don't run as part of the build. Is this really what you want?
>>>>>
>>>>>
>>>> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
>>>>
>>>>> -thomas
>>>>>
>>>>> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>>>>>> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly.
>>>>>>
>>>>>> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>>>>>>
>>>>>> ./build.sh install -DallTests  - To do a full build with all tests
>>>>>> ./integration-tests.sh -DallTests - To just run all integration tests
>>>>>>
>>>>>> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>>>>>>
>>>>>> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>>>>>>
>>>>>> Stuart
>>>>>>
>>>>>> *
>>>> https://docspace.corp.redhat.com/docs/DOC-74146
>>>>
>>>>>> *
>>>> https://docspace.corp.redhat.com/docs/DOC-69049
>>>>
>>>>>> *
>>>> https://github.com/jbossas/jboss-as/pull/491
>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> jboss-as7-dev mailing list
>>>>>>
>>>>>>
>>>> [hidden email]
>>>>
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>
>>>>> --
>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>> Thomas Diesler
>>>>> JBoss OSGi Lead
>>>>> JBoss, a division of Red Hat
>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>>
>>>>> _______________________________________________
>>>>> jboss-as7-dev mailing list
>>>>>
>>>> [hidden email]
>>>>
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> jboss-as7-dev mailing list
>>>>
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

Shelly McGowan


So, the changes to the test suite were based on requirements.  The issues as I see it is that there is no
concensus (making everyone happy) which, in turn, has caused additional changes to the initial pull request[s].

It was anticipated in the requirements discussions, though, that the integration tests would be used for test purposes
other than during a build of AS7.

Thomas, have you looked at the requirements document links mentioned below?  

As far as bugs, fair enough.  They're being addressed as they are uncovered.  JIRAs should be filed.



Shelly



----- Original Message -----
From: "Thomas Diesler" <[hidden email]>
To: "Darran Lofthouse" <[hidden email]>
Cc: [hidden email]
Sent: Thursday, November 3, 2011 9:56:28 AM
Subject: Re: [jboss-as7-dev] Test Suite Changes

Yes, I would also like to see a document that describes how the
testsuite is (more importantly should be) organised and how to execute
individual test sections. The document should start with a list of
requirements.

The argument of "integration-tests.sh makes it easier" is not valid
IMHO. It is a diversion of an established standard and quite the
contrary from "makes it easier" IMHO. In short, start with requirements
and show whether and how they are met and where work still needs to be
done.

Refactoring the test suite into a clean structure, such that standard
mvn calls can be used to execute what we need would be the preferred
solution IMHO.

-thomas

On 11/03/2011 12:40 PM, Darran Lofthouse wrote:

> It would be nice if we could agree which parameters should actually work
> and what they mean - the one that caught me out recently was when
> -DskipTests=true was broken.
>
> Regards,
> Darran Lofthouse.
>
>
>
> On 11/03/2011 11:34 AM, Kabir Khan wrote:
>> Smoke tests are running by default now which is nice. However, a './build.sh clean install -DallTests=true' no longer runs all the tests, while they did last week. This is starting to get very confusing, all I want to do is to process some pull requests and run the tests.
>>
>> If this really needs to keep changing all the time, a this-is-how-we-run-the-tests-today.txt file in the root folder would help :-)
>>
>> On 27 Oct 2011, at 17:20, Ondrej Zizka wrote:
>>
>>> Ok, seems like QA will have to carry the burden of disabling smoke tests whenever they want to run anything else - because Maven's profile activation logic really sucks.
>>> Fixing.
>>>
>>> Ondra
>>>
>>>
>>>
>>> On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
>>>> The smoke tests must be run as part of a standard build. Please fix.
>>>>
>>>> If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
>>>> On 27 Oct 2011, at 09:48, Thomas Diesler wrote:
>>>>
>>>>> Currently, the smoke tests don't run as part of the build. Is this really what you want?
>>>>>
>>>>>
>>>> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
>>>>
>>>>> -thomas
>>>>>
>>>>> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>>>>>> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly.
>>>>>>
>>>>>> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>>>>>>
>>>>>> ./build.sh install -DallTests  - To do a full build with all tests
>>>>>> ./integration-tests.sh -DallTests - To just run all integration tests
>>>>>>
>>>>>> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>>>>>>
>>>>>> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>>>>>>
>>>>>> Stuart
>>>>>>
>>>>>> *
>>>> https://docspace.corp.redhat.com/docs/DOC-74146
>>>>
>>>>>> *
>>>> https://docspace.corp.redhat.com/docs/DOC-69049
>>>>
>>>>>> *
>>>> https://github.com/jbossas/jboss-as/pull/491
>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> jboss-as7-dev mailing list
>>>>>>
>>>>>>
>>>> [hidden email]
>>>>
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>
>>>>> --
>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>> Thomas Diesler
>>>>> JBoss OSGi Lead
>>>>> JBoss, a division of Red Hat
>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>>
>>>>> _______________________________________________
>>>>> jboss-as7-dev mailing list
>>>>>
>>>> [hidden email]
>>>>
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> jboss-as7-dev mailing list
>>>>
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

Shelly McGowan


>From August:

http://lists.jboss.org/pipermail/jboss-as7-dev/2011-August/003452.html

Shelly


----- Original Message -----
From: "Shelly McGowan" <[hidden email]>
To: "Thomas Diesler" <[hidden email]>
Cc: [hidden email], "Darran Lofthouse" <[hidden email]>
Sent: Thursday, November 3, 2011 10:24:46 AM
Subject: Re: [jboss-as7-dev] Test Suite Changes



So, the changes to the test suite were based on requirements.  The issues as I see it is that there is no
concensus (making everyone happy) which, in turn, has caused additional changes to the initial pull request[s].

It was anticipated in the requirements discussions, though, that the integration tests would be used for test purposes
other than during a build of AS7.

Thomas, have you looked at the requirements document links mentioned below?  

As far as bugs, fair enough.  They're being addressed as they are uncovered.  JIRAs should be filed.



Shelly



----- Original Message -----
From: "Thomas Diesler" <[hidden email]>
To: "Darran Lofthouse" <[hidden email]>
Cc: [hidden email]
Sent: Thursday, November 3, 2011 9:56:28 AM
Subject: Re: [jboss-as7-dev] Test Suite Changes

Yes, I would also like to see a document that describes how the
testsuite is (more importantly should be) organised and how to execute
individual test sections. The document should start with a list of
requirements.

The argument of "integration-tests.sh makes it easier" is not valid
IMHO. It is a diversion of an established standard and quite the
contrary from "makes it easier" IMHO. In short, start with requirements
and show whether and how they are met and where work still needs to be
done.

Refactoring the test suite into a clean structure, such that standard
mvn calls can be used to execute what we need would be the preferred
solution IMHO.

-thomas

On 11/03/2011 12:40 PM, Darran Lofthouse wrote:

> It would be nice if we could agree which parameters should actually work
> and what they mean - the one that caught me out recently was when
> -DskipTests=true was broken.
>
> Regards,
> Darran Lofthouse.
>
>
>
> On 11/03/2011 11:34 AM, Kabir Khan wrote:
>> Smoke tests are running by default now which is nice. However, a './build.sh clean install -DallTests=true' no longer runs all the tests, while they did last week. This is starting to get very confusing, all I want to do is to process some pull requests and run the tests.
>>
>> If this really needs to keep changing all the time, a this-is-how-we-run-the-tests-today.txt file in the root folder would help :-)
>>
>> On 27 Oct 2011, at 17:20, Ondrej Zizka wrote:
>>
>>> Ok, seems like QA will have to carry the burden of disabling smoke tests whenever they want to run anything else - because Maven's profile activation logic really sucks.
>>> Fixing.
>>>
>>> Ondra
>>>
>>>
>>>
>>> On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
>>>> The smoke tests must be run as part of a standard build. Please fix.
>>>>
>>>> If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
>>>> On 27 Oct 2011, at 09:48, Thomas Diesler wrote:
>>>>
>>>>> Currently, the smoke tests don't run as part of the build. Is this really what you want?
>>>>>
>>>>>
>>>> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
>>>>
>>>>> -thomas
>>>>>
>>>>> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>>>>>> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly.
>>>>>>
>>>>>> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>>>>>>
>>>>>> ./build.sh install -DallTests  - To do a full build with all tests
>>>>>> ./integration-tests.sh -DallTests - To just run all integration tests
>>>>>>
>>>>>> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>>>>>>
>>>>>> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>>>>>>
>>>>>> Stuart
>>>>>>
>>>>>> *
>>>> https://docspace.corp.redhat.com/docs/DOC-74146
>>>>
>>>>>> *
>>>> https://docspace.corp.redhat.com/docs/DOC-69049
>>>>
>>>>>> *
>>>> https://github.com/jbossas/jboss-as/pull/491
>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> jboss-as7-dev mailing list
>>>>>>
>>>>>>
>>>> [hidden email]
>>>>
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>
>>>>> --
>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>> Thomas Diesler
>>>>> JBoss OSGi Lead
>>>>> JBoss, a division of Red Hat
>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>>
>>>>> _______________________________________________
>>>>> jboss-as7-dev mailing list
>>>>>
>>>> [hidden email]
>>>>
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> jboss-as7-dev mailing list
>>>>
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

Kabir Khan-2
In reply to this post by Stuart Douglas
I still contend that it should be able to do this differently.

Simple mvn install (-DallTest=true) for developers.

Then qe should be able to use separate qe profiles/the antrun plugin/whatever with some trickery in the scripts qe uses to do your stuff. I have a few ideas which I can try out if I have some spare cycles and I am sure Paul Gier could help.

Sent from my Android phone using TouchDown (www.nitrodesk.com)


-----Original Message-----
From: Shelly McGowan [[hidden email]]
Received: Thursday, 03 Nov 2011, 14:35
To: Thomas Diesler [[hidden email]]
CC: [hidden email]
Subject: Re: [jboss-as7-dev] Test Suite Changes




>From August:

http://lists.jboss.org/pipermail/jboss-as7-dev/2011-August/003452.html

Shelly


----- Original Message -----
From: "Shelly McGowan" <[hidden email]>
To: "Thomas Diesler" <[hidden email]>
Cc: [hidden email], "Darran Lofthouse" <[hidden email]>
Sent: Thursday, November 3, 2011 10:24:46 AM
Subject: Re: [jboss-as7-dev] Test Suite Changes



So, the changes to the test suite were based on requirements.  The issues as I see it is that there is no
concensus (making everyone happy) which, in turn, has caused additional changes to the initial pull request[s].

It was anticipated in the requirements discussions, though, that the integration tests would be used for test purposes
other than during a build of AS7.

Thomas, have you looked at the requirements document links mentioned below?  

As far as bugs, fair enough.  They're being addressed as they are uncovered.  JIRAs should be filed.



Shelly



----- Original Message -----
From: "Thomas Diesler" <[hidden email]>
To: "Darran Lofthouse" <[hidden email]>
Cc: [hidden email]
Sent: Thursday, November 3, 2011 9:56:28 AM
Subject: Re: [jboss-as7-dev] Test Suite Changes

Yes, I would also like to see a document that describes how the
testsuite is (more importantly should be) organised and how to execute
individual test sections. The document should start with a list of
requirements.

The argument of "integration-tests.sh makes it easier" is not valid
IMHO. It is a diversion of an established standard and quite the
contrary from "makes it easier" IMHO. In short, start with requirements
and show whether and how they are met and where work still needs to be
done.

Refactoring the test suite into a clean structure, such that standard
mvn calls can be used to execute what we need would be the preferred
solution IMHO.

-thomas

On 11/03/2011 12:40 PM, Darran Lofthouse wrote:

> It would be nice if we could agree which parameters should actually work
> and what they mean - the one that caught me out recently was when
> -DskipTests=true was broken.
>
> Regards,
> Darran Lofthouse.
>
>
>
> On 11/03/2011 11:34 AM, Kabir Khan wrote:
>> Smoke tests are running by default now which is nice. However, a './build.sh clean install -DallTests=true' no longer runs all the tests, while they did last week. This is starting to get very confusing, all I want to do is to process some pull requests and run the tests.
>>
>> If this really needs to keep changing all the time, a this-is-how-we-run-the-tests-today.txt file in the root folder would help :-)
>>
>> On 27 Oct 2011, at 17:20, Ondrej Zizka wrote:
>>
>>> Ok, seems like QA will have to carry the burden of disabling smoke tests whenever they want to run anything else - because Maven's profile activation logic really sucks.
>>> Fixing.
>>>
>>> Ondra
>>>
>>>
>>>
>>> On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
>>>> The smoke tests must be run as part of a standard build. Please fix.
>>>>
>>>> If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
>>>> On 27 Oct 2011, at 09:48, Thomas Diesler wrote:
>>>>
>>>>> Currently, the smoke tests don't run as part of the build. Is this really what you want?
>>>>>
>>>>>
>>>> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
>>>>
>>>>> -thomas
>>>>>
>>>>> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>>>>>> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly.
>>>>>>
>>>>>> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>>>>>>
>>>>>> ./build.sh install -DallTests  - To do a full build with all tests
>>>>>> ./integration-tests.sh -DallTests - To just run all integration tests
>>>>>>
>>>>>> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>>>>>>
>>>>>> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>>>>>>
>>>>>> Stuart
>>>>>>
>>>>>> *
>>>> https://docspace.corp.redhat.com/docs/DOC-74146
>>>>
>>>>>> *
>>>> https://docspace.corp.redhat.com/docs/DOC-69049
>>>>
>>>>>> *
>>>> https://github.com/jbossas/jboss-as/pull/491
>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> jboss-as7-dev mailing list
>>>>>>
>>>>>>
>>>> [hidden email]
>>>>
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>
>>>>> --
>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>> Thomas Diesler
>>>>> JBoss OSGi Lead
>>>>> JBoss, a division of Red Hat
>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>>
>>>>> _______________________________________________
>>>>> jboss-as7-dev mailing list
>>>>>
>>>> [hidden email]
>>>>
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> jboss-as7-dev mailing list
>>>>
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

Scott Marlow
In reply to this post by kkhan
I've been using integration-tests.sh, which seems to run all the
integration tests.

Please don't remove this script, at least not until some equivalent way
of running the integration is working.

On 11/03/2011 07:34 AM, Kabir Khan wrote:
> Smoke tests are running by default now which is nice. However, a './build.sh clean install -DallTests=true' no longer runs all the tests, while they did last week. This is starting to get very confusing, all I want to do is to process some pull requests and run the tests.
>
> If this really needs to keep changing all the time, a this-is-how-we-run-the-tests-today.txt file in the root folder would help :-)

+1 for a 'this-is-how-we-run-the-tests-today.txt' idea.  :-)

>
> On 27 Oct 2011, at 17:20, Ondrej Zizka wrote:
>
>> Ok, seems like QA will have to carry the burden of disabling smoke tests whenever they want to run anything else - because Maven's profile activation logic really sucks.
>> Fixing.
>>
>> Ondra
>>
>>
>>
>> On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
>>> The smoke tests must be run as part of a standard build. Please fix.
>>>
>>> If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
>>> On 27 Oct 2011, at 09:48, Thomas Diesler wrote:
>>>
>>>> Currently, the smoke tests don't run as part of the build. Is this really what you want?
>>>>
>>>>
>>> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
>>>
>>>>
>>>> -thomas
>>>>
>>>> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>>>>> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly.
>>>>>
>>>>> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>>>>>
>>>>> ./build.sh install -DallTests  - To do a full build with all tests
>>>>> ./integration-tests.sh -DallTests - To just run all integration tests
>>>>>
>>>>> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>>>>>
>>>>> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>>>>>
>>>>> Stuart
>>>>>
>>>>> *
>>> https://docspace.corp.redhat.com/docs/DOC-74146
>>>
>>>>> *
>>> https://docspace.corp.redhat.com/docs/DOC-69049
>>>
>>>>> *
>>> https://github.com/jbossas/jboss-as/pull/491
>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> jboss-as7-dev mailing list
>>>>>
>>>>>
>>> [hidden email]
>>>
>>>>>
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>
>>>>
>>>> --
>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>> Thomas Diesler
>>>> JBoss OSGi Lead
>>>> JBoss, a division of Red Hat
>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>
>>>> _______________________________________________
>>>> jboss-as7-dev mailing list
>>>>
>>> [hidden email]
>>>
>>>>
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> jboss-as7-dev mailing list
>>>
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>
>
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

Shelly McGowan
In reply to this post by Kabir Khan-2


Kabir,

Communicate your ideas to Ondra (ozizka).  He can help you.

Shelly


----- Original Message -----
From: "Kabir Khan" <[hidden email]>
To: "thomas diesler" <[hidden email]>, [hidden email]
Cc: [hidden email]
Sent: Thursday, November 3, 2011 10:45:36 AM
Subject: Re: [jboss-as7-dev] Test Suite Changes

I still contend that it should be able to do this differently.

Simple mvn install (-DallTest=true) for developers.

Then qe should be able to use separate qe profiles/the antrun plugin/whatever with some trickery in the scripts qe uses to do your stuff. I have a few ideas which I can try out if I have some spare cycles and I am sure Paul Gier could help.

Sent from my Android phone using TouchDown (www.nitrodesk.com)


-----Original Message-----
From: Shelly McGowan [[hidden email]]
Received: Thursday, 03 Nov 2011, 14:35
To: Thomas Diesler [[hidden email]]
CC: [hidden email]
Subject: Re: [jboss-as7-dev] Test Suite Changes




>From August:

http://lists.jboss.org/pipermail/jboss-as7-dev/2011-August/003452.html

Shelly


----- Original Message -----
From: "Shelly McGowan" <[hidden email]>
To: "Thomas Diesler" <[hidden email]>
Cc: [hidden email], "Darran Lofthouse" <[hidden email]>
Sent: Thursday, November 3, 2011 10:24:46 AM
Subject: Re: [jboss-as7-dev] Test Suite Changes



So, the changes to the test suite were based on requirements.  The issues as I see it is that there is no
concensus (making everyone happy) which, in turn, has caused additional changes to the initial pull request[s].

It was anticipated in the requirements discussions, though, that the integration tests would be used for test purposes
other than during a build of AS7.

Thomas, have you looked at the requirements document links mentioned below?  

As far as bugs, fair enough.  They're being addressed as they are uncovered.  JIRAs should be filed.



Shelly



----- Original Message -----
From: "Thomas Diesler" <[hidden email]>
To: "Darran Lofthouse" <[hidden email]>
Cc: [hidden email]
Sent: Thursday, November 3, 2011 9:56:28 AM
Subject: Re: [jboss-as7-dev] Test Suite Changes

Yes, I would also like to see a document that describes how the
testsuite is (more importantly should be) organised and how to execute
individual test sections. The document should start with a list of
requirements.

The argument of "integration-tests.sh makes it easier" is not valid
IMHO. It is a diversion of an established standard and quite the
contrary from "makes it easier" IMHO. In short, start with requirements
and show whether and how they are met and where work still needs to be
done.

Refactoring the test suite into a clean structure, such that standard
mvn calls can be used to execute what we need would be the preferred
solution IMHO.

-thomas

On 11/03/2011 12:40 PM, Darran Lofthouse wrote:

> It would be nice if we could agree which parameters should actually work
> and what they mean - the one that caught me out recently was when
> -DskipTests=true was broken.
>
> Regards,
> Darran Lofthouse.
>
>
>
> On 11/03/2011 11:34 AM, Kabir Khan wrote:
>> Smoke tests are running by default now which is nice. However, a './build.sh clean install -DallTests=true' no longer runs all the tests, while they did last week. This is starting to get very confusing, all I want to do is to process some pull requests and run the tests.
>>
>> If this really needs to keep changing all the time, a this-is-how-we-run-the-tests-today.txt file in the root folder would help :-)
>>
>> On 27 Oct 2011, at 17:20, Ondrej Zizka wrote:
>>
>>> Ok, seems like QA will have to carry the burden of disabling smoke tests whenever they want to run anything else - because Maven's profile activation logic really sucks.
>>> Fixing.
>>>
>>> Ondra
>>>
>>>
>>>
>>> On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
>>>> The smoke tests must be run as part of a standard build. Please fix.
>>>>
>>>> If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
>>>> On 27 Oct 2011, at 09:48, Thomas Diesler wrote:
>>>>
>>>>> Currently, the smoke tests don't run as part of the build. Is this really what you want?
>>>>>
>>>>>
>>>> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
>>>>
>>>>> -thomas
>>>>>
>>>>> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>>>>>> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly.
>>>>>>
>>>>>> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>>>>>>
>>>>>> ./build.sh install -DallTests  - To do a full build with all tests
>>>>>> ./integration-tests.sh -DallTests - To just run all integration tests
>>>>>>
>>>>>> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>>>>>>
>>>>>> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>>>>>>
>>>>>> Stuart
>>>>>>
>>>>>> *
>>>> https://docspace.corp.redhat.com/docs/DOC-74146
>>>>
>>>>>> *
>>>> https://docspace.corp.redhat.com/docs/DOC-69049
>>>>
>>>>>> *
>>>> https://github.com/jbossas/jboss-as/pull/491
>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> jboss-as7-dev mailing list
>>>>>>
>>>>>>
>>>> [hidden email]
>>>>
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>
>>>>> --
>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>> Thomas Diesler
>>>>> JBoss OSGi Lead
>>>>> JBoss, a division of Red Hat
>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>>
>>>>> _______________________________________________
>>>>> jboss-as7-dev mailing list
>>>>>
>>>> [hidden email]
>>>>
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> jboss-as7-dev mailing list
>>>>
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

--
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
Thomas Diesler
JBoss OSGi Lead
JBoss, a division of Red Hat
xxxxxxxxxxxxxxxxxxxxxxxxxxxx

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Test Suite Changes

kkhan
ok, will do (tomorrow probably)
On 3 Nov 2011, at 15:02, Shelly McGowan wrote:

>
>
> Kabir,
>
> Communicate your ideas to Ondra (ozizka).  He can help you.
>
> Shelly
>
>
> ----- Original Message -----
> From: "Kabir Khan" <[hidden email]>
> To: "thomas diesler" <[hidden email]>, [hidden email]
> Cc: [hidden email]
> Sent: Thursday, November 3, 2011 10:45:36 AM
> Subject: Re: [jboss-as7-dev] Test Suite Changes
>
> I still contend that it should be able to do this differently.
>
> Simple mvn install (-DallTest=true) for developers.
>
> Then qe should be able to use separate qe profiles/the antrun plugin/whatever with some trickery in the scripts qe uses to do your stuff. I have a few ideas which I can try out if I have some spare cycles and I am sure Paul Gier could help.
>
> Sent from my Android phone using TouchDown (www.nitrodesk.com)
>
>
> -----Original Message-----
> From: Shelly McGowan [[hidden email]]
> Received: Thursday, 03 Nov 2011, 14:35
> To: Thomas Diesler [[hidden email]]
> CC: [hidden email]
> Subject: Re: [jboss-as7-dev] Test Suite Changes
>
>
>
>
>> From August:
>
> http://lists.jboss.org/pipermail/jboss-as7-dev/2011-August/003452.html
>
> Shelly
>
>
> ----- Original Message -----
> From: "Shelly McGowan" <[hidden email]>
> To: "Thomas Diesler" <[hidden email]>
> Cc: [hidden email], "Darran Lofthouse" <[hidden email]>
> Sent: Thursday, November 3, 2011 10:24:46 AM
> Subject: Re: [jboss-as7-dev] Test Suite Changes
>
>
>
> So, the changes to the test suite were based on requirements.  The issues as I see it is that there is no
> concensus (making everyone happy) which, in turn, has caused additional changes to the initial pull request[s].
>
> It was anticipated in the requirements discussions, though, that the integration tests would be used for test purposes
> other than during a build of AS7.
>
> Thomas, have you looked at the requirements document links mentioned below?  
>
> As far as bugs, fair enough.  They're being addressed as they are uncovered.  JIRAs should be filed.
>
>
>
> Shelly
>
>
>
> ----- Original Message -----
> From: "Thomas Diesler" <[hidden email]>
> To: "Darran Lofthouse" <[hidden email]>
> Cc: [hidden email]
> Sent: Thursday, November 3, 2011 9:56:28 AM
> Subject: Re: [jboss-as7-dev] Test Suite Changes
>
> Yes, I would also like to see a document that describes how the
> testsuite is (more importantly should be) organised and how to execute
> individual test sections. The document should start with a list of
> requirements.
>
> The argument of "integration-tests.sh makes it easier" is not valid
> IMHO. It is a diversion of an established standard and quite the
> contrary from "makes it easier" IMHO. In short, start with requirements
> and show whether and how they are met and where work still needs to be
> done.
>
> Refactoring the test suite into a clean structure, such that standard
> mvn calls can be used to execute what we need would be the preferred
> solution IMHO.
>
> -thomas
>
> On 11/03/2011 12:40 PM, Darran Lofthouse wrote:
>> It would be nice if we could agree which parameters should actually work
>> and what they mean - the one that caught me out recently was when
>> -DskipTests=true was broken.
>>
>> Regards,
>> Darran Lofthouse.
>>
>>
>>
>> On 11/03/2011 11:34 AM, Kabir Khan wrote:
>>> Smoke tests are running by default now which is nice. However, a './build.sh clean install -DallTests=true' no longer runs all the tests, while they did last week. This is starting to get very confusing, all I want to do is to process some pull requests and run the tests.
>>>
>>> If this really needs to keep changing all the time, a this-is-how-we-run-the-tests-today.txt file in the root folder would help :-)
>>>
>>> On 27 Oct 2011, at 17:20, Ondrej Zizka wrote:
>>>
>>>> Ok, seems like QA will have to carry the burden of disabling smoke tests whenever they want to run anything else - because Maven's profile activation logic really sucks.
>>>> Fixing.
>>>>
>>>> Ondra
>>>>
>>>>
>>>>
>>>> On Thu, 2011-10-27 at 11:36 +0100, Kabir Khan wrote:
>>>>> The smoke tests must be run as part of a standard build. Please fix.
>>>>>
>>>>> If QE have other requirements, which are not possible to combine with the above requirement maybe they should look into using alternate poms or the antrun plugin or something.
>>>>> On 27 Oct 2011, at 09:48, Thomas Diesler wrote:
>>>>>
>>>>>> Currently, the smoke tests don't run as part of the build. Is this really what you want?
>>>>>>
>>>>>>
>>>>> https://hudson.qa.jboss.com/hudson/view/JBossOSGi/job/jboss-as7-master/389/
>>>>>
>>>>>> -thomas
>>>>>>
>>>>>> On 10/27/2011 06:17 AM, Stuart Douglas wrote:
>>>>>>> I just just merged some changes to the test suite that QE requires for EAP (see below for a link to their requirements). As a result the manner in which the tests are run has changed slightly.
>>>>>>>
>>>>>>> Due to limitations in maven it is no longer possible to activate all tests using mvn install -DallTests, instead you should use either:
>>>>>>>
>>>>>>> ./build.sh install -DallTests  - To do a full build with all tests
>>>>>>> ./integration-tests.sh -DallTests - To just run all integration tests
>>>>>>>
>>>>>>> Running individual tests from an IDE or via mvn test -Dtest=MyTest should work as normal.
>>>>>>>
>>>>>>> Ondej Zizka (in CC) has been the person working on these changes, so if you have any questions or comments he is the person to talk to.
>>>>>>>
>>>>>>> Stuart
>>>>>>>
>>>>>>> *
>>>>> https://docspace.corp.redhat.com/docs/DOC-74146
>>>>>
>>>>>>> *
>>>>> https://docspace.corp.redhat.com/docs/DOC-69049
>>>>>
>>>>>>> *
>>>>> https://github.com/jbossas/jboss-as/pull/491
>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> jboss-as7-dev mailing list
>>>>>>>
>>>>>>>
>>>>> [hidden email]
>>>>>
>>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>>
>>>>>> --
>>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>>> Thomas Diesler
>>>>>> JBoss OSGi Lead
>>>>>> JBoss, a division of Red Hat
>>>>>> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>>>>>
>>>>>> _______________________________________________
>>>>>> jboss-as7-dev mailing list
>>>>>>
>>>>> [hidden email]
>>>>>
>>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> jboss-as7-dev mailing list
>>>>>
>>>>> [hidden email]
>>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>
>>> _______________________________________________
>>> jboss-as7-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>
> --
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Thomas Diesler
> JBoss OSGi Lead
> JBoss, a division of Red Hat
> xxxxxxxxxxxxxxxxxxxxxxxxxxxx
>
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
12