Where to put test cases for fixes?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Where to put test cases for fixes?

Brad Maxwell
Hello, is there any particular place we should be putting unit tests for
bugs that get fixed?
I see these various tests are laid out, I'm making a test case for an
issue related to a Sar deployment, which is in integration, but then it
isn't clear where it might go under there, currently I have it in smoke
because there are related tests there, but it seems like my test might
should be elsewhere.

Any thoughts?

* benchmark   - benchmarking integration tests
* compat      - compatibility tests (require special dependencies)
* domain      - domain management integration tests (require special
framework)
* integration - general integration tests
* stress      - stress integration tests

basic/      clust/       iiop/      manualmode/ multinode/ osgi/      
smoke/      xts/

Thanks,
Brad
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Where to put test cases for fixes?

Brian Stansberry
The distinction between basic and smoke is smoke runs every time anyone
does a normal build, while basic only runs if you pass some sort of
control flag to trigger it, such as -DallTests. The CI runs always pass
-DallTests, but devs just doing a build often won't.

We don't want smoke to take forever to run, so sticking a lot of tests
of small details in there isn't good. So, "can I deploy a basic sar and
access a few mbeans" sounds like a smoke test. A bunch of tests of
different sar deployments with different subtle characteristics -- those
belong in basic.

The other modules under testsuite/integration are all for tests that
require some sort of specialized environment to run.

On 10/29/12 11:53 AM, Brad Maxwell wrote:

> Hello, is there any particular place we should be putting unit tests for
> bugs that get fixed?
> I see these various tests are laid out, I'm making a test case for an
> issue related to a Sar deployment, which is in integration, but then it
> isn't clear where it might go under there, currently I have it in smoke
> because there are related tests there, but it seems like my test might
> should be elsewhere.
>
> Any thoughts?
>
> * benchmark   - benchmarking integration tests
> * compat      - compatibility tests (require special dependencies)
> * domain      - domain management integration tests (require special
> framework)
> * integration - general integration tests
> * stress      - stress integration tests
>
> basic/      clust/       iiop/      manualmode/ multinode/ osgi/
> smoke/      xts/
>
> Thanks,
> Brad
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>


--
Brian Stansberry
Principal Software Engineer
JBoss by Red Hat
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Where to put test cases for fixes?

Carlo de Wolf
And if at all possible stay clear of using sleep or other time sensitive
constructs.

Sooner or later those are going to blow up.

Carlo

On 10/29/2012 06:09 PM, Brian Stansberry wrote:

> The distinction between basic and smoke is smoke runs every time anyone
> does a normal build, while basic only runs if you pass some sort of
> control flag to trigger it, such as -DallTests. The CI runs always pass
> -DallTests, but devs just doing a build often won't.
>
> We don't want smoke to take forever to run, so sticking a lot of tests
> of small details in there isn't good. So, "can I deploy a basic sar and
> access a few mbeans" sounds like a smoke test. A bunch of tests of
> different sar deployments with different subtle characteristics -- those
> belong in basic.
>
> The other modules under testsuite/integration are all for tests that
> require some sort of specialized environment to run.
>
> On 10/29/12 11:53 AM, Brad Maxwell wrote:
>> Hello, is there any particular place we should be putting unit tests for
>> bugs that get fixed?
>> I see these various tests are laid out, I'm making a test case for an
>> issue related to a Sar deployment, which is in integration, but then it
>> isn't clear where it might go under there, currently I have it in smoke
>> because there are related tests there, but it seems like my test might
>> should be elsewhere.
>>
>> Any thoughts?
>>
>> * benchmark   - benchmarking integration tests
>> * compat      - compatibility tests (require special dependencies)
>> * domain      - domain management integration tests (require special
>> framework)
>> * integration - general integration tests
>> * stress      - stress integration tests
>>
>> basic/      clust/       iiop/      manualmode/ multinode/ osgi/
>> smoke/      xts/
>>
>> Thanks,
>> Brad
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Where to put test cases for fixes?

Aleksandar Kostadinov
+1 please, we are running the tests in all kinds of environments and
they tend to break sooner rather than later.

Carlo de Wolf wrote, On 29.10.2012 19:48 (EEST):

> And if at all possible stay clear of using sleep or other time sensitive
> constructs.
>
> Sooner or later those are going to blow up.
>
> Carlo
>
> On 10/29/2012 06:09 PM, Brian Stansberry wrote:
>> The distinction between basic and smoke is smoke runs every time anyone
>> does a normal build, while basic only runs if you pass some sort of
>> control flag to trigger it, such as -DallTests. The CI runs always pass
>> -DallTests, but devs just doing a build often won't.
>>
>> We don't want smoke to take forever to run, so sticking a lot of tests
>> of small details in there isn't good. So, "can I deploy a basic sar and
>> access a few mbeans" sounds like a smoke test. A bunch of tests of
>> different sar deployments with different subtle characteristics -- those
>> belong in basic.
>>
>> The other modules under testsuite/integration are all for tests that
>> require some sort of specialized environment to run.
>>
>> On 10/29/12 11:53 AM, Brad Maxwell wrote:
>>> Hello, is there any particular place we should be putting unit tests for
>>> bugs that get fixed?
>>> I see these various tests are laid out, I'm making a test case for an
>>> issue related to a Sar deployment, which is in integration, but then it
>>> isn't clear where it might go under there, currently I have it in smoke
>>> because there are related tests there, but it seems like my test might
>>> should be elsewhere.
>>>
>>> Any thoughts?
>>>
>>> * benchmark   - benchmarking integration tests
>>> * compat      - compatibility tests (require special dependencies)
>>> * domain      - domain management integration tests (require special
>>> framework)
>>> * integration - general integration tests
>>> * stress      - stress integration tests
>>>
>>> basic/      clust/       iiop/      manualmode/ multinode/ osgi/
>>> smoke/      xts/
>>>
>>> Thanks,
>>> Brad
>>> _______________________________________________
>>> jboss-as7-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>
>>
>
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev