[proposal] Move legacy extensions to a separate feature pack

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[proposal] Move legacy extensions to a separate feature pack

Jeff Mesnil
Hi,

We had some discussion recently about improving the release process & development of WildFly after the release of WildFly 11.

One of the proposal was to avoid building and testing code that is not changing often from WildFly every time the project is built.
A good candidate for that kind of code is the legacy extensions.

A legacy extension is a WildFly extension that is no longer usable (they have no runtime) but still provides a management model that can be in some case be migrated to newer extensions.
For example:
* web > undertow
* messaging > messaging-activemq
* jacorb > iiop-openjdk
Other legacy extensions are not migrated (cmp, jaxr and configadmin).
(jacorb legacy extension is a bit different as it is still usable by leveraging the runtime of iiop-openjdk aiui).

The legacy extensions are frozen. Their management model is frozen and they only require changes when the new corresponding extensions have some changes that required to be taken into account during migration.
But every time we build and release WildFly, we have to compile and test that unchanged code.

I started a proof of concept that provides a feature pack (wildfly-legacy-feature-pack)
that contains legacy extensions so they can be removed from WildFly codebase:

https://github.com/jmesnil/wildfly-legacy

These legacy extensions are provided by the wildfly-legacy-feature-pack that contains everything to install them in WildFly (module definitions and jars).
Wildfly’s own feature-pack then depends on it[1] so that the actual distribution of WildFly is not different from the current one.
But a lot of code can be removed from the wildfly codebase by moving these extensions to a separate project.

Since legacy extensions have no runtime and only a management model, they have few dependencies and relies only on the wildfly-core-feature-pack.

There is just an interesting problem with the migrate operation that some of these extensions define (web and messaging).
The code of the :migrate operation itself is not dependent on the new extensions as it only manipulates DMR operations.
However the functional test of the :migrate operation requires the new extension to be able to validate that the management model of the legacy extension has been properly migrated to a valid management model of the new extension.

This means that the legacy extensions depends on the new extensions (defined in wildfly) *with a test scope*[2].

This introduces a circular dependency between wildfly (that depends on the wildlfy-legacy-feature-pack) and the wildfly-legacy-feature-pack (that depends on wildfly extensions *with test scope*).
Since one of the dependency is in scope test, I worked around that by depending on the n-1 version of wildfly from wildlfy-legacy-feature-pack. It’s not ideal but in practice I’m not sure it is a big issue.
Maintainers of legacy extensions can build local snapshots of WildFly and use it as a dependency for the legacy-feature-pack when there might be some changes that impact the legacy extensions.

The only legacy extension that I could not move is the jacorb one. This one is tightly bound to the new iiop-openjdk extension. It inherits from its classes to provide its runtime emulation.

Moving the other legacy extensions still reduces the size of WildFly with 230-ish files removed and almost 50K lines:

$  git diff --shortstat  master legacy-feature-pack
 233 files changed, 14 insertions(+), 49832 deletions(-)

What do you think?
Once we have released WildFly 11, would it be worth moving legacy extensions to a separate feature pack?
 
[1] https://github.com/jmesnil/wildfly-legacy
[2] https://github.com/jmesnil/wildfly-legacy/blob/7b4c548033a3f652380567df60f8596d570c8db9/pom.xml#L305-L317
--
Jeff Mesnil
JBoss, a division of Red Hat
http://jmesnil.net/


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [proposal] Move legacy extensions to a separate feature pack

Brian Stansberry
Sounds good to me.

I don't want to make this sound like a priority or a blocker to what you describe, but it would be nice if we could figure out a clean way to reverse roles a bit when it comes to migrate ops. In the end it's the *new* subsystem that has a responsibility to support migration, and it's the new subsystem that has the potential to break migration. The management model of the old subsystem is fixed and known. We expose the migrate ops as API of the old subsystem for solid UX reasons, but really it's the new stuff that has to ensure they work correctly.

Can something like the undertow (extension) maven module in WF have a test-only dep on a legacy FP, and then use that to build up a legacy config and migrate it? I've forgotten if our subsystem test stuff supports having two separate extensions involved.



Brian Stansberry
Manager, Senior Principal Software Engineer
Red Hat

On Thu, Sep 28, 2017 at 3:11 AM, Jeff Mesnil <[hidden email]> wrote:
Hi,

We had some discussion recently about improving the release process & development of WildFly after the release of WildFly 11.

One of the proposal was to avoid building and testing code that is not changing often from WildFly every time the project is built.
A good candidate for that kind of code is the legacy extensions.

A legacy extension is a WildFly extension that is no longer usable (they have no runtime) but still provides a management model that can be in some case be migrated to newer extensions.
For example:
* web > undertow
* messaging > messaging-activemq
* jacorb > iiop-openjdk
Other legacy extensions are not migrated (cmp, jaxr and configadmin).
(jacorb legacy extension is a bit different as it is still usable by leveraging the runtime of iiop-openjdk aiui).

The legacy extensions are frozen. Their management model is frozen and they only require changes when the new corresponding extensions have some changes that required to be taken into account during migration.
But every time we build and release WildFly, we have to compile and test that unchanged code.

I started a proof of concept that provides a feature pack (wildfly-legacy-feature-pack)
that contains legacy extensions so they can be removed from WildFly codebase:

https://github.com/jmesnil/wildfly-legacy

These legacy extensions are provided by the wildfly-legacy-feature-pack that contains everything to install them in WildFly (module definitions and jars).
Wildfly’s own feature-pack then depends on it[1] so that the actual distribution of WildFly is not different from the current one.
But a lot of code can be removed from the wildfly codebase by moving these extensions to a separate project.

Since legacy extensions have no runtime and only a management model, they have few dependencies and relies only on the wildfly-core-feature-pack.

There is just an interesting problem with the migrate operation that some of these extensions define (web and messaging).
The code of the :migrate operation itself is not dependent on the new extensions as it only manipulates DMR operations.
However the functional test of the :migrate operation requires the new extension to be able to validate that the management model of the legacy extension has been properly migrated to a valid management model of the new extension.

This means that the legacy extensions depends on the new extensions (defined in wildfly) *with a test scope*[2].

This introduces a circular dependency between wildfly (that depends on the wildlfy-legacy-feature-pack) and the wildfly-legacy-feature-pack (that depends on wildfly extensions *with test scope*).
Since one of the dependency is in scope test, I worked around that by depending on the n-1 version of wildfly from wildlfy-legacy-feature-pack. It’s not ideal but in practice I’m not sure it is a big issue.
Maintainers of legacy extensions can build local snapshots of WildFly and use it as a dependency for the legacy-feature-pack when there might be some changes that impact the legacy extensions.

The only legacy extension that I could not move is the jacorb one. This one is tightly bound to the new iiop-openjdk extension. It inherits from its classes to provide its runtime emulation.

Moving the other legacy extensions still reduces the size of WildFly with 230-ish files removed and almost 50K lines:

$  git diff --shortstat  master legacy-feature-pack
 233 files changed, 14 insertions(+), 49832 deletions(-)

What do you think?
Once we have released WildFly 11, would it be worth moving legacy extensions to a separate feature pack?

[1] https://github.com/jmesnil/wildfly-legacy
[2] https://github.com/jmesnil/wildfly-legacy/blob/7b4c548033a3f652380567df60f8596d570c8db9/pom.xml#L305-L317
--
Jeff Mesnil
JBoss, a division of Red Hat
http://jmesnil.net/


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [proposal] Move legacy extensions to a separate feature pack

Kabir Khan-2

> On 28 Sep 2017, at 17:13, Brian Stansberry <[hidden email]> wrote:
>
> Sounds good to me.
>
> I don't want to make this sound like a priority or a blocker to what you describe, but it would be nice if we could figure out a clean way to reverse roles a bit when it comes to migrate ops. In the end it's the *new* subsystem that has a responsibility to support migration, and it's the new subsystem that has the potential to break migration. The management model of the old subsystem is fixed and known. We expose the migrate ops as API of the old subsystem for solid UX reasons, but really it's the new stuff that has to ensure they work correctly.
>
> Can something like the undertow (extension) maven module in WF have a test-only dep on a legacy FP, and then use that to build up a legacy config and migrate it? I've forgotten if our subsystem test stuff supports having two separate extensions involved.
It supports adding more extensions via an AdditionalInitialization, e.g.: https://github.com/wildfly/wildfly-core/blob/master/subsystem-test/tests/src/test/java/org/jboss/as/subsystem/test/extrasubsystem/subsystem/ExtraSubsystemTestCase.java#L196

>
>
>
> Brian Stansberry
> Manager, Senior Principal Software Engineer
> Red Hat
>
> On Thu, Sep 28, 2017 at 3:11 AM, Jeff Mesnil <[hidden email]> wrote:
> Hi,
>
> We had some discussion recently about improving the release process & development of WildFly after the release of WildFly 11.
>
> One of the proposal was to avoid building and testing code that is not changing often from WildFly every time the project is built.
> A good candidate for that kind of code is the legacy extensions.
>
> A legacy extension is a WildFly extension that is no longer usable (they have no runtime) but still provides a management model that can be in some case be migrated to newer extensions.
> For example:
> * web > undertow
> * messaging > messaging-activemq
> * jacorb > iiop-openjdk
> Other legacy extensions are not migrated (cmp, jaxr and configadmin).
> (jacorb legacy extension is a bit different as it is still usable by leveraging the runtime of iiop-openjdk aiui).
>
> The legacy extensions are frozen. Their management model is frozen and they only require changes when the new corresponding extensions have some changes that required to be taken into account during migration.
> But every time we build and release WildFly, we have to compile and test that unchanged code.
>
> I started a proof of concept that provides a feature pack (wildfly-legacy-feature-pack)
> that contains legacy extensions so they can be removed from WildFly codebase:
>
> https://github.com/jmesnil/wildfly-legacy
>
> These legacy extensions are provided by the wildfly-legacy-feature-pack that contains everything to install them in WildFly (module definitions and jars).
> Wildfly’s own feature-pack then depends on it[1] so that the actual distribution of WildFly is not different from the current one.
> But a lot of code can be removed from the wildfly codebase by moving these extensions to a separate project.
>
> Since legacy extensions have no runtime and only a management model, they have few dependencies and relies only on the wildfly-core-feature-pack.
>
> There is just an interesting problem with the migrate operation that some of these extensions define (web and messaging).
> The code of the :migrate operation itself is not dependent on the new extensions as it only manipulates DMR operations.
> However the functional test of the :migrate operation requires the new extension to be able to validate that the management model of the legacy extension has been properly migrated to a valid management model of the new extension.
>
> This means that the legacy extensions depends on the new extensions (defined in wildfly) *with a test scope*[2].
>
> This introduces a circular dependency between wildfly (that depends on the wildlfy-legacy-feature-pack) and the wildfly-legacy-feature-pack (that depends on wildfly extensions *with test scope*).
> Since one of the dependency is in scope test, I worked around that by depending on the n-1 version of wildfly from wildlfy-legacy-feature-pack. It’s not ideal but in practice I’m not sure it is a big issue.
> Maintainers of legacy extensions can build local snapshots of WildFly and use it as a dependency for the legacy-feature-pack when there might be some changes that impact the legacy extensions.
>
> The only legacy extension that I could not move is the jacorb one. This one is tightly bound to the new iiop-openjdk extension. It inherits from its classes to provide its runtime emulation.
>
> Moving the other legacy extensions still reduces the size of WildFly with 230-ish files removed and almost 50K lines:
>
> $  git diff --shortstat  master legacy-feature-pack
>  233 files changed, 14 insertions(+), 49832 deletions(-)
>
> What do you think?
> Once we have released WildFly 11, would it be worth moving legacy extensions to a separate feature pack?
>
> [1] https://github.com/jmesnil/wildfly-legacy
> [2] https://github.com/jmesnil/wildfly-legacy/blob/7b4c548033a3f652380567df60f8596d570c8db9/pom.xml#L305-L317
> --
> Jeff Mesnil
> JBoss, a division of Red Hat
> http://jmesnil.net/
>
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [proposal] Move legacy extensions to a separatefeature pack

Tomaž Cerar-2
In reply to this post by Brian Stansberry
  • Can something like the undertow (extension) maven module in WF have a test-only dep on a legacy FP, and then use that to build up a legacy config and migrate it? I've forgotten if our subsystem test stuff supports having two separate extensions involved.

Yes, that should be doable.

 

 

 

 

From: [hidden email]
Sent: četrtek, 28. september 2017 17:26
To: [hidden email]
Cc: [hidden email]
Subject: Re: [wildfly-dev] [proposal] Move legacy extensions to a separatefeature pack

 

Sounds good to me.

 

I don't want to make this sound like a priority or a blocker to what you describe, but it would be nice if we could figure out a clean way to reverse roles a bit when it comes to migrate ops. In the end it's the *new* subsystem that has a responsibility to support migration, and it's the new subsystem that has the potential to break migration. The management model of the old subsystem is fixed and known. We expose the migrate ops as API of the old subsystem for solid UX reasons, but really it's the new stuff that has to ensure they work correctly.

 

Can something like the undertow (extension) maven module in WF have a test-only dep on a legacy FP, and then use that to build up a legacy config and migrate it? I've forgotten if our subsystem test stuff supports having two separate extensions involved.

 

 


Brian Stansberry

Manager, Senior Principal Software Engineer

Red Hat

 

On Thu, Sep 28, 2017 at 3:11 AM, Jeff Mesnil <[hidden email]> wrote:

Hi,

We had some discussion recently about improving the release process & development of WildFly after the release of WildFly 11.

One of the proposal was to avoid building and testing code that is not changing often from WildFly every time the project is built.
A good candidate for that kind of code is the legacy extensions.

A legacy extension is a WildFly extension that is no longer usable (they have no runtime) but still provides a management model that can be in some case be migrated to newer extensions.
For example:
* web > undertow
* messaging > messaging-activemq
* jacorb > iiop-openjdk
Other legacy extensions are not migrated (cmp, jaxr and configadmin).
(jacorb legacy extension is a bit different as it is still usable by leveraging the runtime of iiop-openjdk aiui).

The legacy extensions are frozen. Their management model is frozen and they only require changes when the new corresponding extensions have some changes that required to be taken into account during migration.
But every time we build and release WildFly, we have to compile and test that unchanged code.

I started a proof of concept that provides a feature pack (wildfly-legacy-feature-pack)
that contains legacy extensions so they can be removed from WildFly codebase:

https://github.com/jmesnil/wildfly-legacy

These legacy extensions are provided by the wildfly-legacy-feature-pack that contains everything to install them in WildFly (module definitions and jars).
Wildfly’s own feature-pack then depends on it[1] so that the actual distribution of WildFly is not different from the current one.
But a lot of code can be removed from the wildfly codebase by moving these extensions to a separate project.

Since legacy extensions have no runtime and only a management model, they have few dependencies and relies only on the wildfly-core-feature-pack.

There is just an interesting problem with the migrate operation that some of these extensions define (web and messaging).
The code of the :migrate operation itself is not dependent on the new extensions as it only manipulates DMR operations.
However the functional test of the :migrate operation requires the new extension to be able to validate that the management model of the legacy extension has been properly migrated to a valid management model of the new extension.

This means that the legacy extensions depends on the new extensions (defined in wildfly) *with a test scope*[2].

This introduces a circular dependency between wildfly (that depends on the wildlfy-legacy-feature-pack) and the wildfly-legacy-feature-pack (that depends on wildfly extensions *with test scope*).
Since one of the dependency is in scope test, I worked around that by depending on the n-1 version of wildfly from wildlfy-legacy-feature-pack. It’s not ideal but in practice I’m not sure it is a big issue.
Maintainers of legacy extensions can build local snapshots of WildFly and use it as a dependency for the legacy-feature-pack when there might be some changes that impact the legacy extensions.

The only legacy extension that I could not move is the jacorb one. This one is tightly bound to the new iiop-openjdk extension. It inherits from its classes to provide its runtime emulation.

Moving the other legacy extensions still reduces the size of WildFly with 230-ish files removed and almost 50K lines:

$  git diff --shortstat  master legacy-feature-pack
 233 files changed, 14 insertions(+), 49832 deletions(-)

What do you think?
Once we have released WildFly 11, would it be worth moving legacy extensions to a separate feature pack?

[1] https://github.com/jmesnil/wildfly-legacy
[2] https://github.com/jmesnil/wildfly-legacy/blob/7b4c548033a3f652380567df60f8596d570c8db9/pom.xml#L305-L317
--
Jeff Mesnil
JBoss, a division of Red Hat
http://jmesnil.net/


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

 

 


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [proposal] Move legacy extensions to a separatefeature pack

Jeff Mesnil


> On 28 Sep 2017, at 22:14, Tomaž Cerar <[hidden email]> wrote:
>
> • Can something like the undertow (extension) maven module in WF have a test-only dep on a legacy FP, and then use that to build up a legacy config and migrate it? I've forgotten if our subsystem test stuff supports having two separate extensions involved.
> Yes, that should be doable.

Thanks, that was a good suggestion and it works fine.

The migration tests were already using the subsystem-test mechanism to load both legacy and new extensions from the same test.
I just moved the migration tests from the legacy code to the new one and added a test dependency to the legacy extension.
This way, the legacy feature pack no longer needs any dependency to wildfly artifacts.
It makes sense to run the migration tests from the new extensions: it’s up to them to check that they can still allow migration from the legacy ones when they are updated.

After WildFly 11 is released, I’ll create a new wildly-legacy project in GitHub and open a PR to move the legacy extensions to it.

jeff


--
Jeff Mesnil
JBoss, a division of Red Hat
http://jmesnil.net/


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev