Proposal for improving the the static modules management in Wildfly

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Proposal for improving the the static modules management in Wildfly

Andrea Battaglia
Hi all,

I would like to bring to your attention a proposal for improving the the static modules management in wildfly.
After a deep discussion with Sanne Grinovero about the static modules management in wildfly and about some interesting use cases, we tried to design out an improvement that can help both testers and developers.
Here are a couple of use cases of great interest:
 - Testing the new version of an existing module or testing its resources/dependencies: Sometimes it would be useful to test the integration of a different version of the new library into wildfly. Let's think about a patched version of hibernate-core or jgroups. This is supposed to be done without embedding the resource into the application, but delegating the library management to wildfly, instead.
 - Provisioning the static resources for third party libraries dynamically: They asked me to create a lot of static modules for third party libraries a lot of times. This happens each time (always) a bunch of applications use the same, shared set of third party libraries and, of course, each time a developer avoids bundling the libraries inside the binaries (ear/war). Let's think about apache commons, security libraries, or hadoop client libraries (see my github: https://github.com/andreabattaglia/cloudera-eap-modules, there you can find a huge set of the static modules for wildfly used to connect to hadoop services without need of embedding the jars into the applications binaries). This usecase applies to the jdbc drivers as well.

 My aim is to avoid wasting time to create the structure of each static module each time a developer needs  brand new one or a different version of it and to avoid wasting time googling for an existing solution. Usually, this activity requires to put a lot of effort in testing the static module as well. Moreover, sometimes , the solution to the same problem can be found on different sites or blogs in different flavours, which is misleading most of all for developers who have no experience in the management of static modules in wildfly.

 The idea: share the descriptor of the static module in a place accessible to all the developers. The tool used to save the descriptor files must be able to version them. Each time the wildfly deployer is asked for a static module which is not bunlded in the default package or is not already available locally, the module descriptor and its resource and dependencies will be built automatically.

 The solution design: wildfly is already able to download the components of a static module from an artifact repository, but this feature strictly relates to the resources.
 It would be useful to improve this feature in order to create a static module locally each time the module is requested by a subsystem or an application being deployed. The deployer should first check for the local module availability. If the module is not available locally, wildfly will first search for the module descriptor into the artifact repository (maven repo implementation), then create folder tree, download resources and repeat iteratively the algorithm for each dependency found in the module descriptor.
 This behaviour will automate the dynamic creation of the static module starting from the module descriptor.

 I'm keen to discuss the proposal with you and to help to design and develop a first implementation as well.

 Thanks a lot in advance for your time,

 Andrea
___________________________
Andrea Battaglia
EMEA Middleware Architect


Red Hat
Via Generale Gustavo Fara, 26
20124 MILANO
www.redhat.com
mobile: +39 328 1093652
fax: +39 02 6693111
email: [hidden email]


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Proposal for improving the the static modules management in Wildfly

jtgreene
Administrator
Hi Andrea,

Good timing! We are currently in a meeting discussing a provisioning infrastructure for WildFly. Expect to see some design proposals in this area on this list very soon. I think your idea is potentially doable on top of what we are thinking of building.

-Jason

On May 25, 2016, at 12:04 PM, Andrea Battaglia <[hidden email]> wrote:

Hi all,

I would like to bring to your attention a proposal for improving the the static modules management in wildfly.
After a deep discussion with Sanne Grinovero about the static modules management in wildfly and about some interesting use cases, we tried to design out an improvement that can help both testers and developers.
Here are a couple of use cases of great interest:
 - Testing the new version of an existing module or testing its resources/dependencies: Sometimes it would be useful to test the integration of a different version of the new library into wildfly. Let's think about a patched version of hibernate-core or jgroups. This is supposed to be done without embedding the resource into the application, but delegating the library management to wildfly, instead.
 - Provisioning the static resources for third party libraries dynamically: They asked me to create a lot of static modules for third party libraries a lot of times. This happens each time (always) a bunch of applications use the same, shared set of third party libraries and, of course, each time a developer avoids bundling the libraries inside the binaries (ear/war). Let's think about apache commons, security libraries, or hadoop client libraries (see my github: https://github.com/andreabattaglia/cloudera-eap-modules, there you can find a huge set of the static modules for wildfly used to connect to hadoop services without need of embedding the jars into the applications binaries). This usecase applies to the jdbc drivers as well.

 My aim is to avoid wasting time to create the structure of each static module each time a developer needs  brand new one or a different version of it and to avoid wasting time googling for an existing solution. Usually, this activity requires to put a lot of effort in testing the static module as well. Moreover, sometimes , the solution to the same problem can be found on different sites or blogs in different flavours, which is misleading most of all for developers who have no experience in the management of static modules in wildfly.

 The idea: share the descriptor of the static module in a place accessible to all the developers. The tool used to save the descriptor files must be able to version them. Each time the wildfly deployer is asked for a static module which is not bunlded in the default package or is not already available locally, the module descriptor and its resource and dependencies will be built automatically.

 The solution design: wildfly is already able to download the components of a static module from an artifact repository, but this feature strictly relates to the resources.
 It would be useful to improve this feature in order to create a static module locally each time the module is requested by a subsystem or an application being deployed. The deployer should first check for the local module availability. If the module is not available locally, wildfly will first search for the module descriptor into the artifact repository (maven repo implementation), then create folder tree, download resources and repeat iteratively the algorithm for each dependency found in the module descriptor.
 This behaviour will automate the dynamic creation of the static module starting from the module descriptor.

 I'm keen to discuss the proposal with you and to help to design and develop a first implementation as well.

 Thanks a lot in advance for your time,

 Andrea
___________________________
Andrea Battaglia
EMEA Middleware Architect
<rh_signature_logo.png>

Red Hat
Via Generale Gustavo Fara, 26
20124 MILANO
www.redhat.com
mobile: +39 328 1093652
fax: +39 02 6693111
email: [hidden email]

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Proposal for improving the the static modules management in Wildfly

Bob McWhirter
Somewhat related, maybe, would be to extend this to the entirety of the modules tree that already exist in the feature-packs.

If every module was Maven addressable, in some form, this could conceivably simplify the way WildFly Swarm stitches stuff together.

We currently have to prowl around and pluck stuff out of feature-packs, and remember which feature-packs provide which modules.

With this, we could point to a small set of root module.xml’s and let maven transitives find the subsequent module.xml files without having to consider feature-packs.

I would recommend, in this case, some flavor of feature-pack plugin to handle the current feature-pack disassembly and deployment into Central.

-Bob

On Thu, May 26, 2016 at 5:21 AM, Jason Greene <[hidden email]> wrote:
Hi Andrea,

Good timing! We are currently in a meeting discussing a provisioning infrastructure for WildFly. Expect to see some design proposals in this area on this list very soon. I think your idea is potentially doable on top of what we are thinking of building.

-Jason

On May 25, 2016, at 12:04 PM, Andrea Battaglia <[hidden email]> wrote:

Hi all,

I would like to bring to your attention a proposal for improving the the static modules management in wildfly.
After a deep discussion with Sanne Grinovero about the static modules management in wildfly and about some interesting use cases, we tried to design out an improvement that can help both testers and developers.
Here are a couple of use cases of great interest:
 - Testing the new version of an existing module or testing its resources/dependencies: Sometimes it would be useful to test the integration of a different version of the new library into wildfly. Let's think about a patched version of hibernate-core or jgroups. This is supposed to be done without embedding the resource into the application, but delegating the library management to wildfly, instead.
 - Provisioning the static resources for third party libraries dynamically: They asked me to create a lot of static modules for third party libraries a lot of times. This happens each time (always) a bunch of applications use the same, shared set of third party libraries and, of course, each time a developer avoids bundling the libraries inside the binaries (ear/war). Let's think about apache commons, security libraries, or hadoop client libraries (see my github: https://github.com/andreabattaglia/cloudera-eap-modules, there you can find a huge set of the static modules for wildfly used to connect to hadoop services without need of embedding the jars into the applications binaries). This usecase applies to the jdbc drivers as well.

 My aim is to avoid wasting time to create the structure of each static module each time a developer needs  brand new one or a different version of it and to avoid wasting time googling for an existing solution. Usually, this activity requires to put a lot of effort in testing the static module as well. Moreover, sometimes , the solution to the same problem can be found on different sites or blogs in different flavours, which is misleading most of all for developers who have no experience in the management of static modules in wildfly.

 The idea: share the descriptor of the static module in a place accessible to all the developers. The tool used to save the descriptor files must be able to version them. Each time the wildfly deployer is asked for a static module which is not bunlded in the default package or is not already available locally, the module descriptor and its resource and dependencies will be built automatically.

 The solution design: wildfly is already able to download the components of a static module from an artifact repository, but this feature strictly relates to the resources.
 It would be useful to improve this feature in order to create a static module locally each time the module is requested by a subsystem or an application being deployed. The deployer should first check for the local module availability. If the module is not available locally, wildfly will first search for the module descriptor into the artifact repository (maven repo implementation), then create folder tree, download resources and repeat iteratively the algorithm for each dependency found in the module descriptor.
 This behaviour will automate the dynamic creation of the static module starting from the module descriptor.

 I'm keen to discuss the proposal with you and to help to design and develop a first implementation as well.

 Thanks a lot in advance for your time,

 Andrea
___________________________
Andrea Battaglia
EMEA Middleware Architect
<rh_signature_logo.png>

Red Hat
Via Generale Gustavo Fara, 26
20124 MILANO
www.redhat.com
mobile: <a href="tel:%2B39%20328%201093652" value="+393281093652" target="_blank">+39 328 1093652
fax: <a href="tel:%2B39%2002%206693111" value="+39026693111" target="_blank">+39 02 6693111
email: [hidden email]

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev