Updating WildFly instance with the Provisioning Tool

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Updating WildFly instance with the Provisioning Tool

Emmanuel Hugonnet
Hey guys,

TL;DR
I've been experimenting with Alexey to update a customized provisioned server using the provisioning tool [1].
I'm using the syncing operations [2] that I created a while back by porting the domain synchronization operations to standalone (to
synchronize standalone instances in a cloud environment).
I'm looking for some feedback on this approach.
Cheers
Emmanuel

[1]: https://github.com/ehsavoie/pm/tree/upgrade-feature-pack
[2]: https://github.com/ehsavoie/wildfly-core/tree/model_diff

----
full version:


  Updating WildFly


    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#terminology>Terminology

Updating is the process of applying a fix pack that will increment the micro version. There should be no compatiblity issue. Upgrading is
the transition of a minor version, compatiblity should be available but there are a lot more changes.

While the mecanisms discussed here are general, they might need more refinement for an upgrade.


    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#use-case>Use case

The use case is quite simple: *I have version 1.0.0 installed and i want to update to 1.0.1 but I have locally customized my server and i’d
like to keep those changes*.
We have several local elements to take into account:

  *

    filesystem changes (files added, removed or deleted).

  *

    configuration changes.

The basic idea is to diff the existing instance with a pure new installation of the same version then apply those changes to a new
provisioned version instance for staging.
We can keep it at the basic filesystem approach with some simple merge strategy (theirs, ours).
We can use the plugin to go into more details. For exemple using the model diff between standalone Wildfly instances we can create a cli
script to reconfigure the upgraded instance in a post-installation step.


    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#diffing-the-filesystem>Diffing the filesystem

The idea is to compare the instance to be upgraded with one instance provisioned with the same feature packs as the one we want to upgrade.
The plugin will provide a list of files or regexp to be excluded.
Each file will be hashed and we compare the hash + the relative path to deduce deleted, modified or added files.
For textual files we can provide a diff (and the means to apply it), but maybe that should be for a later version as some kind of
interaction with the user might be required.


    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#wildfly-standalone-plugin>WildFly standalone
    plugin

This is a specialization of the upgrading algorithm:

  *

    Filtering out some of the 'unimportant' files (tmp, logs).

  *

    Creating diff of textual files (for example the realm properties) which will be applied (merging strategy à la git).

  *

    Using an embedded standalone it creates a jboss-cli script to reconfigure the server (adding/removing extensions and reconfiguring
    subsystems).

  *

    Deleting files that were removed.

This is done on a staging upgraded instance before being copied over the old instance.
I have added a diff/sync operation in standalone that is quite similar to what happens when a slave HC connects to the DC. Thus I start the
current installation, and connect to it from an embedded server using the initial configuration and diff the models.
This is 'experimental' but it works nicely (I was able to 'upgrade' from the standalone.xml of wildfly-core to the standalone-full.xml of
wildfly).
I’m talking only about the model part, I leave the files to the filesystem 'diffing' but it wiill work with managed deployments are those
are added by the filesystem part and then the deployment reference is added in the model.
For a future version of the tooling/plugin we might look for a way to interract more with the user (for example for applying the textual
diffs to choose what to do per file instead of globally).
Also currently the filters for excluding files are defined by the plugin but we could enrich them from the tooling also.


    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#producing-an-update-feature-pack>Producing an
    update feature pack

From the initial upgrade mecanism Alexey has seen the potential to create a feature pack instead of my initial format.
Currently i’m able to create and installa a feature-pack that will supersede the initial installation with its own local modifications.
Thus from my customized instance I can produce a feature pack that will allow me to reinstall the same instance. Maybe this can be also use
to produce upgrade feature pack for patching.


    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#wildfly-domain-mode>WildFly domain mode

Domain mode is a bit more complex, and we need to think how to manager the model changes.
Those can be at the domain level or the host level and depending on the instance target we would need to get the changes from the domain.xml
or/and the host.xml.
I’m thinking about applying the same strategy as what was done for standalone : aka expose the sync operations to an embedded HC.



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Updating WildFly instance with the Provisioning Tool

Brian Stansberry
Thanks for this!

I want to make sure that what comes out of this are clear decisions about what we're willing to handle in terms of various sorts of conflicts. What kinds of things the tool is responsible for solving vs what the user has to resolve, what tooling config settings we support for tweaking what the tool will do for the user, etc.

When I say conflicts I don't just mean file conflicts; I also mean use case conflicts. What's the right way / wrong way to do things and what combinations are ok.

Part of what makes this interesting is the provisioning tool itself is meant to be used for setting up the installation the way the user wants it. So in many many cases if there is a diff it's because the user bypassed the tool. Of course that's going to be very common, particularly when it comes to the config files, where people have been using tools like the CLI to set up their configs for years. Manually adding, removing and modifying modules, hopefully we can wean people off that and get them to use the tool, but still, people won't. 

The 3-way diffing approach you describe sounds like it should be flexible enough to handle various permutations.

Re: the "update feature pack" what happens if the user has produced and stored one of those, then updates their base configuration, and then applies that update feature pack. So, the user has taken an install provisioned by the tool, with standalone.xml produced according to the inputs to the tool. Then modified that via CLI/HAL and saved those mods in an update feature pack. Then taken the bulk of those mods, adjusted their inputs to the provisioning tool such that tool produces the desired config, does an installation and wishes to apply that update feature pack?

Detail question:

"Filtering out some of the 'unimportant' files (tmp, logs)."

What does that mean? The tmp and logs dirs end up empty in the new installation? That might be ok for tmp but for logs it sounds wrong. I'm not sure it's necessary for tmp.


Brian Stansberry
Manager, Senior Principal Software Engineer
Red Hat

On Mon, Sep 25, 2017 at 4:06 AM, Emmanuel Hugonnet <[hidden email]> wrote:
Hey guys,

TL;DR
I've been experimenting with Alexey to update a customized provisioned server using the provisioning tool [1].
I'm using the syncing operations [2] that I created a while back by porting the domain synchronization operations to standalone (to
synchronize standalone instances in a cloud environment).
I'm looking for some feedback on this approach.
Cheers
Emmanuel

[1]: https://github.com/ehsavoie/pm/tree/upgrade-feature-pack
[2]: https://github.com/ehsavoie/wildfly-core/tree/model_diff

----
full version:


  Updating WildFly


    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#terminology>Terminology

Updating is the process of applying a fix pack that will increment the micro version. There should be no compatiblity issue. Upgrading is
the transition of a minor version, compatiblity should be available but there are a lot more changes.

While the mecanisms discussed here are general, they might need more refinement for an upgrade.


    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#use-case>Use case

The use case is quite simple: *I have version 1.0.0 installed and i want to update to 1.0.1 but I have locally customized my server and i’d
like to keep those changes*.
We have several local elements to take into account:

  *

    filesystem changes (files added, removed or deleted).

  *

    configuration changes.

The basic idea is to diff the existing instance with a pure new installation of the same version then apply those changes to a new
provisioned version instance for staging.
We can keep it at the basic filesystem approach with some simple merge strategy (theirs, ours).
We can use the plugin to go into more details. For exemple using the model diff between standalone Wildfly instances we can create a cli
script to reconfigure the upgraded instance in a post-installation step.


    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#diffing-the-filesystem>Diffing the filesystem

The idea is to compare the instance to be upgraded with one instance provisioned with the same feature packs as the one we want to upgrade.
The plugin will provide a list of files or regexp to be excluded.
Each file will be hashed and we compare the hash + the relative path to deduce deleted, modified or added files.
For textual files we can provide a diff (and the means to apply it), but maybe that should be for a later version as some kind of
interaction with the user might be required.


    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#wildfly-standalone-plugin>WildFly standalone
    plugin

This is a specialization of the upgrading algorithm:

  *

    Filtering out some of the 'unimportant' files (tmp, logs).

  *

    Creating diff of textual files (for example the realm properties) which will be applied (merging strategy à la git).

  *

    Using an embedded standalone it creates a jboss-cli script to reconfigure the server (adding/removing extensions and reconfiguring
    subsystems).

  *

    Deleting files that were removed.

This is done on a staging upgraded instance before being copied over the old instance.
I have added a diff/sync operation in standalone that is quite similar to what happens when a slave HC connects to the DC. Thus I start the
current installation, and connect to it from an embedded server using the initial configuration and diff the models.
This is 'experimental' but it works nicely (I was able to 'upgrade' from the standalone.xml of wildfly-core to the standalone-full.xml of
wildfly).
I’m talking only about the model part, I leave the files to the filesystem 'diffing' but it wiill work with managed deployments are those
are added by the filesystem part and then the deployment reference is added in the model.
For a future version of the tooling/plugin we might look for a way to interract more with the user (for example for applying the textual
diffs to choose what to do per file instead of globally).
Also currently the filters for excluding files are defined by the plugin but we could enrich them from the tooling also.


    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#producing-an-update-feature-pack>Producing an
    update feature pack

>From the initial upgrade mecanism Alexey has seen the potential to create a feature pack instead of my initial format.
Currently i’m able to create and installa a feature-pack that will supersede the initial installation with its own local modifications.
Thus from my customized instance I can produce a feature pack that will allow me to reinstall the same instance. Maybe this can be also use
to produce upgrade feature pack for patching.


    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#wildfly-domain-mode>WildFly domain mode

Domain mode is a bit more complex, and we need to think how to manager the model changes.
Those can be at the domain level or the host level and depending on the instance target we would need to get the changes from the domain.xml
or/and the host.xml.
I’m thinking about applying the same strategy as what was done for standalone : aka expose the sync operations to an embedded HC.



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Updating WildFly instance with the Provisioning Tool

Emmanuel Hugonnet
Currently the update feature pack inherit the provisioned feature packs so he would be able to reproduce the whole installation in a single
step since the 'initial' installation would be applied then the customization.

So basically a user makes some changes manually and we provide him with a feature pack. The produced feature pack is not just a feature pack
with the changes but a 'real' feature pack installing everything (initial installation + changes).

Of course he would have to either rebuild the feature pack or adjust it when he wants to use a new version of the installation feature packs
(at least update the gav coordinates in the dependencies). 


Detail question answer :

Currently producing a feature pack is different from updating a provisioned instance : we could specify which files to ignore based on this
(logs files don't mean anything if you want to install a customized instance multiple times but shouldn't be deleted if you apply an update).

Emmanuel



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Updating WildFly instance with the Provisioning Tool

Emmanuel Hugonnet
Hi,

For those who can take 7'26 of their time to look at what we can do with the new provisioning tool to build and share custom feature packs.

https://www.dropbox.com/s/84133sgsjef7pqs/feature_pack.mp4?dl=0



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Updating WildFly instance with the Provisioning Tool

Ken Wills-2
Thanks for sharing that Emmanuel, seems to be coming along well!

On Mon, Oct 2, 2017 at 9:36 AM Emmanuel Hugonnet <[hidden email]> wrote:
Hi,

For those who can take 7'26 of their time to look at what we can do with the new provisioning tool to build and share custom feature packs.

https://www.dropbox.com/s/84133sgsjef7pqs/feature_pack.mp4?dl=0


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Updating WildFly instance with the Provisioning Tool

Brian Stansberry
+1. I encourage folks to check this out; it's short and sweet.

Brian Stansberry
Manager, Senior Principal Software Engineer
Red Hat

On Mon, Oct 2, 2017 at 10:28 AM, Ken Wills <[hidden email]> wrote:
Thanks for sharing that Emmanuel, seems to be coming along well!

On Mon, Oct 2, 2017 at 9:36 AM Emmanuel Hugonnet <[hidden email]> wrote:
Hi,

For those who can take 7'26 of their time to look at what we can do with the new provisioning tool to build and share custom feature packs.

https://www.dropbox.com/s/84133sgsjef7pqs/feature_pack.mp4?dl=0


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Updating WildFly instance with the Provisioning Tool

Scott Stark
In reply to this post by Emmanuel Hugonnet
Interesting.

It brings up a discussion point I have been meaning to raise across the Wildfly and Wildfly-Swarm teams regarding tools for the step before this assembly step of feature-packs and fractions into a distributable archive.

The issue I have seen is that when authoring a feature-pack or fraction, it is difficult to know how to configure the module dependencies. One is often starting with GAV dependencies from a spec, and it is difficult to know how those map onto modules in existing feature-packs for fractions. Do we have any tooling in this area to make this task not a trial and error effort?



On Mon, Oct 2, 2017 at 7:34 AM, Emmanuel Hugonnet <[hidden email]> wrote:
Hi,

For those who can take 7'26 of their time to look at what we can do with the new provisioning tool to build and share custom feature packs.

https://www.dropbox.com/s/84133sgsjef7pqs/feature_pack.mp4?dl=0



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Updating WildFly instance with the Provisioning Tool

Eduardo Martins-2
In reply to this post by Emmanuel Hugonnet
I think it would make more sense to expand the capabilities of the migration tool to handle the update, since the update is the reverse task of the migration. But more than this, the tool is now integrated in server dist, and it already has all the functionality, though spis, to handle updates of everything in the dist, i.e modules, server configs (both xml and management level) and their referenced resources (property files, keystores, certs, etc). Why rebuild distort through feature packs anyway, wasn’t it designed to build a dist?
 
—E

> On 25 Sep 2017, at 10:06, Emmanuel Hugonnet <[hidden email]> wrote:
>
> Hey guys,
>
> TL;DR
> I've been experimenting with Alexey to update a customized provisioned server using the provisioning tool [1].
> I'm using the syncing operations [2] that I created a while back by porting the domain synchronization operations to standalone (to
> synchronize standalone instances in a cloud environment).
> I'm looking for some feedback on this approach.
> Cheers
> Emmanuel
>
> [1]: https://github.com/ehsavoie/pm/tree/upgrade-feature-pack
> [2]: https://github.com/ehsavoie/wildfly-core/tree/model_diff
>
> ----
> full version:
>
>
>  Updating WildFly
>
>
>    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#terminology>Terminology
>
> Updating is the process of applying a fix pack that will increment the micro version. There should be no compatiblity issue. Upgrading is
> the transition of a minor version, compatiblity should be available but there are a lot more changes.
>
> While the mecanisms discussed here are general, they might need more refinement for an upgrade.
>
>
>    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#use-case>Use case
>
> The use case is quite simple: *I have version 1.0.0 installed and i want to update to 1.0.1 but I have locally customized my server and i’d
> like to keep those changes*.
> We have several local elements to take into account:
>
>  *
>
>    filesystem changes (files added, removed or deleted).
>
>  *
>
>    configuration changes.
>
> The basic idea is to diff the existing instance with a pure new installation of the same version then apply those changes to a new
> provisioned version instance for staging.
> We can keep it at the basic filesystem approach with some simple merge strategy (theirs, ours).
> We can use the plugin to go into more details. For exemple using the model diff between standalone Wildfly instances we can create a cli
> script to reconfigure the upgraded instance in a post-installation step.
>
>
>    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#diffing-the-filesystem>Diffing the filesystem
>
> The idea is to compare the instance to be upgraded with one instance provisioned with the same feature packs as the one we want to upgrade.
> The plugin will provide a list of files or regexp to be excluded.
> Each file will be hashed and we compare the hash + the relative path to deduce deleted, modified or added files.
> For textual files we can provide a diff (and the means to apply it), but maybe that should be for a later version as some kind of
> interaction with the user might be required.
>
>
>    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#wildfly-standalone-plugin>WildFly standalone
>    plugin
>
> This is a specialization of the upgrading algorithm:
>
>  *
>
>    Filtering out some of the 'unimportant' files (tmp, logs).
>
>  *
>
>    Creating diff of textual files (for example the realm properties) which will be applied (merging strategy à la git).
>
>  *
>
>    Using an embedded standalone it creates a jboss-cli script to reconfigure the server (adding/removing extensions and reconfiguring
>    subsystems).
>
>  *
>
>    Deleting files that were removed.
>
> This is done on a staging upgraded instance before being copied over the old instance.
> I have added a diff/sync operation in standalone that is quite similar to what happens when a slave HC connects to the DC. Thus I start the
> current installation, and connect to it from an embedded server using the initial configuration and diff the models.
> This is 'experimental' but it works nicely (I was able to 'upgrade' from the standalone.xml of wildfly-core to the standalone-full.xml of
> wildfly).
> I’m talking only about the model part, I leave the files to the filesystem 'diffing' but it wiill work with managed deployments are those
> are added by the filesystem part and then the deployment reference is added in the model.
> For a future version of the tooling/plugin we might look for a way to interract more with the user (for example for applying the textual
> diffs to choose what to do per file instead of globally).
> Also currently the filters for excluding files are defined by the plugin but we could enrich them from the tooling also.
>
>
>    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#producing-an-update-feature-pack>Producing an
>    update feature pack
>
> From the initial upgrade mecanism Alexey has seen the potential to create a feature pack instead of my initial format.
> Currently i’m able to create and installa a feature-pack that will supersede the initial installation with its own local modifications.
> Thus from my customized instance I can produce a feature pack that will allow me to reinstall the same instance. Maybe this can be also use
> to produce upgrade feature pack for patching.
>
>
>    <https://github.com/ehsavoie/pm/blob/upgrade-feature-pack/docs/guide/wildfly/upgrade.adoc#wildfly-domain-mode>WildFly domain mode
>
> Domain mode is a bit more complex, and we need to think how to manager the model changes.
> Those can be at the domain level or the host level and depending on the instance target we would need to get the changes from the domain.xml
> or/and the host.xml.
> I’m thinking about applying the same strategy as what was done for standalone : aka expose the sync operations to an embedded HC.
>
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Updating WildFly instance with the Provisioning Tool

Eduardo Martins-2

On 03 Oct 2017, at 15:11, Eduardo Martins <[hidden email]> wrote:

Why rebuild distort through feature packs anyway, wasn’t it designed to build a dist? 

Why rebuild it all through feature packs anyway, wasn’t it designed to build artifacts and composed a set of these to build a dist? Feels like several in-between and not needed steps to handle the update…

—E


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Updating WildFly instance with the Provisioning Tool

Alexey Loubyansky
In reply to this post by Scott Stark
It's not exactly clear to me what issue you are describing. But I can provide some basic info on how feature-packs are authored for the wildfly releases (core, servlet, full). Perhaps then you could ask a more specific question.

A feature-pack represents a specific release. So there will be a feature-pack for the core, another one for the servlet distribution and another one for the full one. In feature-packs, modules are organized into packages (which is an atomic unit of content possibly with dependencies on other packages from the same or another feature-pack).
When a feature-pack is generated, the packages are generated from the modules. Each module becomes are package in a feature-pack. And module dependencies become package dependencies. Is it any close to the issue you described?

On Tue, Oct 3, 2017 at 1:51 AM, Scott Stark <[hidden email]> wrote:
Interesting.

It brings up a discussion point I have been meaning to raise across the Wildfly and Wildfly-Swarm teams regarding tools for the step before this assembly step of feature-packs and fractions into a distributable archive.

The issue I have seen is that when authoring a feature-pack or fraction, it is difficult to know how to configure the module dependencies. One is often starting with GAV dependencies from a spec, and it is difficult to know how those map onto modules in existing feature-packs for fractions. Do we have any tooling in this area to make this task not a trial and error effort?



On Mon, Oct 2, 2017 at 7:34 AM, Emmanuel Hugonnet <[hidden email]> wrote:
Hi,

For those who can take 7'26 of their time to look at what we can do with the new provisioning tool to build and share custom feature packs.

https://www.dropbox.com/s/84133sgsjef7pqs/feature_pack.mp4?dl=0



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [wildfly-swarm-internal] Updating WildFly instance with the Provisioning Tool

Bob McWhirter
I think the issue (at least as I’ve found it) is that we either:

a) scrub around in existing feature-packs and modules/ directory to determine if there’s a module.xml matching the thing and version we need or

b) hand-author a module.xml based upon pom.xml, adding a chance to mess things up.

-Bob

On Tue, Oct 3, 2017 at 10:37 AM, Alexey Loubyansky <[hidden email]> wrote:
It's not exactly clear to me what issue you are describing. But I can provide some basic info on how feature-packs are authored for the wildfly releases (core, servlet, full). Perhaps then you could ask a more specific question.

A feature-pack represents a specific release. So there will be a feature-pack for the core, another one for the servlet distribution and another one for the full one. In feature-packs, modules are organized into packages (which is an atomic unit of content possibly with dependencies on other packages from the same or another feature-pack).
When a feature-pack is generated, the packages are generated from the modules. Each module becomes are package in a feature-pack. And module dependencies become package dependencies. Is it any close to the issue you described?

On Tue, Oct 3, 2017 at 1:51 AM, Scott Stark <[hidden email]> wrote:
Interesting.

It brings up a discussion point I have been meaning to raise across the Wildfly and Wildfly-Swarm teams regarding tools for the step before this assembly step of feature-packs and fractions into a distributable archive.

The issue I have seen is that when authoring a feature-pack or fraction, it is difficult to know how to configure the module dependencies. One is often starting with GAV dependencies from a spec, and it is difficult to know how those map onto modules in existing feature-packs for fractions. Do we have any tooling in this area to make this task not a trial and error effort?



On Mon, Oct 2, 2017 at 7:34 AM, Emmanuel Hugonnet <[hidden email]> wrote:
Hi,

For those who can take 7'26 of their time to look at what we can do with the new provisioning tool to build and share custom feature packs.

https://www.dropbox.com/s/84133sgsjef7pqs/feature_pack.mp4?dl=0



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Updating WildFly instance with the Provisioning Tool

Emmanuel Hugonnet
In reply to this post by Eduardo Martins-2
Hi,
Yes there is a blank installation to compute the diff( model and filesystem). The filesystem diff is not specific to WildFly/EAP and happens
in the general scope.
The 'wildfly' part is about the deletion of files only.
The demo is more about creating a customized 'image' of your installation that you can replicate.
This comes from the initial goal which was to update an instance.
Maybe we should be using the migration tool from the wildfly plugin to upgrade an instance, this is worth more discussions and tests.
But if I want to upgrade from 6.4 to 7.0 with the deployment my_super_application.war this would be handled automatically as the application
would be part of the feature pack and would simply be replaced with the new one during the upgrade and the configuration changes would be
'migrated' thought the legacy subsystem or the new handlers.
Cheers,
Emmanuel



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [wildfly-swarm-internal] Updating WildFly instance with the Provisioning Tool

Scott Stark
In reply to this post by Bob McWhirter
Correct. Starting from a pom that contains the artifact dependencies, how to translate this into an appropriate module.xml for the output fraction is a trial and error process currently. Does the information exist so that one could go from GAV and/or package names dependencies to the module that provides said dependencies from amongst a set of wildfly feature pack?

On Tue, Oct 3, 2017 at 8:03 AM, Bob McWhirter <[hidden email]> wrote:
I think the issue (at least as I’ve found it) is that we either:

a) scrub around in existing feature-packs and modules/ directory to determine if there’s a module.xml matching the thing and version we need or

b) hand-author a module.xml based upon pom.xml, adding a chance to mess things up.

-Bob

On Tue, Oct 3, 2017 at 10:37 AM, Alexey Loubyansky <[hidden email]> wrote:
It's not exactly clear to me what issue you are describing. But I can provide some basic info on how feature-packs are authored for the wildfly releases (core, servlet, full). Perhaps then you could ask a more specific question.

A feature-pack represents a specific release. So there will be a feature-pack for the core, another one for the servlet distribution and another one for the full one. In feature-packs, modules are organized into packages (which is an atomic unit of content possibly with dependencies on other packages from the same or another feature-pack).
When a feature-pack is generated, the packages are generated from the modules. Each module becomes are package in a feature-pack. And module dependencies become package dependencies. Is it any close to the issue you described?

On Tue, Oct 3, 2017 at 1:51 AM, Scott Stark <[hidden email]> wrote:
Interesting.

It brings up a discussion point I have been meaning to raise across the Wildfly and Wildfly-Swarm teams regarding tools for the step before this assembly step of feature-packs and fractions into a distributable archive.

The issue I have seen is that when authoring a feature-pack or fraction, it is difficult to know how to configure the module dependencies. One is often starting with GAV dependencies from a spec, and it is difficult to know how those map onto modules in existing feature-packs for fractions. Do we have any tooling in this area to make this task not a trial and error effort?



On Mon, Oct 2, 2017 at 7:34 AM, Emmanuel Hugonnet <[hidden email]> wrote:
Hi,

For those who can take 7'26 of their time to look at what we can do with the new provisioning tool to build and share custom feature packs.

https://www.dropbox.com/s/84133sgsjef7pqs/feature_pack.mp4?dl=0



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev




_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [wildfly-swarm-internal] Updating WildFly instance with the Provisioning Tool

Bob McWhirter
My method is to keep a WildFly distribution handy, and grep through the $JBOSS_HOME/modules/… directory, to help find existing ones.

If I have to create new ones, it’s just a tedious effort of converting the Maven transitive tree into a transitive module.xml tree.  Tedious.

-Bob

On Wed, Oct 4, 2017 at 1:23 PM, Scott Stark <[hidden email]> wrote:
Correct. Starting from a pom that contains the artifact dependencies, how to translate this into an appropriate module.xml for the output fraction is a trial and error process currently. Does the information exist so that one could go from GAV and/or package names dependencies to the module that provides said dependencies from amongst a set of wildfly feature pack?

On Tue, Oct 3, 2017 at 8:03 AM, Bob McWhirter <[hidden email]> wrote:
I think the issue (at least as I’ve found it) is that we either:

a) scrub around in existing feature-packs and modules/ directory to determine if there’s a module.xml matching the thing and version we need or

b) hand-author a module.xml based upon pom.xml, adding a chance to mess things up.

-Bob

On Tue, Oct 3, 2017 at 10:37 AM, Alexey Loubyansky <[hidden email]> wrote:
It's not exactly clear to me what issue you are describing. But I can provide some basic info on how feature-packs are authored for the wildfly releases (core, servlet, full). Perhaps then you could ask a more specific question.

A feature-pack represents a specific release. So there will be a feature-pack for the core, another one for the servlet distribution and another one for the full one. In feature-packs, modules are organized into packages (which is an atomic unit of content possibly with dependencies on other packages from the same or another feature-pack).
When a feature-pack is generated, the packages are generated from the modules. Each module becomes are package in a feature-pack. And module dependencies become package dependencies. Is it any close to the issue you described?

On Tue, Oct 3, 2017 at 1:51 AM, Scott Stark <[hidden email]> wrote:
Interesting.

It brings up a discussion point I have been meaning to raise across the Wildfly and Wildfly-Swarm teams regarding tools for the step before this assembly step of feature-packs and fractions into a distributable archive.

The issue I have seen is that when authoring a feature-pack or fraction, it is difficult to know how to configure the module dependencies. One is often starting with GAV dependencies from a spec, and it is difficult to know how those map onto modules in existing feature-packs for fractions. Do we have any tooling in this area to make this task not a trial and error effort?



On Mon, Oct 2, 2017 at 7:34 AM, Emmanuel Hugonnet <[hidden email]> wrote:
Hi,

For those who can take 7'26 of their time to look at what we can do with the new provisioning tool to build and share custom feature packs.

https://www.dropbox.com/s/84133sgsjef7pqs/feature_pack.mp4?dl=0



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev





_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [wildfly-swarm-internal] Updating WildFly instance with the Provisioning Tool

Alexey Loubyansky
In reply to this post by Scott Stark
In the case of the provisioning tools, we don't generate module.xml files. Modules are provided by the developers/build process. We provide the mechanism to generate packages from the modules and then the feature-pack.
When a feature-pack is generated, we know which other feature-packs it depends on. Then when we generate a package from a module we process the module dependencies and figure out on which package(s) from which feature-pack(s) the package we generate should depend on.

I am still not sure whether that answers your question since it's formulated in the context of Swarm and I have a trouble translating it into the context of the feature-pack-based provisioning. I am interested in covering your requirements though. So if it makes sense to you let's continue the discussion or schedule a call to clear things out.

On Wed, Oct 4, 2017 at 7:23 PM, Scott Stark <[hidden email]> wrote:
Correct. Starting from a pom that contains the artifact dependencies, how to translate this into an appropriate module.xml for the output fraction is a trial and error process currently. Does the information exist so that one could go from GAV and/or package names dependencies to the module that provides said dependencies from amongst a set of wildfly feature pack?

On Tue, Oct 3, 2017 at 8:03 AM, Bob McWhirter <[hidden email]> wrote:
I think the issue (at least as I’ve found it) is that we either:

a) scrub around in existing feature-packs and modules/ directory to determine if there’s a module.xml matching the thing and version we need or

b) hand-author a module.xml based upon pom.xml, adding a chance to mess things up.

-Bob

On Tue, Oct 3, 2017 at 10:37 AM, Alexey Loubyansky <[hidden email]> wrote:
It's not exactly clear to me what issue you are describing. But I can provide some basic info on how feature-packs are authored for the wildfly releases (core, servlet, full). Perhaps then you could ask a more specific question.

A feature-pack represents a specific release. So there will be a feature-pack for the core, another one for the servlet distribution and another one for the full one. In feature-packs, modules are organized into packages (which is an atomic unit of content possibly with dependencies on other packages from the same or another feature-pack).
When a feature-pack is generated, the packages are generated from the modules. Each module becomes are package in a feature-pack. And module dependencies become package dependencies. Is it any close to the issue you described?

On Tue, Oct 3, 2017 at 1:51 AM, Scott Stark <[hidden email]> wrote:
Interesting.

It brings up a discussion point I have been meaning to raise across the Wildfly and Wildfly-Swarm teams regarding tools for the step before this assembly step of feature-packs and fractions into a distributable archive.

The issue I have seen is that when authoring a feature-pack or fraction, it is difficult to know how to configure the module dependencies. One is often starting with GAV dependencies from a spec, and it is difficult to know how those map onto modules in existing feature-packs for fractions. Do we have any tooling in this area to make this task not a trial and error effort?



On Mon, Oct 2, 2017 at 7:34 AM, Emmanuel Hugonnet <[hidden email]> wrote:
Hi,

For those who can take 7'26 of their time to look at what we can do with the new provisioning tool to build and share custom feature packs.

https://www.dropbox.com/s/84133sgsjef7pqs/feature_pack.mp4?dl=0



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev





_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [wildfly-swarm-internal] Updating WildFly instance with the Provisioning Tool

Carlo de Wolf
Basically a project build doesn't produce the stuff that can be provisioned into a runtime environment.

Be it WildFly Swarm, WildFly Modules or a more "alien" runtime, Fedora RPMs.

Carlo

On 10/04/2017 10:12 PM, Alexey Loubyansky wrote:
In the case of the provisioning tools, we don't generate module.xml files. Modules are provided by the developers/build process. We provide the mechanism to generate packages from the modules and then the feature-pack.
When a feature-pack is generated, we know which other feature-packs it depends on. Then when we generate a package from a module we process the module dependencies and figure out on which package(s) from which feature-pack(s) the package we generate should depend on.

I am still not sure whether that answers your question since it's formulated in the context of Swarm and I have a trouble translating it into the context of the feature-pack-based provisioning. I am interested in covering your requirements though. So if it makes sense to you let's continue the discussion or schedule a call to clear things out.

On Wed, Oct 4, 2017 at 7:23 PM, Scott Stark <[hidden email]> wrote:
Correct. Starting from a pom that contains the artifact dependencies, how to translate this into an appropriate module.xml for the output fraction is a trial and error process currently. Does the information exist so that one could go from GAV and/or package names dependencies to the module that provides said dependencies from amongst a set of wildfly feature pack?

On Tue, Oct 3, 2017 at 8:03 AM, Bob McWhirter <[hidden email]> wrote:
I think the issue (at least as I’ve found it) is that we either:

a) scrub around in existing feature-packs and modules/ directory to determine if there’s a module.xml matching the thing and version we need or

b) hand-author a module.xml based upon pom.xml, adding a chance to mess things up.

-Bob

On Tue, Oct 3, 2017 at 10:37 AM, Alexey Loubyansky <[hidden email]> wrote:
It's not exactly clear to me what issue you are describing. But I can provide some basic info on how feature-packs are authored for the wildfly releases (core, servlet, full). Perhaps then you could ask a more specific question.

A feature-pack represents a specific release. So there will be a feature-pack for the core, another one for the servlet distribution and another one for the full one. In feature-packs, modules are organized into packages (which is an atomic unit of content possibly with dependencies on other packages from the same or another feature-pack).
When a feature-pack is generated, the packages are generated from the modules. Each module becomes are package in a feature-pack. And module dependencies become package dependencies. Is it any close to the issue you described?

On Tue, Oct 3, 2017 at 1:51 AM, Scott Stark <[hidden email]> wrote:
Interesting.

It brings up a discussion point I have been meaning to raise across the Wildfly and Wildfly-Swarm teams regarding tools for the step before this assembly step of feature-packs and fractions into a distributable archive.

The issue I have seen is that when authoring a feature-pack or fraction, it is difficult to know how to configure the module dependencies. One is often starting with GAV dependencies from a spec, and it is difficult to know how those map onto modules in existing feature-packs for fractions. Do we have any tooling in this area to make this task not a trial and error effort?



On Mon, Oct 2, 2017 at 7:34 AM, Emmanuel Hugonnet <[hidden email]> wrote:
Hi,

For those who can take 7'26 of their time to look at what we can do with the new provisioning tool to build and share custom feature packs.

https://www.dropbox.com/s/84133sgsjef7pqs/feature_pack.mp4?dl=0



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev






_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: [wildfly-swarm-internal] Updating WildFly instance with the Provisioning Tool

Scott Stark
In reply to this post by Alexey Loubyansky
The usecase is that someone is authoring a new swarm fraction, and they have a set of known GAV style dependencies as well as other swarm fraction dependencies and wildfly feature-pack dependencies. What the appropriate module.xml for the new fraction should be is a tool I would like to be able to create. The output module.xml has dependencies on existing modules exported by the dependent fractions and feature-packs along with the GAVs as local resources that are unique to the fraction.

Does the jandex layer support this? 

On Wed, Oct 4, 2017 at 1:12 PM, Alexey Loubyansky <[hidden email]> wrote:
In the case of the provisioning tools, we don't generate module.xml files. Modules are provided by the developers/build process. We provide the mechanism to generate packages from the modules and then the feature-pack.
When a feature-pack is generated, we know which other feature-packs it depends on. Then when we generate a package from a module we process the module dependencies and figure out on which package(s) from which feature-pack(s) the package we generate should depend on.

I am still not sure whether that answers your question since it's formulated in the context of Swarm and I have a trouble translating it into the context of the feature-pack-based provisioning. I am interested in covering your requirements though. So if it makes sense to you let's continue the discussion or schedule a call to clear things out.

On Wed, Oct 4, 2017 at 7:23 PM, Scott Stark <[hidden email]> wrote:
Correct. Starting from a pom that contains the artifact dependencies, how to translate this into an appropriate module.xml for the output fraction is a trial and error process currently. Does the information exist so that one could go from GAV and/or package names dependencies to the module that provides said dependencies from amongst a set of wildfly feature pack?

On Tue, Oct 3, 2017 at 8:03 AM, Bob McWhirter <[hidden email]> wrote:
I think the issue (at least as I’ve found it) is that we either:

a) scrub around in existing feature-packs and modules/ directory to determine if there’s a module.xml matching the thing and version we need or

b) hand-author a module.xml based upon pom.xml, adding a chance to mess things up.

-Bob

On Tue, Oct 3, 2017 at 10:37 AM, Alexey Loubyansky <[hidden email]> wrote:
It's not exactly clear to me what issue you are describing. But I can provide some basic info on how feature-packs are authored for the wildfly releases (core, servlet, full). Perhaps then you could ask a more specific question.

A feature-pack represents a specific release. So there will be a feature-pack for the core, another one for the servlet distribution and another one for the full one. In feature-packs, modules are organized into packages (which is an atomic unit of content possibly with dependencies on other packages from the same or another feature-pack).
When a feature-pack is generated, the packages are generated from the modules. Each module becomes are package in a feature-pack. And module dependencies become package dependencies. Is it any close to the issue you described?

On Tue, Oct 3, 2017 at 1:51 AM, Scott Stark <[hidden email]> wrote:
Interesting.

It brings up a discussion point I have been meaning to raise across the Wildfly and Wildfly-Swarm teams regarding tools for the step before this assembly step of feature-packs and fractions into a distributable archive.

The issue I have seen is that when authoring a feature-pack or fraction, it is difficult to know how to configure the module dependencies. One is often starting with GAV dependencies from a spec, and it is difficult to know how those map onto modules in existing feature-packs for fractions. Do we have any tooling in this area to make this task not a trial and error effort?



On Mon, Oct 2, 2017 at 7:34 AM, Emmanuel Hugonnet <[hidden email]> wrote:
Hi,

For those who can take 7'26 of their time to look at what we can do with the new provisioning tool to build and share custom feature packs.

https://www.dropbox.com/s/84133sgsjef7pqs/feature_pack.mp4?dl=0



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev






_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Updating WildFly instance with the Provisioning Tool

Emmanuel Hugonnet
In reply to this post by Emmanuel Hugonnet
Hi,

This is a second video demonstrating the update of a customized instance :

https://www.dropbox.com/s/b9ma59wtwnrz5ep/update.mp4?dl=0

Please note that this is still a POC and it is run in verbose mode but it shows the diff of textual files and what happens in case of failure.

I have introduced the notion of strategy when merging fails BUT this is not currently enabled ;)

Cheers,

Emmanuel



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

signature.asc (484 bytes) Download Attachment