Management operation for legacy subsystem migration

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Management operation for legacy subsystem migration

Jeff Mesnil
With WildFly 9 and 10, we will have new subsystems that will replace some older subsystems (called legacy subsystems below).

We have to deal with migrating these subsystems:

* migrate from web (JBoss Web)     to undertow
* migrate from messaging (HornetQ) to messaging-activemq (with Apache ActiveMQ Artemis)
* migrate from jacorb              to iiop-openjdk

These 3 tasks are about providing a management operation to perform one-time migration (i.e. the migration is an operation performed by the server on its management model).

I have started to look at this from the messaging perspective.

To constrain this task, I have added some requirements:

1. the legacy subsystem must be an empty shell and has no runtime
=> in WildFLy 10, /subsystem=messaging is only exposing its management model but there is no runtime (HornetQ server library is not included)
2. the server must be in admin-only mode during migration
=> the server is not serving any client during migration.
=> the migration deals only with the server management model by creating the model for the new subsystem based on the legacy subsystem's model
3. Data are not moved during this migration operation
=> moving messages from HornetQ to ActiveMQ destinations is not performed during this management migration.
=> we already have process (such as using JMS bridges) to move messages from one messaging provider to another

Having these three requirements simplifies the migration task and sounds reasonable.
Do you foresee any issues with having them?
 
Given these requirements, the legacy subsystem would need to expose a :migrate operation (at the root of the subsystem) to perform the actual migration of the management model.

Its pseudo code would be something like:

* check the server is in admin-only mode
defined any child resource)
* :describe the legacy subsystem model
* transform the legacy subsystem description to the new subsystem
=> if everything is successful
  * create a composite operation to add the new messaging-activemq extension and all the transformed :add operations
  * report the composite operation outcome to the user
=> else
  * report the error(s) to the user

It is possible that the legacy subsystem can not be fully migrated (e.g. if it defines an attribute that has no equivalent on the new subsystem). In that case, the :migrate operation reports the error(s) to the user.
The user can then change the legacy subsystem model to remove the problematic resource/attributes and invoke :migrate again

For the messaging subsystem, I expect that it will not be possible to fully migrate the replication configuration of the legacy subsystem to the new subsystem (the configuration has significantly changed between HornetQ and ActiveMQ, some configuration will be incompatible).
In that case, I'd expect the user to migrate to the new messaging-activemq subsystem by discarding the legacy subsystem's replication configuration, invoke :migrate and then configure replication for the new subsystem.

In my proof of concept, the :migrate operation has a dry-run boolean attribute. If set to true, the operation will not run the composite operation. It will instead return to the user the list of operations that will be executed when the :migrate operation is actually performed.

I have talked to Tomek which is charge of the iiop migration and he has an additional requirement to emulate the legacy jacorb subsystem with the new iioop-openjdk subsystem. I have not this requirement for the messaging subsystem so I have not given much thought about it...
Same goes for the web -> undertow migration.

It's also important to note that this operation to migrate the management model of a legacy subsystem to a new one is only one step of the whole migration story.
For messaging, the workflow to upgrade an WFLY 9 server to WFLY 10 is made of several other steps (and I may have forgotten some)

* install the new server
* copy the old configuration (with the legacy messaging subsystem)
* start the new server in admin-only mode
* invoke /subsystem=messaging:migrate
  => rinse and repeat by tweaking the legacy subsystem until the migration is successful
* if migration of data can be done offline, do it now (the server is in admin-only mode, so it's ok)
* reload the server to return to running mode with the new messaging subsystem
* if the migration of data must be done offline, it can be done now
  (e.g. create a new JMS bridge from the old running WFLY9/messaging server to this new WFLY10/messaging-activemq server)
* if everything is fine, invoke /subsystem=messaging:remove to remote the legacy subsystem model.

Any comment, critic, feedback?

--
Jeff Mesnil
JBoss, a division of Red Hat
http://jmesnil.net/


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Management operation for legacy subsystem migration

Brian Stansberry
On 4/29/15 9:00 AM, Jeff Mesnil wrote:

> With WildFly 9 and 10, we will have new subsystems that will replace some older subsystems (called legacy subsystems below).
>
> We have to deal with migrating these subsystems:
>
> * migrate from web (JBoss Web)     to undertow
> * migrate from messaging (HornetQ) to messaging-activemq (with Apache ActiveMQ Artemis)
> * migrate from jacorb              to iiop-openjdk
>
> These 3 tasks are about providing a management operation to perform one-time migration (i.e. the migration is an operation performed by the server on its management model).
>
> I have started to look at this from the messaging perspective.
>
> To constrain this task, I have added some requirements:
>
> 1. the legacy subsystem must be an empty shell and has no runtime
> => in WildFLy 10, /subsystem=messaging is only exposing its management model but there is no runtime (HornetQ server library is not included)

I don't see why this needs to be a requirement. In the jacorb case, it
will be violated, as the jacorb subsystem can run as a kind of
"compatibility subsystem", delegating to the iiop-openjdk runtime
services as long as the config doesn't specify any settings that can't
be translated.

> 2. the server must be in admin-only mode during migration
> => the server is not serving any client during migration.
> => the migration deals only with the server management model by creating the model for the new subsystem based on the legacy subsystem's model

+1 for server

It would be very nice to say the same for the DC, but it's worth
discussing. Requiring the DC to be in admin-only creates a fair bit of
usability downside. The whole domain reacts when the DC goes away. Once
we have automatic failover of the DC, it may result in the failover DC
taking over, introducing other problems. (Ken Wills -- this is one to
think about in general re: DC failover.)

It's possible we could allow a migration to happen on a profile where
there aren't any servers running. The basic domain rollout behavior
would allow this without any change. If the HCs allowed the migrate op
to run, and then it gets rolled out to any affected servers, the servers
will reject the op (as they aren't admin-only.) So the op will be rolled
back. (Except in an extreme corner case where the user specifies a
rollout plan that says all servers can fail.)

> 3. Data are not moved during this migration operation
> => moving messages from HornetQ to ActiveMQ destinations is not performed during this management migration.
> => we already have process (such as using JMS bridges) to move messages from one messaging provider to another

+1

>
> Having these three requirements simplifies the migration task and sounds reasonable.
> Do you foresee any issues with having them?
>
> Given these requirements, the legacy subsystem would need to expose a :migrate operation (at the root of the subsystem) to perform the actual migration of the management model.
>
> Its pseudo code would be something like:
>
> * check the server is in admin-only mode
> defined any child resource)
> * :describe the legacy subsystem model
> * transform the legacy subsystem description to the new subsystem
> => if everything is successful
>    * create a composite operation to add the new messaging-activemq extension and all the transformed :add operations
>    * report the composite operation outcome to the user
> => else
>    * report the error(s) to the user
>

There needs to be validation as to whether the extension is already
present; if so skip the add.

This is an edge case on a server but is fairly likely on a DC, where the
extension may be used in profile-new, and now the users wants to migrate
the subsystem in profile-old.

Also, we're going to need to change how the "composite" op works, as
including an extension=foo:add in the same composite as steps that touch
its subsystems won't work. There's a JIRA for this, but JIRA doesn't
seem to be responding at the moment.

There's a workaround for that issue if necessary; just have the migrate
op add a step that deals with the extension add and then a next step
with the composite.

> It is possible that the legacy subsystem can not be fully migrated (e.g. if it defines an attribute that has no equivalent on the new subsystem). In that case, the :migrate operation reports the error(s) to the user.
> The user can then change the legacy subsystem model to remove the problematic resource/attributes and invoke :migrate again
>
> For the messaging subsystem, I expect that it will not be possible to fully migrate the replication configuration of the legacy subsystem to the new subsystem (the configuration has significantly changed between HornetQ and ActiveMQ, some configuration will be incompatible).
> In that case, I'd expect the user to migrate to the new messaging-activemq subsystem by discarding the legacy subsystem's replication configuration, invoke :migrate and then configure replication for the new subsystem.
>
> In my proof of concept, the :migrate operation has a dry-run boolean attribute. If set to true, the operation will not run the composite operation. It will instead return to the user the list of operations that will be executed when the :migrate operation is actually performed.
>

The return value should be the same regardless of the value of that
attribute. Ops have single return value description.

We need to be careful about RBAC. This basically amounts to one op that
then convinces the server to do a whole bunch of other stuff.

A simple thing there is to just mark the op as sensitive. That's
conservative but inelegant.

I don't think that's necessary though. The return value from this op is
a bunch of steps. That provides a lot of data the user may not be
authorized to see. But, to provide that data, you're going to do a lot
of reads, and the access control layer will reject the reads if the user
is not authorized. So the user shouldn't be able to use this to see data
they shouldn't.

As for writes, if the users isn't authorized to do the writes, then they
will fail.

> I have talked to Tomek which is charge of the iiop migration and he has an additional requirement to emulate the legacy jacorb subsystem with the new iioop-openjdk subsystem. I have not this requirement for the messaging subsystem so I have not given much thought about it...
> Same goes for the web -> undertow migration.
>

Oh, I should have read the whole thing before starting to reply!

> It's also important to note that this operation to migrate the management model of a legacy subsystem to a new one is only one step of the whole migration story.
> For messaging, the workflow to upgrade an WFLY 9 server to WFLY 10 is made of several other steps (and I may have forgotten some)
>
> * install the new server
> * copy the old configuration (with the legacy messaging subsystem)
> * start the new server in admin-only mode
> * invoke /subsystem=messaging:migrate
>    => rinse and repeat by tweaking the legacy subsystem until the migration is successful
> * if migration of data can be done offline, do it now (the server is in admin-only mode, so it's ok)
> * reload the server to return to running mode with the new messaging subsystem
> * if the migration of data must be done offline, it can be done now
>    (e.g. create a new JMS bridge from the old running WFLY9/messaging server to this new WFLY10/messaging-activemq server)
> * if everything is fine, invoke /subsystem=messaging:remove to remote the legacy subsystem model.
>
> Any comment, critic, feedback?
>

Sounds good. We need to think in general about the domain case.


--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Management operation for legacy subsystem migration

Tomasz Adamski
>From the iiop perspective:
Conversion between subsystems is fairy staightforward as they are very similiar. The problem is that the jacorb subsystem, from which :migrate operation is executed, has also a compatibility subsystem role. What is more it behaves differently in standalone and domain mode. Standalone subsystem emulates jacorb and as a result of that it checks that all properties can be emulated - it they can't the server won't start. On the other hand domain controller does not reject properties as it may send them to hosts with jacorb subsystem. So migrate operation in standalone mode works the way Jeff describe it but the "rinse and repeat" stage is moved to server startup. It has a runtime though. Migrate operation in domain mode requires discussion. Point 3 does not apply to iiop - no data is moved.

--
Tomasz Adamski
Software Engineer
JBoss by Red Hat

----- Original Message -----
From: "Brian Stansberry" <[hidden email]>
To: [hidden email]
Sent: Wednesday, April 29, 2015 4:58:07 PM
Subject: Re: [wildfly-dev] Management operation for legacy subsystem migration

On 4/29/15 9:00 AM, Jeff Mesnil wrote:

> With WildFly 9 and 10, we will have new subsystems that will replace some older subsystems (called legacy subsystems below).
>
> We have to deal with migrating these subsystems:
>
> * migrate from web (JBoss Web)     to undertow
> * migrate from messaging (HornetQ) to messaging-activemq (with Apache ActiveMQ Artemis)
> * migrate from jacorb              to iiop-openjdk
>
> These 3 tasks are about providing a management operation to perform one-time migration (i.e. the migration is an operation performed by the server on its management model).
>
> I have started to look at this from the messaging perspective.
>
> To constrain this task, I have added some requirements:
>
> 1. the legacy subsystem must be an empty shell and has no runtime
> => in WildFLy 10, /subsystem=messaging is only exposing its management model but there is no runtime (HornetQ server library is not included)

I don't see why this needs to be a requirement. In the jacorb case, it
will be violated, as the jacorb subsystem can run as a kind of
"compatibility subsystem", delegating to the iiop-openjdk runtime
services as long as the config doesn't specify any settings that can't
be translated.

> 2. the server must be in admin-only mode during migration
> => the server is not serving any client during migration.
> => the migration deals only with the server management model by creating the model for the new subsystem based on the legacy subsystem's model

+1 for server

It would be very nice to say the same for the DC, but it's worth
discussing. Requiring the DC to be in admin-only creates a fair bit of
usability downside. The whole domain reacts when the DC goes away. Once
we have automatic failover of the DC, it may result in the failover DC
taking over, introducing other problems. (Ken Wills -- this is one to
think about in general re: DC failover.)

It's possible we could allow a migration to happen on a profile where
there aren't any servers running. The basic domain rollout behavior
would allow this without any change. If the HCs allowed the migrate op
to run, and then it gets rolled out to any affected servers, the servers
will reject the op (as they aren't admin-only.) So the op will be rolled
back. (Except in an extreme corner case where the user specifies a
rollout plan that says all servers can fail.)

> 3. Data are not moved during this migration operation
> => moving messages from HornetQ to ActiveMQ destinations is not performed during this management migration.
> => we already have process (such as using JMS bridges) to move messages from one messaging provider to another

+1

>
> Having these three requirements simplifies the migration task and sounds reasonable.
> Do you foresee any issues with having them?
>
> Given these requirements, the legacy subsystem would need to expose a :migrate operation (at the root of the subsystem) to perform the actual migration of the management model.
>
> Its pseudo code would be something like:
>
> * check the server is in admin-only mode
> defined any child resource)
> * :describe the legacy subsystem model
> * transform the legacy subsystem description to the new subsystem
> => if everything is successful
>    * create a composite operation to add the new messaging-activemq extension and all the transformed :add operations
>    * report the composite operation outcome to the user
> => else
>    * report the error(s) to the user
>

There needs to be validation as to whether the extension is already
present; if so skip the add.

This is an edge case on a server but is fairly likely on a DC, where the
extension may be used in profile-new, and now the users wants to migrate
the subsystem in profile-old.

Also, we're going to need to change how the "composite" op works, as
including an extension=foo:add in the same composite as steps that touch
its subsystems won't work. There's a JIRA for this, but JIRA doesn't
seem to be responding at the moment.

There's a workaround for that issue if necessary; just have the migrate
op add a step that deals with the extension add and then a next step
with the composite.

> It is possible that the legacy subsystem can not be fully migrated (e.g. if it defines an attribute that has no equivalent on the new subsystem). In that case, the :migrate operation reports the error(s) to the user.
> The user can then change the legacy subsystem model to remove the problematic resource/attributes and invoke :migrate again
>
> For the messaging subsystem, I expect that it will not be possible to fully migrate the replication configuration of the legacy subsystem to the new subsystem (the configuration has significantly changed between HornetQ and ActiveMQ, some configuration will be incompatible).
> In that case, I'd expect the user to migrate to the new messaging-activemq subsystem by discarding the legacy subsystem's replication configuration, invoke :migrate and then configure replication for the new subsystem.
>
> In my proof of concept, the :migrate operation has a dry-run boolean attribute. If set to true, the operation will not run the composite operation. It will instead return to the user the list of operations that will be executed when the :migrate operation is actually performed.
>

The return value should be the same regardless of the value of that
attribute. Ops have single return value description.

We need to be careful about RBAC. This basically amounts to one op that
then convinces the server to do a whole bunch of other stuff.

A simple thing there is to just mark the op as sensitive. That's
conservative but inelegant.

I don't think that's necessary though. The return value from this op is
a bunch of steps. That provides a lot of data the user may not be
authorized to see. But, to provide that data, you're going to do a lot
of reads, and the access control layer will reject the reads if the user
is not authorized. So the user shouldn't be able to use this to see data
they shouldn't.

As for writes, if the users isn't authorized to do the writes, then they
will fail.

> I have talked to Tomek which is charge of the iiop migration and he has an additional requirement to emulate the legacy jacorb subsystem with the new iioop-openjdk subsystem. I have not this requirement for the messaging subsystem so I have not given much thought about it...
> Same goes for the web -> undertow migration.
>

Oh, I should have read the whole thing before starting to reply!

> It's also important to note that this operation to migrate the management model of a legacy subsystem to a new one is only one step of the whole migration story.
> For messaging, the workflow to upgrade an WFLY 9 server to WFLY 10 is made of several other steps (and I may have forgotten some)
>
> * install the new server
> * copy the old configuration (with the legacy messaging subsystem)
> * start the new server in admin-only mode
> * invoke /subsystem=messaging:migrate
>    => rinse and repeat by tweaking the legacy subsystem until the migration is successful
> * if migration of data can be done offline, do it now (the server is in admin-only mode, so it's ok)
> * reload the server to return to running mode with the new messaging subsystem
> * if the migration of data must be done offline, it can be done now
>    (e.g. create a new JMS bridge from the old running WFLY9/messaging server to this new WFLY10/messaging-activemq server)
> * if everything is fine, invoke /subsystem=messaging:remove to remote the legacy subsystem model.
>
> Any comment, critic, feedback?
>

Sounds good. We need to think in general about the domain case.


--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Management operation for legacy subsystem migration

jtgreene
Administrator
In reply to this post by Jeff Mesnil
Instead of hard errors, and manipulating the old model, wouldn't it be better to just convert and then report back in the response what was dropped?

The outcome is the same right, but with just less steps.

If we are worried about config data loss or something like that, the dry run option cold be he default.

> On Apr 29, 2015, at 9:01 AM, Jeff Mesnil <[hidden email]> wrote:
>
> With WildFly 9 and 10, we will have new subsystems that will replace some older subsystems (called legacy subsystems below).
>
> We have to deal with migrating these subsystems:
>
> * migrate from web (JBoss Web)     to undertow
> * migrate from messaging (HornetQ) to messaging-activemq (with Apache ActiveMQ Artemis)
> * migrate from jacorb              to iiop-openjdk
>
> These 3 tasks are about providing a management operation to perform one-time migration (i.e. the migration is an operation performed by the server on its management model).
>
> I have started to look at this from the messaging perspective.
>
> To constrain this task, I have added some requirements:
>
> 1. the legacy subsystem must be an empty shell and has no runtime
> => in WildFLy 10, /subsystem=messaging is only exposing its management model but there is no runtime (HornetQ server library is not included)
> 2. the server must be in admin-only mode during migration
> => the server is not serving any client during migration.
> => the migration deals only with the server management model by creating the model for the new subsystem based on the legacy subsystem's model
> 3. Data are not moved during this migration operation
> => moving messages from HornetQ to ActiveMQ destinations is not performed during this management migration.
> => we already have process (such as using JMS bridges) to move messages from one messaging provider to another
>
> Having these three requirements simplifies the migration task and sounds reasonable.
> Do you foresee any issues with having them?
>
> Given these requirements, the legacy subsystem would need to expose a :migrate operation (at the root of the subsystem) to perform the actual migration of the management model.
>
> Its pseudo code would be something like:
>
> * check the server is in admin-only mode
> defined any child resource)
> * :describe the legacy subsystem model
> * transform the legacy subsystem description to the new subsystem
> => if everything is successful
>  * create a composite operation to add the new messaging-activemq extension and all the transformed :add operations
>  * report the composite operation outcome to the user
> => else
>  * report the error(s) to the user
>
> It is possible that the legacy subsystem can not be fully migrated (e.g. if it defines an attribute that has no equivalent on the new subsystem). In that case, the :migrate operation reports the error(s) to the user.
> The user can then change the legacy subsystem model to remove the problematic resource/attributes and invoke :migrate again
>
> For the messaging subsystem, I expect that it will not be possible to fully migrate the replication configuration of the legacy subsystem to the new subsystem (the configuration has significantly changed between HornetQ and ActiveMQ, some configuration will be incompatible).
> In that case, I'd expect the user to migrate to the new messaging-activemq subsystem by discarding the legacy subsystem's replication configuration, invoke :migrate and then configure replication for the new subsystem.
>
> In my proof of concept, the :migrate operation has a dry-run boolean attribute. If set to true, the operation will not run the composite operation. It will instead return to the user the list of operations that will be executed when the :migrate operation is actually performed.
>
> I have talked to Tomek which is charge of the iiop migration and he has an additional requirement to emulate the legacy jacorb subsystem with the new iioop-openjdk subsystem. I have not this requirement for the messaging subsystem so I have not given much thought about it...
> Same goes for the web -> undertow migration.
>
> It's also important to note that this operation to migrate the management model of a legacy subsystem to a new one is only one step of the whole migration story.
> For messaging, the workflow to upgrade an WFLY 9 server to WFLY 10 is made of several other steps (and I may have forgotten some)
>
> * install the new server
> * copy the old configuration (with the legacy messaging subsystem)
> * start the new server in admin-only mode
> * invoke /subsystem=messaging:migrate
>  => rinse and repeat by tweaking the legacy subsystem until the migration is successful
> * if migration of data can be done offline, do it now (the server is in admin-only mode, so it's ok)
> * reload the server to return to running mode with the new messaging subsystem
> * if the migration of data must be done offline, it can be done now
>  (e.g. create a new JMS bridge from the old running WFLY9/messaging server to this new WFLY10/messaging-activemq server)
> * if everything is fine, invoke /subsystem=messaging:remove to remote the legacy subsystem model.
>
> Any comment, critic, feedback?
>
> --
> Jeff Mesnil
> JBoss, a division of Red Hat
> http://jmesnil.net/
>
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Management operation for legacy subsystem migration

Jeff Mesnil
In reply to this post by Brian Stansberry
comments embedded.

> On 29 Apr 2015, at 16:58, Brian Stansberry <[hidden email]> wrote:
>
> On 4/29/15 9:00 AM, Jeff Mesnil wrote:
>> With WildFly 9 and 10, we will have new subsystems that will replace some older subsystems (called legacy subsystems below).
>>
>> We have to deal with migrating these subsystems:
>>
>> * migrate from web (JBoss Web)     to undertow
>> * migrate from messaging (HornetQ) to messaging-activemq (with Apache ActiveMQ Artemis)
>> * migrate from jacorb              to iiop-openjdk
>>
>> These 3 tasks are about providing a management operation to perform one-time migration (i.e. the migration is an operation performed by the server on its management model).
>>
>> I have started to look at this from the messaging perspective.
>>
>> To constrain this task, I have added some requirements:
>>
>> 1. the legacy subsystem must be an empty shell and has no runtime
>> => in WildFLy 10, /subsystem=messaging is only exposing its management model but there is no runtime (HornetQ server library is not included)
>
> I don't see why this needs to be a requirement. In the jacorb case, it
> will be violated, as the jacorb subsystem can run as a kind of
> "compatibility subsystem", delegating to the iiop-openjdk runtime
> services as long as the config doesn't specify any settings that can't
> be translated.

You are right, this is not a requirement. It’s specific to the legacy messaging subsystem and is not related to the generic migrate operation of legacy subsystem.
 
> There needs to be validation as to whether the extension is already
> present; if so skip the add.
>
> This is an edge case on a server but is fairly likely on a DC, where the
> extension may be used in profile-new, and now the users wants to migrate
> the subsystem in profile-old.

Ok, I’ll add this check.

>
> Also, we're going to need to change how the "composite" op works, as
> including an extension=foo:add in the same composite as steps that touch
> its subsystems won't work. There's a JIRA for this, but JIRA doesn't
> seem to be responding at the moment.
>
> There's a workaround for that issue if necessary; just have the migrate
> op add a step that deals with the extension add and then a next step
> with the composite.

Ok, that’s what I did in my POC[1]. I was not sure if it was a bug or not.

The JIRA should be https://issues.jboss.org/browse/WFCORE-323.
You even used the messaging subsystem for its example :)

>
>> It is possible that the legacy subsystem can not be fully migrated (e.g. if it defines an attribute that has no equivalent on the new subsystem). In that case, the :migrate operation reports the error(s) to the user.
>> The user can then change the legacy subsystem model to remove the problematic resource/attributes and invoke :migrate again
>>
>> For the messaging subsystem, I expect that it will not be possible to fully migrate the replication configuration of the legacy subsystem to the new subsystem (the configuration has significantly changed between HornetQ and ActiveMQ, some configuration will be incompatible).
>> In that case, I'd expect the user to migrate to the new messaging-activemq subsystem by discarding the legacy subsystem's replication configuration, invoke :migrate and then configure replication for the new subsystem.
>>
>> In my proof of concept, the :migrate operation has a dry-run boolean attribute. If set to true, the operation will not run the composite operation. It will instead return to the user the list of operations that will be executed when the :migrate operation is actually performed.
>>
>
> The return value should be the same regardless of the value of that
> attribute. Ops have single return value description.

Good point. In that case, the :migrate operation will return the outcome of the migration operation.
And I can add a :describe-migration that lists the required operations.

> We need to be careful about RBAC. This basically amounts to one op that
> then convinces the server to do a whole bunch of other stuff.

This :migrate operation is using the outcome of the :describe operation on the legacy subsystem (which has a READ_WHOLE_CONFIG access constraint) and execute :add operations.
Do we need to flag it specifically or are its constraints the byproduct of the :describe and :add operations constraints?

jeff

[1] https://github.com/jmesnil/wildfly/blob/messaging-activemq-xml/messaging/src/main/java/org/jboss/as/messaging/MigrateOperation.java#L119

--
Jeff Mesnil
JBoss, a division of Red Hat
http://jmesnil.net/


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Management operation for legacy subsystem migration

Jeff Mesnil
In reply to this post by jtgreene
On 30 Apr 2015, at 04:53, Jason T. Greene <[hidden email]> wrote:
>
> Instead of hard errors, and manipulating the old model, wouldn't it be better to just convert and then report back in the response what was dropped?

I’m not sure to understand your remark. Could you clarify?

I don’t manipulate the old model. I describe it and use the description to create the :add operations for the new subsystem.

At first, I was reading the old model, manipulating it to create the new subsystem model and create a new Resource with that manipulated model.
It turns out to be a bad idea as I was bypassing any validation steps performed by the :add operations and ended up with an invalid new subsystem (that could not be loaded).

Using the ":describe + new :add operations" approach ensures that my new subsystem will be valid if my :migrate operation succeeds.

--
Jeff Mesnil
JBoss, a division of Red Hat
http://jmesnil.net/


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Management operation for legacy subsystem migration

Brian Stansberry
In reply to this post by Jeff Mesnil
On 4/30/15 9:03 AM, Jeff Mesnil wrote:

> comments embedded.
>
>> On 29 Apr 2015, at 16:58, Brian Stansberry <[hidden email]> wrote:
>>
>> On 4/29/15 9:00 AM, Jeff Mesnil wrote:
>>> With WildFly 9 and 10, we will have new subsystems that will replace some older subsystems (called legacy subsystems below).
>>>
>>> We have to deal with migrating these subsystems:
>>>
>>> * migrate from web (JBoss Web)     to undertow
>>> * migrate from messaging (HornetQ) to messaging-activemq (with Apache ActiveMQ Artemis)
>>> * migrate from jacorb              to iiop-openjdk
>>>
>>> These 3 tasks are about providing a management operation to perform one-time migration (i.e. the migration is an operation performed by the server on its management model).
>>>
>>> I have started to look at this from the messaging perspective.
>>>
>>> To constrain this task, I have added some requirements:
>>>
>>> 1. the legacy subsystem must be an empty shell and has no runtime
>>> => in WildFLy 10, /subsystem=messaging is only exposing its management model but there is no runtime (HornetQ server library is not included)
>>
>> I don't see why this needs to be a requirement. In the jacorb case, it
>> will be violated, as the jacorb subsystem can run as a kind of
>> "compatibility subsystem", delegating to the iiop-openjdk runtime
>> services as long as the config doesn't specify any settings that can't
>> be translated.
>
> You are right, this is not a requirement. It’s specific to the legacy messaging subsystem and is not related to the generic migrate operation of legacy subsystem.
>
>> There needs to be validation as to whether the extension is already
>> present; if so skip the add.
>>
>> This is an edge case on a server but is fairly likely on a DC, where the
>> extension may be used in profile-new, and now the users wants to migrate
>> the subsystem in profile-old.
>
> Ok, I’ll add this check.
>
>>
>> Also, we're going to need to change how the "composite" op works, as
>> including an extension=foo:add in the same composite as steps that touch
>> its subsystems won't work. There's a JIRA for this, but JIRA doesn't
>> seem to be responding at the moment.
>>
>> There's a workaround for that issue if necessary; just have the migrate
>> op add a step that deals with the extension add and then a next step
>> with the composite.
>
> Ok, that’s what I did in my POC[1]. I was not sure if it was a bug or not.
>
> The JIRA should be https://issues.jboss.org/browse/WFCORE-323.
> You even used the messaging subsystem for its example :)
>
>>
>>> It is possible that the legacy subsystem can not be fully migrated (e.g. if it defines an attribute that has no equivalent on the new subsystem). In that case, the :migrate operation reports the error(s) to the user.
>>> The user can then change the legacy subsystem model to remove the problematic resource/attributes and invoke :migrate again
>>>
>>> For the messaging subsystem, I expect that it will not be possible to fully migrate the replication configuration of the legacy subsystem to the new subsystem (the configuration has significantly changed between HornetQ and ActiveMQ, some configuration will be incompatible).
>>> In that case, I'd expect the user to migrate to the new messaging-activemq subsystem by discarding the legacy subsystem's replication configuration, invoke :migrate and then configure replication for the new subsystem.
>>>
>>> In my proof of concept, the :migrate operation has a dry-run boolean attribute. If set to true, the operation will not run the composite operation. It will instead return to the user the list of operations that will be executed when the :migrate operation is actually performed.
>>>
>>
>> The return value should be the same regardless of the value of that
>> attribute. Ops have single return value description.
>
> Good point. In that case, the :migrate operation will return the outcome of the migration operation.

What would that be? If the describe-migration is a separate op, then I
figure for this op, the response would be

{"outcome"=>"success","result"=>undefined}

> And I can add a :describe-migration that lists the required operations.
>
>> We need to be careful about RBAC. This basically amounts to one op that
>> then convinces the server to do a whole bunch of other stuff.
>
> This :migrate operation is using the outcome of the :describe operation on the legacy subsystem (which has a READ_WHOLE_CONFIG access constraint) and execute :add operations.
> Do we need to flag it specifically or are its constraints the byproduct of the :describe and :add operations constraints?
>

So this basically means these ops would be limited to SuperUser or
Administrator (unless the users change the settings for READ_WHOLE_CONFIG.)

I'd forgotten that "describe" worked that way.

The constraints would be a byproduct, but we should flag the op anyway,
otherwise it's unintuitive. The metadata for the op inform the user of
the effective constraint. Plus if we're not careful the user may end up
getting a failure message saying they aren't authorized to run the
"describe" op when they didn't request that op.

> jeff
>
> [1] https://github.com/jmesnil/wildfly/blob/messaging-activemq-xml/messaging/src/main/java/org/jboss/as/messaging/MigrateOperation.java#L119
>


--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Management operation for legacy subsystem migration

Brian Stansberry
In reply to this post by Tomasz Adamski
On 4/29/15 3:09 PM, Tomasz Adamski wrote:
>>From the iiop perspective:
> Conversion between subsystems is fairy staightforward as they are very similiar. The problem is that the jacorb subsystem, from which :migrate operation is executed, has also a compatibility subsystem role. What is more it behaves differently in standalone and domain mode. Standalone subsystem emulates jacorb and as a result of that it checks that all properties can be emulated - it they can't the server won't start. On the other hand domain controller does not reject properties as it may send them to hosts with jacorb subsystem. So migrate operation in standalone mode works the way Jeff describe it but the "rinse and repeat" stage is moved to server startup.

We could consider being more forgiving in admin-only mode; instead of
failing, log ERROR messages. That gives the user the option to fix stuff.

It would be very good to be able to take an AS 7 / WF 8 / EAP 6 config
and be able to bring it into compliance with WF 10 without having to use
anything but the management API.

> It has a runtime though. Migrate operation in domain mode requires discussion. Point 3 does not apply to iiop - no data is moved.
>


--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Management operation for legacy subsystem migration

Tomasz Adamski
----- Original Message -----

> From: "Brian Stansberry" <[hidden email]>
> To: [hidden email]
> Sent: Thursday, April 30, 2015 8:21:41 PM
> Subject: Re: [wildfly-dev] Management operation for legacy subsystem migration
>
> On 4/29/15 3:09 PM, Tomasz Adamski wrote:
> >>From the iiop perspective:
> > Conversion between subsystems is fairy staightforward as they are very
> > similiar. The problem is that the jacorb subsystem, from which :migrate
> > operation is executed, has also a compatibility subsystem role. What is
> > more it behaves differently in standalone and domain mode. Standalone
> > subsystem emulates jacorb and as a result of that it checks that all
> > properties can be emulated - it they can't the server won't start. On the
> > other hand domain controller does not reject properties as it may send
> > them to hosts with jacorb subsystem. So migrate operation in standalone
> > mode works the way Jeff describe it but the "rinse and repeat" stage is
> > moved to server startup.
>
> We could consider being more forgiving in admin-only mode; instead of
> failing, log ERROR messages. That gives the user the option to fix stuff.
>
> It would be very good to be able to take an AS 7 / WF 8 / EAP 6 config
> and be able to bring it into compliance with WF 10 without having to use
> anything but the management API.

OK. Will fix it.

>
> > It has a runtime though. Migrate operation in domain mode requires
> > discussion. Point 3 does not apply to iiop - no data is moved.
> >
>
>
> --
> Brian Stansberry
> Senior Principal Software Engineer
> JBoss by Red Hat
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Management operation for legacy subsystem migration

jtgreene
Administrator
In reply to this post by Jeff Mesnil


Sent from my iPhone

> On Apr 30, 2015, at 9:08 AM, Jeff Mesnil <[hidden email]> wrote:
>
>> On 30 Apr 2015, at 04:53, Jason T. Greene <[hidden email]> wrote:
>>
>> Instead of hard errors, and manipulating the old model, wouldn't it be better to just convert and then report back in the response what was dropped?
>
> I’m not sure to understand your remark. Could you clarify?

Sorry I meant wouldn't it be better to skip/ignore configuration that does not have an equivalent mapping and report just warnings about what doesn't map. My point was that it's probably easier for the user to fixup the new subsystem  then it is to repeatedly alter the old subsystem until you get something that will convert.

>
> I don’t manipulate the old model. I describe it and use the description to create the :add operations for the new subsystem.
>
> At first, I was reading the old model, manipulating it to create the new subsystem model and create a new Resource with that manipulated model.
> It turns out to be a bad idea as I was bypassing any validation steps performed by the :add operations and ended up with an invalid new subsystem (that could not be loaded).
>
> Using the ":describe + new :add operations" approach ensures that my new subsystem will be valid if my :migrate operation succeeds.
>
> --
> Jeff Mesnil
> JBoss, a division of Red Hat
> http://jmesnil.net/
>

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev