Batch Subsystem Changes

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Batch Subsystem Changes

James Perkins
Hello All,
The past couple weeks I've been working on basically a redo of the batch
subsystem. Almost the entire management model is changing to hopefully
make it more user friendly.

In WildFly 8 and WildFly 9 the model looked like the following:

{
     "job-repository-type" => "in-memory",
     "job-repository" => {"jdbc" => {"jndi-name" => undefined}},
     "thread-factory" => undefined,
     "thread-pool" => {"batch" => {
         "keepalive-time" => {
             "time" => 30L,
             "unit" => "SECONDS"
         },
         "max-threads" => 10,
         "name" => "batch",
         "thread-factory" => undefined
     }}
}

The job-repository-type could either be jdcb or in-memory. The jndi-name
attribute on the single job-repository=jdbc resource could be undefined
indicating the default data-source should be used or JNDI name to look
up the data-source with no validation being done until the user actually
tries to deploy a batch deployment.

The thread-pool and thread-factory are the same as other resources that
use the thread "subsystem" shared resources.

As you can see it's not very intuitive and somewhat clumsy to say the
least. Only a single job-repository could be defined which isn't great
for multiple deployments.

In WildFly 10 the model, at least currently, will look like:

{
     "default-job-repository" => "default",
     "in-memory-job-repository" => {"default" => {}},
     "jdbc-job-repository" => {"jdbc" => {"data-source" => "ExampleDS"}},
     "thread-factory" => undefined,
     "thread-pool" => {"batch" => {
         "active-count" => 0,
         "completed-task-count" => 0L,
         "current-thread-count" => 0,
         "keepalive-time" => {
             "time" => 30L,
             "unit" => "SECONDS"
         },
         "largest-thread-count" => 0,
         "max-threads" => 10,
         "name" => "batch",
         "queue-size" => 0,
         "rejected-count" => 0,
         "task-count" => 0L,
         "thread-factory" => undefined
     }}
}

The default-job-repository will be an attribute similar to the previous
job-repository attribute. The difference being you can use any named
in-memory-job-repository or jdbc-job-repository. You can have any number
of in-memory or JDBC job repositories.

The data-source attribute value on a jdbc-job-repository resource will
use the org.wildfly.data-source [1]. The name of the data-source is used
instead of the JNDI which is a much cleaner approach.

The thread-factory may be removed and the thread-pool may be changed to
use attribute groups (once I figure out how to use them :)).

As part of this I considered changing the name from batch to
batch-jberet. The main concern I had with this was the web console, but
I seem to have broken that anyway with the changes to the model. Does
anyone have opinions on a name change to batch-jberet?

Also parsing an old configuration may have some issues if the user was
using a JDBC job repository. I've currently not found a good way to find
a data-source resource name based on a JNDI name. I'm not sure if we
should just fail when adding a legacy JDBC job repository. Any
suggestions here would be helpful.

Any comments or concerns in general are welcome. This is our chance to
get it right this time.


[1]: https://github.com/wildfly/wildfly/pull/7682

--
James R. Perkins
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Batch Subsystem Changes

jtgreene
Administrator
IMO we should avoid breaking compatibility unless there is no other option. If we do break compatibility we should rename the subsystem and provide a migration operation.

> On Jul 6, 2015, at 2:13 PM, James R. Perkins <[hidden email]> wrote:
>
> Hello All,
> The past couple weeks I've been working on basically a redo of the batch
> subsystem. Almost the entire management model is changing to hopefully
> make it more user friendly.
>
> In WildFly 8 and WildFly 9 the model looked like the following:
>
> {
>     "job-repository-type" => "in-memory",
>     "job-repository" => {"jdbc" => {"jndi-name" => undefined}},
>     "thread-factory" => undefined,
>     "thread-pool" => {"batch" => {
>         "keepalive-time" => {
>             "time" => 30L,
>             "unit" => "SECONDS"
>         },
>         "max-threads" => 10,
>         "name" => "batch",
>         "thread-factory" => undefined
>     }}
> }
>
> The job-repository-type could either be jdcb or in-memory. The jndi-name
> attribute on the single job-repository=jdbc resource could be undefined
> indicating the default data-source should be used or JNDI name to look
> up the data-source with no validation being done until the user actually
> tries to deploy a batch deployment.
>
> The thread-pool and thread-factory are the same as other resources that
> use the thread "subsystem" shared resources.
>
> As you can see it's not very intuitive and somewhat clumsy to say the
> least. Only a single job-repository could be defined which isn't great
> for multiple deployments.
>
> In WildFly 10 the model, at least currently, will look like:
>
> {
>     "default-job-repository" => "default",
>     "in-memory-job-repository" => {"default" => {}},
>     "jdbc-job-repository" => {"jdbc" => {"data-source" => "ExampleDS"}},
>     "thread-factory" => undefined,
>     "thread-pool" => {"batch" => {
>         "active-count" => 0,
>         "completed-task-count" => 0L,
>         "current-thread-count" => 0,
>         "keepalive-time" => {
>             "time" => 30L,
>             "unit" => "SECONDS"
>         },
>         "largest-thread-count" => 0,
>         "max-threads" => 10,
>         "name" => "batch",
>         "queue-size" => 0,
>         "rejected-count" => 0,
>         "task-count" => 0L,
>         "thread-factory" => undefined
>     }}
> }
>
> The default-job-repository will be an attribute similar to the previous
> job-repository attribute. The difference being you can use any named
> in-memory-job-repository or jdbc-job-repository. You can have any number
> of in-memory or JDBC job repositories.
>
> The data-source attribute value on a jdbc-job-repository resource will
> use the org.wildfly.data-source [1]. The name of the data-source is used
> instead of the JNDI which is a much cleaner approach.
>
> The thread-factory may be removed and the thread-pool may be changed to
> use attribute groups (once I figure out how to use them :)).
>
> As part of this I considered changing the name from batch to
> batch-jberet. The main concern I had with this was the web console, but
> I seem to have broken that anyway with the changes to the model. Does
> anyone have opinions on a name change to batch-jberet?
>
> Also parsing an old configuration may have some issues if the user was
> using a JDBC job repository. I've currently not found a good way to find
> a data-source resource name based on a JNDI name. I'm not sure if we
> should just fail when adding a legacy JDBC job repository. Any
> suggestions here would be helpful.
>
> Any comments or concerns in general are welcome. This is our chance to
> get it right this time.
>
>
> [1]: https://github.com/wildfly/wildfly/pull/7682
>
> --
> James R. Perkins
> JBoss by Red Hat
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Batch Subsystem Changes

jtgreene
Administrator
Actually in this case you could support both the old and the new subsystem as its a one to one mapping. Then you do not need the migration bit.

Sent from my iPhone

> On Jul 6, 2015, at 8:02 PM, Jason T. Greene <[hidden email]> wrote:
>
> IMO we should avoid breaking compatibility unless there is no other option. If we do break compatibility we should rename the subsystem and provide a migration operation.
>
>> On Jul 6, 2015, at 2:13 PM, James R. Perkins <[hidden email]> wrote:
>>
>> Hello All,
>> The past couple weeks I've been working on basically a redo of the batch
>> subsystem. Almost the entire management model is changing to hopefully
>> make it more user friendly.
>>
>> In WildFly 8 and WildFly 9 the model looked like the following:
>>
>> {
>>    "job-repository-type" => "in-memory",
>>    "job-repository" => {"jdbc" => {"jndi-name" => undefined}},
>>    "thread-factory" => undefined,
>>    "thread-pool" => {"batch" => {
>>        "keepalive-time" => {
>>            "time" => 30L,
>>            "unit" => "SECONDS"
>>        },
>>        "max-threads" => 10,
>>        "name" => "batch",
>>        "thread-factory" => undefined
>>    }}
>> }
>>
>> The job-repository-type could either be jdcb or in-memory. The jndi-name
>> attribute on the single job-repository=jdbc resource could be undefined
>> indicating the default data-source should be used or JNDI name to look
>> up the data-source with no validation being done until the user actually
>> tries to deploy a batch deployment.
>>
>> The thread-pool and thread-factory are the same as other resources that
>> use the thread "subsystem" shared resources.
>>
>> As you can see it's not very intuitive and somewhat clumsy to say the
>> least. Only a single job-repository could be defined which isn't great
>> for multiple deployments.
>>
>> In WildFly 10 the model, at least currently, will look like:
>>
>> {
>>    "default-job-repository" => "default",
>>    "in-memory-job-repository" => {"default" => {}},
>>    "jdbc-job-repository" => {"jdbc" => {"data-source" => "ExampleDS"}},
>>    "thread-factory" => undefined,
>>    "thread-pool" => {"batch" => {
>>        "active-count" => 0,
>>        "completed-task-count" => 0L,
>>        "current-thread-count" => 0,
>>        "keepalive-time" => {
>>            "time" => 30L,
>>            "unit" => "SECONDS"
>>        },
>>        "largest-thread-count" => 0,
>>        "max-threads" => 10,
>>        "name" => "batch",
>>        "queue-size" => 0,
>>        "rejected-count" => 0,
>>        "task-count" => 0L,
>>        "thread-factory" => undefined
>>    }}
>> }
>>
>> The default-job-repository will be an attribute similar to the previous
>> job-repository attribute. The difference being you can use any named
>> in-memory-job-repository or jdbc-job-repository. You can have any number
>> of in-memory or JDBC job repositories.
>>
>> The data-source attribute value on a jdbc-job-repository resource will
>> use the org.wildfly.data-source [1]. The name of the data-source is used
>> instead of the JNDI which is a much cleaner approach.
>>
>> The thread-factory may be removed and the thread-pool may be changed to
>> use attribute groups (once I figure out how to use them :)).
>>
>> As part of this I considered changing the name from batch to
>> batch-jberet. The main concern I had with this was the web console, but
>> I seem to have broken that anyway with the changes to the model. Does
>> anyone have opinions on a name change to batch-jberet?
>>
>> Also parsing an old configuration may have some issues if the user was
>> using a JDBC job repository. I've currently not found a good way to find
>> a data-source resource name based on a JNDI name. I'm not sure if we
>> should just fail when adding a legacy JDBC job repository. Any
>> suggestions here would be helpful.
>>
>> Any comments or concerns in general are welcome. This is our chance to
>> get it right this time.
>>
>>
>> [1]: https://github.com/wildfly/wildfly/pull/7682
>>
>> --
>> James R. Perkins
>> JBoss by Red Hat
>>
>> _______________________________________________
>> wildfly-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Batch Subsystem Changes

Cheng Fang
In reply to this post by James Perkins
How does it support adding new types of job repository (e.g., infinispan
job repository) in the future?

Cheng

On 7/6/15 3:13 PM, James R. Perkins wrote:

> Hello All,
> The past couple weeks I've been working on basically a redo of the batch
> subsystem. Almost the entire management model is changing to hopefully
> make it more user friendly.
>
> In WildFly 8 and WildFly 9 the model looked like the following:
>
> {
>       "job-repository-type" => "in-memory",
>       "job-repository" => {"jdbc" => {"jndi-name" => undefined}},
>       "thread-factory" => undefined,
>       "thread-pool" => {"batch" => {
>           "keepalive-time" => {
>               "time" => 30L,
>               "unit" => "SECONDS"
>           },
>           "max-threads" => 10,
>           "name" => "batch",
>           "thread-factory" => undefined
>       }}
> }
>
> The job-repository-type could either be jdcb or in-memory. The jndi-name
> attribute on the single job-repository=jdbc resource could be undefined
> indicating the default data-source should be used or JNDI name to look
> up the data-source with no validation being done until the user actually
> tries to deploy a batch deployment.
>
> The thread-pool and thread-factory are the same as other resources that
> use the thread "subsystem" shared resources.
>
> As you can see it's not very intuitive and somewhat clumsy to say the
> least. Only a single job-repository could be defined which isn't great
> for multiple deployments.
>
> In WildFly 10 the model, at least currently, will look like:
>
> {
>       "default-job-repository" => "default",
>       "in-memory-job-repository" => {"default" => {}},
>       "jdbc-job-repository" => {"jdbc" => {"data-source" => "ExampleDS"}},
>       "thread-factory" => undefined,
>       "thread-pool" => {"batch" => {
>           "active-count" => 0,
>           "completed-task-count" => 0L,
>           "current-thread-count" => 0,
>           "keepalive-time" => {
>               "time" => 30L,
>               "unit" => "SECONDS"
>           },
>           "largest-thread-count" => 0,
>           "max-threads" => 10,
>           "name" => "batch",
>           "queue-size" => 0,
>           "rejected-count" => 0,
>           "task-count" => 0L,
>           "thread-factory" => undefined
>       }}
> }
>
> The default-job-repository will be an attribute similar to the previous
> job-repository attribute. The difference being you can use any named
> in-memory-job-repository or jdbc-job-repository. You can have any number
> of in-memory or JDBC job repositories.
>
> The data-source attribute value on a jdbc-job-repository resource will
> use the org.wildfly.data-source [1]. The name of the data-source is used
> instead of the JNDI which is a much cleaner approach.
>
> The thread-factory may be removed and the thread-pool may be changed to
> use attribute groups (once I figure out how to use them :)).
>
> As part of this I considered changing the name from batch to
> batch-jberet. The main concern I had with this was the web console, but
> I seem to have broken that anyway with the changes to the model. Does
> anyone have opinions on a name change to batch-jberet?
>
> Also parsing an old configuration may have some issues if the user was
> using a JDBC job repository. I've currently not found a good way to find
> a data-source resource name based on a JNDI name. I'm not sure if we
> should just fail when adding a legacy JDBC job repository. Any
> suggestions here would be helpful.
>
> Any comments or concerns in general are welcome. This is our chance to
> get it right this time.
>
>
> [1]: https://github.com/wildfly/wildfly/pull/7682
>

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Batch Subsystem Changes

Darran Lofthouse
Related to that - how does this fit with capabilities and requirements?
  Could other subsystems register their own job repository or does this
all need to happen within the batch subsystem?

On 07/07/15 03:31, Cheng Fang wrote:

> How does it support adding new types of job repository (e.g., infinispan
> job repository) in the future?
>
> Cheng
>
> On 7/6/15 3:13 PM, James R. Perkins wrote:
>> Hello All,
>> The past couple weeks I've been working on basically a redo of the batch
>> subsystem. Almost the entire management model is changing to hopefully
>> make it more user friendly.
>>
>> In WildFly 8 and WildFly 9 the model looked like the following:
>>
>> {
>>        "job-repository-type" => "in-memory",
>>        "job-repository" => {"jdbc" => {"jndi-name" => undefined}},
>>        "thread-factory" => undefined,
>>        "thread-pool" => {"batch" => {
>>            "keepalive-time" => {
>>                "time" => 30L,
>>                "unit" => "SECONDS"
>>            },
>>            "max-threads" => 10,
>>            "name" => "batch",
>>            "thread-factory" => undefined
>>        }}
>> }
>>
>> The job-repository-type could either be jdcb or in-memory. The jndi-name
>> attribute on the single job-repository=jdbc resource could be undefined
>> indicating the default data-source should be used or JNDI name to look
>> up the data-source with no validation being done until the user actually
>> tries to deploy a batch deployment.
>>
>> The thread-pool and thread-factory are the same as other resources that
>> use the thread "subsystem" shared resources.
>>
>> As you can see it's not very intuitive and somewhat clumsy to say the
>> least. Only a single job-repository could be defined which isn't great
>> for multiple deployments.
>>
>> In WildFly 10 the model, at least currently, will look like:
>>
>> {
>>        "default-job-repository" => "default",
>>        "in-memory-job-repository" => {"default" => {}},
>>        "jdbc-job-repository" => {"jdbc" => {"data-source" => "ExampleDS"}},
>>        "thread-factory" => undefined,
>>        "thread-pool" => {"batch" => {
>>            "active-count" => 0,
>>            "completed-task-count" => 0L,
>>            "current-thread-count" => 0,
>>            "keepalive-time" => {
>>                "time" => 30L,
>>                "unit" => "SECONDS"
>>            },
>>            "largest-thread-count" => 0,
>>            "max-threads" => 10,
>>            "name" => "batch",
>>            "queue-size" => 0,
>>            "rejected-count" => 0,
>>            "task-count" => 0L,
>>            "thread-factory" => undefined
>>        }}
>> }
>>
>> The default-job-repository will be an attribute similar to the previous
>> job-repository attribute. The difference being you can use any named
>> in-memory-job-repository or jdbc-job-repository. You can have any number
>> of in-memory or JDBC job repositories.
>>
>> The data-source attribute value on a jdbc-job-repository resource will
>> use the org.wildfly.data-source [1]. The name of the data-source is used
>> instead of the JNDI which is a much cleaner approach.
>>
>> The thread-factory may be removed and the thread-pool may be changed to
>> use attribute groups (once I figure out how to use them :)).
>>
>> As part of this I considered changing the name from batch to
>> batch-jberet. The main concern I had with this was the web console, but
>> I seem to have broken that anyway with the changes to the model. Does
>> anyone have opinions on a name change to batch-jberet?
>>
>> Also parsing an old configuration may have some issues if the user was
>> using a JDBC job repository. I've currently not found a good way to find
>> a data-source resource name based on a JNDI name. I'm not sure if we
>> should just fail when adding a legacy JDBC job repository. Any
>> suggestions here would be helpful.
>>
>> Any comments or concerns in general are welcome. This is our chance to
>> get it right this time.
>>
>>
>> [1]: https://github.com/wildfly/wildfly/pull/7682
>>
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Batch Subsystem Changes

James Perkins
In reply to this post by jtgreene
Yeah that's definitely an option. One of the biggest issues with the
current model is the JNDI name can't be validated.

In a perfect world I'd definitely not like to change the model. However
my original design of the model was rather poor. Since there will be
other subsystems changing in WildFly 10 I thought now would be the right
time to break if we're going to break it at all.

On 07/06/2015 06:05 PM, Jason T. Greene wrote:

> Actually in this case you could support both the old and the new subsystem as its a one to one mapping. Then you do not need the migration bit.
>
> Sent from my iPhone
>
>> On Jul 6, 2015, at 8:02 PM, Jason T. Greene <[hidden email]> wrote:
>>
>> IMO we should avoid breaking compatibility unless there is no other option. If we do break compatibility we should rename the subsystem and provide a migration operation.
>>
>>> On Jul 6, 2015, at 2:13 PM, James R. Perkins <[hidden email]> wrote:
>>>
>>> Hello All,
>>> The past couple weeks I've been working on basically a redo of the batch
>>> subsystem. Almost the entire management model is changing to hopefully
>>> make it more user friendly.
>>>
>>> In WildFly 8 and WildFly 9 the model looked like the following:
>>>
>>> {
>>>     "job-repository-type" => "in-memory",
>>>     "job-repository" => {"jdbc" => {"jndi-name" => undefined}},
>>>     "thread-factory" => undefined,
>>>     "thread-pool" => {"batch" => {
>>>         "keepalive-time" => {
>>>             "time" => 30L,
>>>             "unit" => "SECONDS"
>>>         },
>>>         "max-threads" => 10,
>>>         "name" => "batch",
>>>         "thread-factory" => undefined
>>>     }}
>>> }
>>>
>>> The job-repository-type could either be jdcb or in-memory. The jndi-name
>>> attribute on the single job-repository=jdbc resource could be undefined
>>> indicating the default data-source should be used or JNDI name to look
>>> up the data-source with no validation being done until the user actually
>>> tries to deploy a batch deployment.
>>>
>>> The thread-pool and thread-factory are the same as other resources that
>>> use the thread "subsystem" shared resources.
>>>
>>> As you can see it's not very intuitive and somewhat clumsy to say the
>>> least. Only a single job-repository could be defined which isn't great
>>> for multiple deployments.
>>>
>>> In WildFly 10 the model, at least currently, will look like:
>>>
>>> {
>>>     "default-job-repository" => "default",
>>>     "in-memory-job-repository" => {"default" => {}},
>>>     "jdbc-job-repository" => {"jdbc" => {"data-source" => "ExampleDS"}},
>>>     "thread-factory" => undefined,
>>>     "thread-pool" => {"batch" => {
>>>         "active-count" => 0,
>>>         "completed-task-count" => 0L,
>>>         "current-thread-count" => 0,
>>>         "keepalive-time" => {
>>>             "time" => 30L,
>>>             "unit" => "SECONDS"
>>>         },
>>>         "largest-thread-count" => 0,
>>>         "max-threads" => 10,
>>>         "name" => "batch",
>>>         "queue-size" => 0,
>>>         "rejected-count" => 0,
>>>         "task-count" => 0L,
>>>         "thread-factory" => undefined
>>>     }}
>>> }
>>>
>>> The default-job-repository will be an attribute similar to the previous
>>> job-repository attribute. The difference being you can use any named
>>> in-memory-job-repository or jdbc-job-repository. You can have any number
>>> of in-memory or JDBC job repositories.
>>>
>>> The data-source attribute value on a jdbc-job-repository resource will
>>> use the org.wildfly.data-source [1]. The name of the data-source is used
>>> instead of the JNDI which is a much cleaner approach.
>>>
>>> The thread-factory may be removed and the thread-pool may be changed to
>>> use attribute groups (once I figure out how to use them :)).
>>>
>>> As part of this I considered changing the name from batch to
>>> batch-jberet. The main concern I had with this was the web console, but
>>> I seem to have broken that anyway with the changes to the model. Does
>>> anyone have opinions on a name change to batch-jberet?
>>>
>>> Also parsing an old configuration may have some issues if the user was
>>> using a JDBC job repository. I've currently not found a good way to find
>>> a data-source resource name based on a JNDI name. I'm not sure if we
>>> should just fail when adding a legacy JDBC job repository. Any
>>> suggestions here would be helpful.
>>>
>>> Any comments or concerns in general are welcome. This is our chance to
>>> get it right this time.
>>>
>>>
>>> [1]: https://github.com/wildfly/wildfly/pull/7682
>>>
>>> --
>>> James R. Perkins
>>> JBoss by Red Hat
>>>
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev

--
James R. Perkins
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Batch Subsystem Changes

James Perkins
In reply to this post by Cheng Fang


On 07/06/2015 07:31 PM, Cheng Fang wrote:
> How does it support adding new types of job repository (e.g., infinispan
> job repository) in the future?
They would just be new resources on the subsystem. So for infinispan
we'd just have an infinispan-job-repository resource.

>
> Cheng
>
> On 7/6/15 3:13 PM, James R. Perkins wrote:
>> Hello All,
>> The past couple weeks I've been working on basically a redo of the batch
>> subsystem. Almost the entire management model is changing to hopefully
>> make it more user friendly.
>>
>> In WildFly 8 and WildFly 9 the model looked like the following:
>>
>> {
>>        "job-repository-type" => "in-memory",
>>        "job-repository" => {"jdbc" => {"jndi-name" => undefined}},
>>        "thread-factory" => undefined,
>>        "thread-pool" => {"batch" => {
>>            "keepalive-time" => {
>>                "time" => 30L,
>>                "unit" => "SECONDS"
>>            },
>>            "max-threads" => 10,
>>            "name" => "batch",
>>            "thread-factory" => undefined
>>        }}
>> }
>>
>> The job-repository-type could either be jdcb or in-memory. The jndi-name
>> attribute on the single job-repository=jdbc resource could be undefined
>> indicating the default data-source should be used or JNDI name to look
>> up the data-source with no validation being done until the user actually
>> tries to deploy a batch deployment.
>>
>> The thread-pool and thread-factory are the same as other resources that
>> use the thread "subsystem" shared resources.
>>
>> As you can see it's not very intuitive and somewhat clumsy to say the
>> least. Only a single job-repository could be defined which isn't great
>> for multiple deployments.
>>
>> In WildFly 10 the model, at least currently, will look like:
>>
>> {
>>        "default-job-repository" => "default",
>>        "in-memory-job-repository" => {"default" => {}},
>>        "jdbc-job-repository" => {"jdbc" => {"data-source" => "ExampleDS"}},
>>        "thread-factory" => undefined,
>>        "thread-pool" => {"batch" => {
>>            "active-count" => 0,
>>            "completed-task-count" => 0L,
>>            "current-thread-count" => 0,
>>            "keepalive-time" => {
>>                "time" => 30L,
>>                "unit" => "SECONDS"
>>            },
>>            "largest-thread-count" => 0,
>>            "max-threads" => 10,
>>            "name" => "batch",
>>            "queue-size" => 0,
>>            "rejected-count" => 0,
>>            "task-count" => 0L,
>>            "thread-factory" => undefined
>>        }}
>> }
>>
>> The default-job-repository will be an attribute similar to the previous
>> job-repository attribute. The difference being you can use any named
>> in-memory-job-repository or jdbc-job-repository. You can have any number
>> of in-memory or JDBC job repositories.
>>
>> The data-source attribute value on a jdbc-job-repository resource will
>> use the org.wildfly.data-source [1]. The name of the data-source is used
>> instead of the JNDI which is a much cleaner approach.
>>
>> The thread-factory may be removed and the thread-pool may be changed to
>> use attribute groups (once I figure out how to use them :)).
>>
>> As part of this I considered changing the name from batch to
>> batch-jberet. The main concern I had with this was the web console, but
>> I seem to have broken that anyway with the changes to the model. Does
>> anyone have opinions on a name change to batch-jberet?
>>
>> Also parsing an old configuration may have some issues if the user was
>> using a JDBC job repository. I've currently not found a good way to find
>> a data-source resource name based on a JNDI name. I'm not sure if we
>> should just fail when adding a legacy JDBC job repository. Any
>> suggestions here would be helpful.
>>
>> Any comments or concerns in general are welcome. This is our chance to
>> get it right this time.
>>
>>
>> [1]: https://github.com/wildfly/wildfly/pull/7682
>>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

--
James R. Perkins
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Batch Subsystem Changes

James Perkins
In reply to this post by Darran Lofthouse


On 07/07/2015 04:10 AM, Darran Lofthouse wrote:
> Related to that - how does this fit with capabilities and requirements?
>    Could other subsystems register their own job repository or does this
> all need to happen within the batch subsystem?
That's an interesting idea I hadn't thought of. It should work though as
I am using a dynamic capability for the job repository.

>
> On 07/07/15 03:31, Cheng Fang wrote:
>> How does it support adding new types of job repository (e.g., infinispan
>> job repository) in the future?
>>
>> Cheng
>>
>> On 7/6/15 3:13 PM, James R. Perkins wrote:
>>> Hello All,
>>> The past couple weeks I've been working on basically a redo of the batch
>>> subsystem. Almost the entire management model is changing to hopefully
>>> make it more user friendly.
>>>
>>> In WildFly 8 and WildFly 9 the model looked like the following:
>>>
>>> {
>>>         "job-repository-type" => "in-memory",
>>>         "job-repository" => {"jdbc" => {"jndi-name" => undefined}},
>>>         "thread-factory" => undefined,
>>>         "thread-pool" => {"batch" => {
>>>             "keepalive-time" => {
>>>                 "time" => 30L,
>>>                 "unit" => "SECONDS"
>>>             },
>>>             "max-threads" => 10,
>>>             "name" => "batch",
>>>             "thread-factory" => undefined
>>>         }}
>>> }
>>>
>>> The job-repository-type could either be jdcb or in-memory. The jndi-name
>>> attribute on the single job-repository=jdbc resource could be undefined
>>> indicating the default data-source should be used or JNDI name to look
>>> up the data-source with no validation being done until the user actually
>>> tries to deploy a batch deployment.
>>>
>>> The thread-pool and thread-factory are the same as other resources that
>>> use the thread "subsystem" shared resources.
>>>
>>> As you can see it's not very intuitive and somewhat clumsy to say the
>>> least. Only a single job-repository could be defined which isn't great
>>> for multiple deployments.
>>>
>>> In WildFly 10 the model, at least currently, will look like:
>>>
>>> {
>>>         "default-job-repository" => "default",
>>>         "in-memory-job-repository" => {"default" => {}},
>>>         "jdbc-job-repository" => {"jdbc" => {"data-source" => "ExampleDS"}},
>>>         "thread-factory" => undefined,
>>>         "thread-pool" => {"batch" => {
>>>             "active-count" => 0,
>>>             "completed-task-count" => 0L,
>>>             "current-thread-count" => 0,
>>>             "keepalive-time" => {
>>>                 "time" => 30L,
>>>                 "unit" => "SECONDS"
>>>             },
>>>             "largest-thread-count" => 0,
>>>             "max-threads" => 10,
>>>             "name" => "batch",
>>>             "queue-size" => 0,
>>>             "rejected-count" => 0,
>>>             "task-count" => 0L,
>>>             "thread-factory" => undefined
>>>         }}
>>> }
>>>
>>> The default-job-repository will be an attribute similar to the previous
>>> job-repository attribute. The difference being you can use any named
>>> in-memory-job-repository or jdbc-job-repository. You can have any number
>>> of in-memory or JDBC job repositories.
>>>
>>> The data-source attribute value on a jdbc-job-repository resource will
>>> use the org.wildfly.data-source [1]. The name of the data-source is used
>>> instead of the JNDI which is a much cleaner approach.
>>>
>>> The thread-factory may be removed and the thread-pool may be changed to
>>> use attribute groups (once I figure out how to use them :)).
>>>
>>> As part of this I considered changing the name from batch to
>>> batch-jberet. The main concern I had with this was the web console, but
>>> I seem to have broken that anyway with the changes to the model. Does
>>> anyone have opinions on a name change to batch-jberet?
>>>
>>> Also parsing an old configuration may have some issues if the user was
>>> using a JDBC job repository. I've currently not found a good way to find
>>> a data-source resource name based on a JNDI name. I'm not sure if we
>>> should just fail when adding a legacy JDBC job repository. Any
>>> suggestions here would be helpful.
>>>
>>> Any comments or concerns in general are welcome. This is our chance to
>>> get it right this time.
>>>
>>>
>>> [1]: https://github.com/wildfly/wildfly/pull/7682
>>>
>> _______________________________________________
>> wildfly-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

--
James R. Perkins
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Batch Subsystem Changes

James Perkins
In reply to this post by James Perkins
I've got the initial work done for this now [1] (only the last commit).
It still needs to be determined how a legacy configuration with a JDBC
job repository will work. The legacy configuration used a JNDI name
which won't work with the data source capabilities. If anyone has ideas
here that would be helpful.

I didn't do any work to support both subsystems running side-by-side. If
that's something that should be done I think it would be fairly easy to do.

[1]: https://github.com/wildfly/wildfly/compare/master...jamezp:WFLY-4811

On 07/06/2015 12:13 PM, James R. Perkins wrote:

> Hello All,
> The past couple weeks I've been working on basically a redo of the
> batch subsystem. Almost the entire management model is changing to
> hopefully make it more user friendly.
>
> In WildFly 8 and WildFly 9 the model looked like the following:
>
> {
>     "job-repository-type" => "in-memory",
>     "job-repository" => {"jdbc" => {"jndi-name" => undefined}},
>     "thread-factory" => undefined,
>     "thread-pool" => {"batch" => {
>         "keepalive-time" => {
>             "time" => 30L,
>             "unit" => "SECONDS"
>         },
>         "max-threads" => 10,
>         "name" => "batch",
>         "thread-factory" => undefined
>     }}
> }
>
> The job-repository-type could either be jdcb or in-memory. The
> jndi-name attribute on the single job-repository=jdbc resource could
> be undefined indicating the default data-source should be used or JNDI
> name to look up the data-source with no validation being done until
> the user actually tries to deploy a batch deployment.
>
> The thread-pool and thread-factory are the same as other resources
> that use the thread "subsystem" shared resources.
>
> As you can see it's not very intuitive and somewhat clumsy to say the
> least. Only a single job-repository could be defined which isn't great
> for multiple deployments.
>
> In WildFly 10 the model, at least currently, will look like:
>
> {
>     "default-job-repository" => "default",
>     "in-memory-job-repository" => {"default" => {}},
>     "jdbc-job-repository" => {"jdbc" => {"data-source" => "ExampleDS"}},
>     "thread-factory" => undefined,
>     "thread-pool" => {"batch" => {
>         "active-count" => 0,
>         "completed-task-count" => 0L,
>         "current-thread-count" => 0,
>         "keepalive-time" => {
>             "time" => 30L,
>             "unit" => "SECONDS"
>         },
>         "largest-thread-count" => 0,
>         "max-threads" => 10,
>         "name" => "batch",
>         "queue-size" => 0,
>         "rejected-count" => 0,
>         "task-count" => 0L,
>         "thread-factory" => undefined
>     }}
> }
>
> The default-job-repository will be an attribute similar to the
> previous job-repository attribute. The difference being you can use
> any named in-memory-job-repository or jdbc-job-repository. You can
> have any number of in-memory or JDBC job repositories.
>
> The data-source attribute value on a jdbc-job-repository resource will
> use the org.wildfly.data-source [1]. The name of the data-source is
> used instead of the JNDI which is a much cleaner approach.
>
> The thread-factory may be removed and the thread-pool may be changed
> to use attribute groups (once I figure out how to use them :)).
>
> As part of this I considered changing the name from batch to
> batch-jberet. The main concern I had with this was the web console,
> but I seem to have broken that anyway with the changes to the model.
> Does anyone have opinions on a name change to batch-jberet?
>
> Also parsing an old configuration may have some issues if the user was
> using a JDBC job repository. I've currently not found a good way to
> find a data-source resource name based on a JNDI name. I'm not sure if
> we should just fail when adding a legacy JDBC job repository. Any
> suggestions here would be helpful.
>
> Any comments or concerns in general are welcome. This is our chance to
> get it right this time.
>
>
> [1]: https://github.com/wildfly/wildfly/pull/7682
>

--
James R. Perkins
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev