Batch Deployments

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Batch Deployments

James Perkins
I'm working on allowing batch jobs to be viewed in the management model
and I'm running into some issues.

Batch jobs require a XML file to start. Per the batch spec that the XML
files can be found outside of the archive, for example somewhere on the
file system. If the job XML isn't found there it looks in the
META-INF/batch-jobs directory for the job XML.

Batch repositories are global for all applications. All deployments can
see all other deployments job status and query information about the
jobs. They can't start or restart jobs for other deployments, but
they're viewable.

Here lies the problem. It seems batch jobs, at least from the management
view, should be limited to the deployment the job was run on. I'm
considering only allowing job XML files in the META-INF/batch-jobs to be
viewable via management. Though there is still a chance two different
deployments could use the same job name (the name of the job XML file)
which would show the jobs run by the two different deployments with the
same job name.

I can't think of another way to isolate jobs from the repository to link
to a deployment. If anyone else has any ideas let me know. There is no
spec for the repository so we can do whatever we want really.

--
James R. Perkins
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Batch Deployments

Kohei Nozaki
As to querying history of execution, job_instance.applicationname would help distinguish between deployments.

On Feb 20, 2015, at 1:49, James R. Perkins <[hidden email]> wrote:

> I'm working on allowing batch jobs to be viewed in the management model
> and I'm running into some issues.
>
> Batch jobs require a XML file to start. Per the batch spec that the XML
> files can be found outside of the archive, for example somewhere on the
> file system. If the job XML isn't found there it looks in the
> META-INF/batch-jobs directory for the job XML.
>
> Batch repositories are global for all applications. All deployments can
> see all other deployments job status and query information about the
> jobs. They can't start or restart jobs for other deployments, but
> they're viewable.
>
> Here lies the problem. It seems batch jobs, at least from the management
> view, should be limited to the deployment the job was run on. I'm
> considering only allowing job XML files in the META-INF/batch-jobs to be
> viewable via management. Though there is still a chance two different
> deployments could use the same job name (the name of the job XML file)
> which would show the jobs run by the two different deployments with the
> same job name.
>
> I can't think of another way to isolate jobs from the repository to link
> to a deployment. If anyone else has any ideas let me know. There is no
> spec for the repository so we can do whatever we want really.
>
> --
> James R. Perkins
> JBoss by Red Hat
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Batch Deployments

James Perkins
I'm not sure I could use that. It would likely be a performance impact
doing a JNDI lookup on each JobInstance. Also if the name of the app
changes, for example it's deployed with a suffix of 1.0.1 and then
1.0.2, you'd lose  previously run jobs for the app.

Unless of course I'm misunderstanding what you meant :)

On 02/19/2015 03:13 PM, Kohei Nozaki wrote:

> As to querying history of execution, job_instance.applicationname would help distinguish between deployments.
>
> On Feb 20, 2015, at 1:49, James R. Perkins <[hidden email]> wrote:
>
>> I'm working on allowing batch jobs to be viewed in the management model
>> and I'm running into some issues.
>>
>> Batch jobs require a XML file to start. Per the batch spec that the XML
>> files can be found outside of the archive, for example somewhere on the
>> file system. If the job XML isn't found there it looks in the
>> META-INF/batch-jobs directory for the job XML.
>>
>> Batch repositories are global for all applications. All deployments can
>> see all other deployments job status and query information about the
>> jobs. They can't start or restart jobs for other deployments, but
>> they're viewable.
>>
>> Here lies the problem. It seems batch jobs, at least from the management
>> view, should be limited to the deployment the job was run on. I'm
>> considering only allowing job XML files in the META-INF/batch-jobs to be
>> viewable via management. Though there is still a chance two different
>> deployments could use the same job name (the name of the job XML file)
>> which would show the jobs run by the two different deployments with the
>> same job name.
>>
>> I can't think of another way to isolate jobs from the repository to link
>> to a deployment. If anyone else has any ideas let me know. There is no
>> spec for the repository so we can do whatever we want really.
>>
>> --
>> James R. Perkins
>> JBoss by Red Hat
>>
>> _______________________________________________
>> wildfly-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

--
James R. Perkins
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Batch Deployments

Cheng Fang
applicationName in JobInstance or JOB_INSTANCE table, along with
jobName, should be able to uniquely identify a job.  applicationName is
set when the jobInstance is created.  There may be some internal API to
obtain it instead of JNDI lookup to avoid any performance hit, though we
currently don't know how big it will be.

If a deployment's name changes, e.g., thru versioning, then I would
think the intent is to separate it from previous versions, and in that
case, any historical job data should only belong to the previous ones,
and not visible to the deployment with the new name.

Just my 2 cents.

Cheng

On 2/19/15 7:29 PM, James R. Perkins wrote:

> I'm not sure I could use that. It would likely be a performance impact
> doing a JNDI lookup on each JobInstance. Also if the name of the app
> changes, for example it's deployed with a suffix of 1.0.1 and then
> 1.0.2, you'd lose  previously run jobs for the app.
>
> Unless of course I'm misunderstanding what you meant :)
>
> On 02/19/2015 03:13 PM, Kohei Nozaki wrote:
>> As to querying history of execution, job_instance.applicationname would help distinguish between deployments.
>>
>> On Feb 20, 2015, at 1:49, James R. Perkins <[hidden email]> wrote:
>>
>>> I'm working on allowing batch jobs to be viewed in the management model
>>> and I'm running into some issues.
>>>
>>> Batch jobs require a XML file to start. Per the batch spec that the XML
>>> files can be found outside of the archive, for example somewhere on the
>>> file system. If the job XML isn't found there it looks in the
>>> META-INF/batch-jobs directory for the job XML.
>>>
>>> Batch repositories are global for all applications. All deployments can
>>> see all other deployments job status and query information about the
>>> jobs. They can't start or restart jobs for other deployments, but
>>> they're viewable.
>>>
>>> Here lies the problem. It seems batch jobs, at least from the management
>>> view, should be limited to the deployment the job was run on. I'm
>>> considering only allowing job XML files in the META-INF/batch-jobs to be
>>> viewable via management. Though there is still a chance two different
>>> deployments could use the same job name (the name of the job XML file)
>>> which would show the jobs run by the two different deployments with the
>>> same job name.
>>>
>>> I can't think of another way to isolate jobs from the repository to link
>>> to a deployment. If anyone else has any ideas let me know. There is no
>>> spec for the repository so we can do whatever we want really.
>>>
>>> --
>>> James R. Perkins
>>> JBoss by Red Hat
>>>
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>> _______________________________________________
>> wildfly-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Batch Deployments

James Perkins
If we use the application name we lose some performance as everything
needs to be dynamic. We'll have to query the job names and determine
which jobs can be displayed for the deployment each time a read
operation is done on the /deployment=some.war/subsystem=batch/job=*
resource. We also may lose the visibility to batch jobs that have run
previously if the application name changes.

If we use the job XML files from the META-INF/batch-jobs we get better
performance because we know the job names during deployment processing
and we can register those resources during processing rather than having
them dynamic. We do however lose visibility over batch jobs where the
job XML lives outside the deployment. We could also run into collisions
where two deployments have the same job name, though that seems like it
would be rather rare.

I'm not really sure how often job XML files will be used that are stored
outside the deployment. If we knew it was going to be quite often, I'd
lean towards the first option. If it's rare I'd lean towards the second
option.

On 02/19/2015 06:02 PM, Cheng Fang wrote:

> applicationName in JobInstance or JOB_INSTANCE table, along with
> jobName, should be able to uniquely identify a job.  applicationName is
> set when the jobInstance is created.  There may be some internal API to
> obtain it instead of JNDI lookup to avoid any performance hit, though we
> currently don't know how big it will be.
>
> If a deployment's name changes, e.g., thru versioning, then I would
> think the intent is to separate it from previous versions, and in that
> case, any historical job data should only belong to the previous ones,
> and not visible to the deployment with the new name.
>
> Just my 2 cents.
>
> Cheng
>
> On 2/19/15 7:29 PM, James R. Perkins wrote:
>> I'm not sure I could use that. It would likely be a performance impact
>> doing a JNDI lookup on each JobInstance. Also if the name of the app
>> changes, for example it's deployed with a suffix of 1.0.1 and then
>> 1.0.2, you'd lose  previously run jobs for the app.
>>
>> Unless of course I'm misunderstanding what you meant :)
>>
>> On 02/19/2015 03:13 PM, Kohei Nozaki wrote:
>>> As to querying history of execution, job_instance.applicationname would help distinguish between deployments.
>>>
>>> On Feb 20, 2015, at 1:49, James R. Perkins <[hidden email]> wrote:
>>>
>>>> I'm working on allowing batch jobs to be viewed in the management model
>>>> and I'm running into some issues.
>>>>
>>>> Batch jobs require a XML file to start. Per the batch spec that the XML
>>>> files can be found outside of the archive, for example somewhere on the
>>>> file system. If the job XML isn't found there it looks in the
>>>> META-INF/batch-jobs directory for the job XML.
>>>>
>>>> Batch repositories are global for all applications. All deployments can
>>>> see all other deployments job status and query information about the
>>>> jobs. They can't start or restart jobs for other deployments, but
>>>> they're viewable.
>>>>
>>>> Here lies the problem. It seems batch jobs, at least from the management
>>>> view, should be limited to the deployment the job was run on. I'm
>>>> considering only allowing job XML files in the META-INF/batch-jobs to be
>>>> viewable via management. Though there is still a chance two different
>>>> deployments could use the same job name (the name of the job XML file)
>>>> which would show the jobs run by the two different deployments with the
>>>> same job name.
>>>>
>>>> I can't think of another way to isolate jobs from the repository to link
>>>> to a deployment. If anyone else has any ideas let me know. There is no
>>>> spec for the repository so we can do whatever we want really.
>>>>
>>>> --
>>>> James R. Perkins
>>>> JBoss by Red Hat
>>>>
>>>> _______________________________________________
>>>> wildfly-dev mailing list
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

--
James R. Perkins
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev