JBoss Modules - "advanced" PathFilter for ResourceLoaderSpec?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

JBoss Modules - "advanced" PathFilter for ResourceLoaderSpec?

Jaikiran Pai-2
(Using this list since I couldn't find a place to ask JBoss Modules question anywhere else, feel free to direct me there if there's one)

Hello everyone :)

I have been using JBoss Modules for one of the projects I'm involved in. The usage there is pretty much similar to how we use it for setting up classloaders in WildFly server for deployments. A new module M1 gets dynamically created and assigned to a component and this module is added with dependencies to some pre-defined static modules A, B, C and such.  M1 also gets N number of resource roots (backed by ResourceLoaderSpec), each pointing to a jar file within some well known directory. So to put in some sort of a code, it looks like:

// add each jar as a resource root for the module
for (final File jar : jars) {
    final ResourceLoader jarResourceLoader;
    try {
        jarResourceLoader = ResourceLoaders.createJarResourceLoader(jar.getName(), new JarFile(jar));
    } catch (IOException e) {
        // log and continue
        logger.warn("....", e);
        continue;
    }
    moduleSpecBuilder.addResourceRoot(ResourceLoaderSpec.createResourceLoaderSpec(jarResourceLoader));
}



All works fine without any issues and the module M1 has access to the resources in these jars. Now there are times where the components for which I've created this module M1 and attach these jars, "accidentally" ship/package jars which have resources (classes to be precise) which are also exposed/present in one of the dependency modules (remember A, B, C...). So this then leads to the same old thing where I have go back and tell my users not to package such jars.

I want to try and make this a bit more robust and get away with having to tell users not to package xyz jars. I had a look at the ResourceLoaderSpec interface and it takes a PathFilter https://github.com/jboss-modules/jboss-modules/blob/1.x/src/main/java/org/jboss/modules/ResourceLoaderSpec.java#L48 which gets me one step closer to what I want to achieve. So if I know that static module A, B, C etc... expose classes belonging to package foo.bar.blah, I can setup a PathFilter on these jar resource loader to skip/decline the path in the accept() method. I think that should work out fine (I need to test it out tonight) and I wouldn't have to worry that some jar packaged within that component will introduce these classes belonging to the foo.bar.blah package.

However, although it might work, I then have to keep a very close vigil or rather keep inspecting what packages (or resources in general) the modules A, B and C provide. Instead what I'm thinking of is a "smart" PathFilter or anything along those lines whose semantics would be to "skip/don't accept/filter out all those resources, from a resource root, if the resource is provided by any of the specified modules". So something like:

ResourceLoaderSpec.createResourceLoaderSpec(jarResourceLoader, PathFilters.excludeResourcesExposedByModules("A:slot", "B:slot", "C:slot" ...));


Having looked at the JBoss Modules code, I don't think this is possible currently. But that's OK. What I really want to check is, is this something that would be feasible to implement (doesn't have to be in JBoss Modules itself) and is there any obvious issues with the approach? Also, is this something that would be useful to have in JBoss Modules itself?


-Jaikiran 

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: JBoss Modules - "advanced" PathFilter for ResourceLoaderSpec?

David M. Lloyd
Hi Jaikiran!

On 12/08/2016 05:59 AM, Jaikiran Pai wrote:

> (Using this list since I couldn't find a place to ask JBoss Modules
> question anywhere else, feel free to direct me there if there's one)
>
> Hello everyone :)
>
> I have been using JBoss Modules for one of the projects I'm involved in.
> The usage there is pretty much similar to how we use it for setting up
> classloaders in WildFly server for deployments. A new module M1 gets
> dynamically created and assigned to a component and this module is added
> with dependencies to some pre-defined static modules A, B, C and such.
> M1 also gets N number of resource roots (backed by ResourceLoaderSpec),
> each pointing to a jar file within some well known directory. So to put
> in some sort of a code, it looks like:
>
> // add each jar as a resource root for the module
> for (final File jar : jars) {
>     final ResourceLoader jarResourceLoader;
>     try {
>         jarResourceLoader =
> ResourceLoaders.createJarResourceLoader(jar.getName(), new JarFile(jar));
>     } catch (IOException e) {
>         // log and continue
>         logger.warn("....", e);
>         continue;
>     }
>
> moduleSpecBuilder.addResourceRoot(ResourceLoaderSpec.createResourceLoaderSpec(jarResourceLoader));
> }
>
>
>
> All works fine without any issues and the module M1 has access to the
> resources in these jars. Now there are times where the components for
> which I've created this module M1 and attach these jars, "accidentally"
> ship/package jars which have resources (classes to be precise) which are
> also exposed/present in one of the dependency modules (remember A, B,
> C...). So this then leads to the same old thing where I have go back and
> tell my users not to package such jars.
>
> I want to try and make this a bit more robust and get away with having
> to tell users not to package xyz jars. I had a look at the
> ResourceLoaderSpec interface and it takes a PathFilter
> https://github.com/jboss-modules/jboss-modules/blob/1.x/src/main/java/org/jboss/modules/ResourceLoaderSpec.java#L48
> which gets me one step closer to what I want to achieve. So if I know
> that static module A, B, C etc... expose classes belonging to package
> foo.bar.blah, I can setup a PathFilter on these jar resource loader to
> skip/decline the path in the accept() method. I think that should work
> out fine (I need to test it out tonight) and I wouldn't have to worry
> that some jar packaged within that component will introduce these
> classes belonging to the foo.bar.blah package.
>
> However, although it might work, I then have to keep a very close vigil
> or rather keep inspecting what packages (or resources in general) the
> modules A, B and C provide. Instead what I'm thinking of is a "smart"
> PathFilter or anything along those lines whose semantics would be to
> "skip/don't accept/filter out all those resources, from a resource root,
> if the resource is provided by any of the specified modules". So
> something like:
>
> ResourceLoaderSpec.createResourceLoaderSpec(jarResourceLoader,
> *PathFilters.excludeResourcesExposedByModules("A**:slot", "B:slot",
> "C:slot" ...)*);
>
>
> Having looked at the JBoss Modules code, I don't think this is possible
> currently. But that's OK. What I really want to check is, is this
> something that would be feasible to implement (doesn't have to be in
> JBoss Modules itself) and is there any obvious issues with the approach?
> Also, is this something that would be useful to have in JBoss Modules
> itself?

A smart filter is a pretty good idea!  I think that it might make more
sense as part of a dependency specification though, from a user's
perspective: if I say "I depend on module org.foo.bar:main" I should
also be able to say "...and use their packages" which would exclude all
paths in the source module's resource roots (other than META-INF and the
root path).

In fact, maybe that ought to be the default setting, with the user
having to opt in to override packages.  So a user might say:

    <dependencies>
        ...
        <module name="org.foo.bar">
            <override-paths/> <!-- now my paths take precedence -->
        </module>
        ...
    </dependencies>

In the programmatic API it'd be fairly straightforward as well.  We
could add an overridePaths path filter to the dependency specification
(which ought to be builder-based at this point, I now realize, what with
the parameter explosion).  Any dependency paths that are not matched by
that filter would be subtracted from all resource roots.

As far as XML compatibility goes, the default for 1.5 and earlier can be
to override all paths, whereas the default for 1.6 and later can be to
only override META-INF and the root path.

We'd have to have logic to allow for dependency recalculation when a
module is relinked, but that shouldn't be too hard.

What do you think?
--
- DML
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: JBoss Modules - "advanced" PathFilter for ResourceLoaderSpec?

Jaikiran Pai-2
Hi David,

On Thursday 08 December 2016 07:29 PM, David M. Lloyd wrote:

> Hi Jaikiran!
>
> On 12/08/2016 05:59 AM, Jaikiran Pai wrote:
>> However, although it might work, I then have to keep a very close vigil
>> or rather keep inspecting what packages (or resources in general) the
>> modules A, B and C provide. Instead what I'm thinking of is a "smart"
>> PathFilter or anything along those lines whose semantics would be to
>> "skip/don't accept/filter out all those resources, from a resource root,
>> if the resource is provided by any of the specified modules". So
>> something like:
>>
>> ResourceLoaderSpec.createResourceLoaderSpec(jarResourceLoader,
>> *PathFilters.excludeResourcesExposedByModules("A**:slot", "B:slot",
>> "C:slot" ...)*);
>>
>>
>> Having looked at the JBoss Modules code, I don't think this is possible
>> currently. But that's OK. What I really want to check is, is this
>> something that would be feasible to implement (doesn't have to be in
>> JBoss Modules itself) and is there any obvious issues with the approach?
>> Also, is this something that would be useful to have in JBoss Modules
>> itself?
> A smart filter is a pretty good idea!  I think that it might make more
> sense as part of a dependency specification though,

Agreed, having it on the dependency spec makes more sense and puts it in
the right context.


>   from a user's
> perspective: if I say "I depend on module org.foo.bar:main" I should
> also be able to say "...and use their packages" which would exclude all
> paths in the source module's resource roots (other than META-INF and the
> root path).

When you say "would exclude all paths in the source module's resource
roots...", you mean precedence will be given to the paths of the
dependency module, right? Not literally exclude all paths of the
source's resource roots? That way if (one or more) resource roots of the
source module has a path "hello/world" which isn't part of the
dependency module's paths, then it will get "accepted" and be available
to the source module. Did I understand this right or did you intend to
literally mean excluding all paths of the source module's resource roots?


> In fact, maybe that ought to be the default setting, with the user
> having to opt in to override packages.  So a user might say:
>
>      <dependencies>
>          ...
>          <module name="org.foo.bar">
>              <override-paths/> <!-- now my paths take precedence -->
>          </module>
>          ...
>      </dependencies>

Yes, this looks logical.

> In the programmatic API it'd be fairly straightforward as well.  We
> could add an overridePaths path filter to the dependency specification
> (which ought to be builder-based at this point, I now realize, what with
> the parameter explosion).  Any dependency paths that are not matched by
> that filter would be subtracted from all resource roots.

Yes, this matches with my expectation.


-Jaikiran
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: JBoss Modules - "advanced" PathFilter for ResourceLoaderSpec?

David M. Lloyd
On 12/08/2016 08:41 AM, Jaikiran Pai wrote:

> Hi David,
>
> On Thursday 08 December 2016 07:29 PM, David M. Lloyd wrote:
>> Hi Jaikiran!
>>
>> On 12/08/2016 05:59 AM, Jaikiran Pai wrote:
>>> However, although it might work, I then have to keep a very close vigil
>>> or rather keep inspecting what packages (or resources in general) the
>>> modules A, B and C provide. Instead what I'm thinking of is a "smart"
>>> PathFilter or anything along those lines whose semantics would be to
>>> "skip/don't accept/filter out all those resources, from a resource root,
>>> if the resource is provided by any of the specified modules". So
>>> something like:
>>>
>>> ResourceLoaderSpec.createResourceLoaderSpec(jarResourceLoader,
>>> *PathFilters.excludeResourcesExposedByModules("A**:slot", "B:slot",
>>> "C:slot" ...)*);
>>>
>>>
>>> Having looked at the JBoss Modules code, I don't think this is possible
>>> currently. But that's OK. What I really want to check is, is this
>>> something that would be feasible to implement (doesn't have to be in
>>> JBoss Modules itself) and is there any obvious issues with the approach?
>>> Also, is this something that would be useful to have in JBoss Modules
>>> itself?
>> A smart filter is a pretty good idea!  I think that it might make more
>> sense as part of a dependency specification though,
>
> Agreed, having it on the dependency spec makes more sense and puts it in
> the right context.
>
>
>>   from a user's
>> perspective: if I say "I depend on module org.foo.bar:main" I should
>> also be able to say "...and use their packages" which would exclude all
>> paths in the source module's resource roots (other than META-INF and the
>> root path).
>
> When you say "would exclude all paths in the source module's resource
> roots...", you mean precedence will be given to the paths of the
> dependency module, right? Not literally exclude all paths of the
> source's resource roots? That way if (one or more) resource roots of the
> source module has a path "hello/world" which isn't part of the
> dependency module's paths, then it will get "accepted" and be available
> to the source module. Did I understand this right or did you intend to
> literally mean excluding all paths of the source module's resource roots?

I mean literally excluding.  Dependency-first (aka "parent-first")
loading is fundamentally flawed in any non-tree graph.  To boil it down
to the degenerate case, if I have two modules A and B which have a
mutual interdependency and a common resource or class, using
parent-first loading, A gets B's resource, while B gets A's resource,
which is just plain weird.

Using local-first (aka "child-first") loading (as we do) and excluding
local resource paths that are found in dependencies, A always gets A's
resource, and B always gets B's resource.  If you want both, then you
simply don't override, and A gets A's and then B's, and B gets B's and
then A's.

Does that make sense?

>> In fact, maybe that ought to be the default setting, with the user
>> having to opt in to override packages.  So a user might say:
>>
>>      <dependencies>
>>          ...
>>          <module name="org.foo.bar">
>>              <override-paths/> <!-- now my paths take precedence -->
>>          </module>
>>          ...
>>      </dependencies>
>
> Yes, this looks logical.
>
>> In the programmatic API it'd be fairly straightforward as well.  We
>> could add an overridePaths path filter to the dependency specification
>> (which ought to be builder-based at this point, I now realize, what with
>> the parameter explosion).  Any dependency paths that are not matched by
>> that filter would be subtracted from all resource roots.
>
> Yes, this matches with my expectation.

Great!  I'll open up a JIRA for this enhancement.
--
- DML
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: JBoss Modules - "advanced" PathFilter for ResourceLoaderSpec?

Jaikiran Pai-2


On Thursday 08 December 2016 08:28 PM, David M. Lloyd wrote:

> On 12/08/2016 08:41 AM, Jaikiran Pai wrote:
>>
>>>    from a user's
>>> perspective: if I say "I depend on module org.foo.bar:main" I should
>>> also be able to say "...and use their packages" which would exclude all
>>> paths in the source module's resource roots (other than META-INF and the
>>> root path).
>> When you say "would exclude all paths in the source module's resource
>> roots...", you mean precedence will be given to the paths of the
>> dependency module, right? Not literally exclude all paths of the
>> source's resource roots? That way if (one or more) resource roots of the
>> source module has a path "hello/world" which isn't part of the
>> dependency module's paths, then it will get "accepted" and be available
>> to the source module. Did I understand this right or did you intend to
>> literally mean excluding all paths of the source module's resource roots?
> I mean literally excluding.  Dependency-first (aka "parent-first")
> loading is fundamentally flawed in any non-tree graph.  To boil it down
> to the degenerate case, if I have two modules A and B which have a
> mutual interdependency and a common resource or class, using
> parent-first loading, A gets B's resource, while B gets A's resource,
> which is just plain weird.
>
> Using local-first (aka "child-first") loading (as we do) and excluding
> local resource paths that are found in dependencies, A always gets A's
> resource, and B always gets B's resource.  If you want both, then you
> simply don't override, and A gets A's and then B's, and B gets B's and
> then A's.
>
> Does that make sense?

Yes, it does now. Thanks for explaining that part.



> Great!  I'll open up a JIRA for this enhancement.

Thanks, I'll keep a watch and test it out whenever it's available.


-Jaikiran
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Loading...