Progress toward WildFly support for JDK 17 -- Avoiding JPMS problems

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Progress toward WildFly support for JDK 17 -- Avoiding JPMS problems

Brian Stansberry
For the last month we've been focusing quite a bit of energy toward seeing what it will take for WildFly to run well on the upcoming JDK 17. This post is one of two I plan. This one is an attempt to start a discussion around a couple topics; the other will be more of a status update. 

Status summary is things are progressing well, no show-stoppers so far, but plenty more to do. More on that in the other post....

WF 23 runs well on SE 13. We want to get to 17. The key barriers are:

1) SE 14 dropped the java.security.acl package.
2) SE 15 introduced hidden classes (JEP 371)
3) SE 16 strongly encapsulated JDK internals by default (JEP 396)

The discussion points relate to #3. WildFly does quite a lot reflection stuff, plus we have some use cases where end users may want to use internal JDK classes. SE 16 locks this down. For a good primer on the basic things SE allows us when we need things to be made available, see [1]. Richard Opalka did a lot of good analysis of what JPMS-related VM launch settings we need for WF to work properly; see [2] and [3]. It's not a huge set, which is nice.

But, it's not complete, because it doesn't account for user applications. If application code requires additional deep reflection, then additional VM launch settings will be needed. I think that's ok in general; we provide hooks for users to add things to the JAVA_OPTS flags that are passed to java. But the less users need to do that the better. Hence the discussion topics:

1) The ClassReflectionIndex[4] constructor iterates over all fields and methods in a class and marks them as accessible. For any class that is used as an EE component type, *as well its superclasses*, a ClassReflectionIndex is created. This means if an application uses some JDK class as a superclass (ignore Object, which gets some special handling that makes it not a problem in this discussion), then that superclass's package is going to need to be opened. We have no way to know what superclasses our users' component might have, so we can't open them up in for them as part of our standard launch args.

My general understanding is we do this in order to allow things like injection of values into fields or wrapping calls to non-public methods with interceptors.

Is there anything we can do about this? Any intelligence we can apply to avoid doing unnecessary opening? (See [5] for a very specific example of such a thing.) 

Or is this maybe not a big problem? We already need to open the java.util package for other reasons, so EE component based on classes in that package won't have a problem.


2) There are cases where our configuration allows users to specify a class to use as the impl of interface, as an instruction for the server to instantiate an instance and use it. Examples include NamingContext and java.security.Policy impls.  In some cases well known examples of those interfaces are internal JDK classes.

Should we identify likely cases of these things and proactively include those packages in our server launch --add-opens set? My general instinct is no, but there may be cases where my instincts are wrong.




Best regards,
Brian

_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
Reply | Threaded
Open this post in threaded view
|

Re: Progress toward WildFly support for JDK 17 -- Avoiding JPMS problems

Richard Opalka
Helo,

Comments inlined:

On Thu, May 27, 2021 at 9:44 PM Brian Stansberry <[hidden email]> wrote:
For the last month we've been focusing quite a bit of energy toward seeing what it will take for WildFly to run well on the upcoming JDK 17. This post is one of two I plan. This one is an attempt to start a discussion around a couple topics; the other will be more of a status update. 

Status summary is things are progressing well, no show-stoppers so far, but plenty more to do. More on that in the other post....

WF 23 runs well on SE 13. We want to get to 17. The key barriers are:

1) SE 14 dropped the java.security.acl package.
2) SE 15 introduced hidden classes (JEP 371)
3) SE 16 strongly encapsulated JDK internals by default (JEP 396)
SE 17 will eliminate global --illegal-access command line option (JEP 403) so explicit enumeration
of --add-opens and --add-exports will be the only possibility to open JDK internal packages.

The discussion points relate to #3. WildFly does quite a lot reflection stuff, plus we have some use cases where end users may want to use internal JDK classes. SE 16 locks this down. For a good primer on the basic things SE allows us when we need things to be made available, see [1]. Richard Opalka did a lot of good analysis of what JPMS-related VM launch settings we need for WF to work properly; see [2] and [3]. It's not a huge set, which is nice.

But, it's not complete, because it doesn't account for user applications. If application code requires additional deep reflection, then additional VM launch settings will be needed. I think that's ok in general; we provide hooks for users to add things to the JAVA_OPTS flags that are passed to java. But the less users need to do that the better. Hence the discussion topics:
AFAIR there are two places where modular jdk params must be specified.
One place is shell scripts (common.sh(.bat), etc) and second place is org.jboss.as.host.controller.jvm.JvmType .
That is a minor problem. We should configure these params either in shell scripts or in config files but not in Java code.

The problematic part is also how to propagate (configure) modular jdk parameters?
1) Should we have just one global place where modular jdk params will be specified (e.g. common.sh(.bat)) and propagated ( e.g. domain -> host controller(s) -> server(s) )?
2) Or should users be able to specify modular jdk params for each entity like domain, server, host controller?
Maybe that is already supported via standard configuration?

1) The ClassReflectionIndex[4] constructor iterates over all fields and methods in a class and marks them as accessible. For any class that is used as an EE component type, *as well its superclasses*, a ClassReflectionIndex is created. This means if an application uses some JDK class as a superclass (ignore Object, which gets some special handling that makes it not a problem in this discussion), then that superclass's package is going to need to be opened. We have no way to know what superclasses our users' component might have, so we can't open them up in for them as part of our standard launch args.

My general understanding is we do this in order to allow things like injection of values into fields or wrapping calls to non-public methods with interceptors.

Is there anything we can do about this? Any intelligence we can apply to avoid doing unnecessary opening? (See [5] for a very specific example of such a thing.) 

Or is this maybe not a big problem? We already need to open the java.util package for other reasons, so EE component based on classes in that package won't have a problem.
There are many utility classes in various JDK packages, e.g.
 * Readable or Runnable in java.lang. 
 * Closeable or Flushable in java.io.
We do open all these packages already so maybe it will be sufficient for our users too without need to open other packages.


2) There are cases where our configuration allows users to specify a class to use as the impl of interface, as an instruction for the server to instantiate an instance and use it. Examples include NamingContext and java.security.Policy impls.  In some cases well known examples of those interfaces are internal JDK classes.

Should we identify likely cases of these things and proactively include those packages in our server launch --add-opens set? My general instinct is no, but there may be cases where my instincts are wrong.
I would also say no. Few of the answers will be obvious after a successful 100% TCK run.
_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
Reply | Threaded
Open this post in threaded view
|

Re: Progress toward WildFly support for JDK 17 -- Avoiding JPMS problems

Brian Stansberry


On Mon, May 31, 2021 at 10:44 AM Richard Opalka <[hidden email]> wrote:
Helo,

Comments inlined:

On Thu, May 27, 2021 at 9:44 PM Brian Stansberry <[hidden email]> wrote:
For the last month we've been focusing quite a bit of energy toward seeing what it will take for WildFly to run well on the upcoming JDK 17. This post is one of two I plan. This one is an attempt to start a discussion around a couple topics; the other will be more of a status update. 

Status summary is things are progressing well, no show-stoppers so far, but plenty more to do. More on that in the other post....

WF 23 runs well on SE 13. We want to get to 17. The key barriers are:

1) SE 14 dropped the java.security.acl package.
2) SE 15 introduced hidden classes (JEP 371)
3) SE 16 strongly encapsulated JDK internals by default (JEP 396)
SE 17 will eliminate global --illegal-access command line option (JEP 403) so explicit enumeration
of --add-opens and --add-exports will be the only possibility to open JDK internal packages.

The discussion points relate to #3. WildFly does quite a lot reflection stuff, plus we have some use cases where end users may want to use internal JDK classes. SE 16 locks this down. For a good primer on the basic things SE allows us when we need things to be made available, see [1]. Richard Opalka did a lot of good analysis of what JPMS-related VM launch settings we need for WF to work properly; see [2] and [3]. It's not a huge set, which is nice.

But, it's not complete, because it doesn't account for user applications. If application code requires additional deep reflection, then additional VM launch settings will be needed. I think that's ok in general; we provide hooks for users to add things to the JAVA_OPTS flags that are passed to java. But the less users need to do that the better. Hence the discussion topics:
AFAIR there are two places where modular jdk params must be specified.
One place is shell scripts (common.sh(.bat), etc) and second place is org.jboss.as.host.controller.jvm.JvmType .
That is a minor problem. We should configure these params either in shell scripts or in config files but not in Java code.

The problematic part is also how to propagate (configure) modular jdk parameters?
1) Should we have just one global place where modular jdk params will be specified (e.g. common.sh(.bat)) and propagated ( e.g. domain -> host controller(s) -> server(s) )?
2) Or should users be able to specify modular jdk params for each entity like domain, server, host controller?
Maybe that is already supported via standard configuration?

The key requirement is that each HC (DC or otherwise) is independently configurable, and each server is as well. A domain mode server doesn't have to run on the same VM as its HC, so we cannot force HC-level settings (e.g. stuff from common.sh) onto the servers.

The various 'jvm' config settings in domain.xml/host.xml let users customize these things (or any other JVM launch setting) on a per-server basis. JvmType provides a kind of simple ease-of-use thing. Otherwise we'd have to have standard config blocks per JVM type in the standard config files we ship, and then users (and our own testsuite) would have to be taught to use them.

It is clunky that there is no way to turn off applying those defaults though.


1) The ClassReflectionIndex[4] constructor iterates over all fields and methods in a class and marks them as accessible. For any class that is used as an EE component type, *as well its superclasses*, a ClassReflectionIndex is created. This means if an application uses some JDK class as a superclass (ignore Object, which gets some special handling that makes it not a problem in this discussion), then that superclass's package is going to need to be opened. We have no way to know what superclasses our users' component might have, so we can't open them up in for them as part of our standard launch args.

My general understanding is we do this in order to allow things like injection of values into fields or wrapping calls to non-public methods with interceptors.

Is there anything we can do about this? Any intelligence we can apply to avoid doing unnecessary opening? (See [5] for a very specific example of such a thing.) 

Or is this maybe not a big problem? We already need to open the java.util package for other reasons, so EE component based on classes in that package won't have a problem.
There are many utility classes in various JDK packages, e.g.
 * Readable or Runnable in java.lang. 
 * Closeable or Flushable in java.io.
We do open all these packages already so maybe it will be sufficient for our users too without need to open other packages.

Thanks; that's a good point.


2) There are cases where our configuration allows users to specify a class to use as the impl of interface, as an instruction for the server to instantiate an instance and use it. Examples include NamingContext and java.security.Policy impls.  In some cases well known examples of those interfaces are internal JDK classes.

Should we identify likely cases of these things and proactively include those packages in our server launch --add-opens set? My general instinct is no, but there may be cases where my instincts are wrong.
I would also say no. Few of the answers will be obvious after a successful 100% TCK run.
_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His

_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
Reply | Threaded
Open this post in threaded view
|

Re: Progress toward WildFly support for JDK 17 -- Avoiding JPMS problems

Brian Stansberry
Cody Lerum made me aware of a pain point in this area -- bootable jar.

We use our standard launch scripts (e.g. [1]) to make setting the needed JPMS settings easy for users doing traditional launches, but with bootable jar the expectation is 'java -jar' would be used.

Of course even on SE 8 people may have non-trivial args to java, and not just 'java -jar'.  But we'll need to be sure our bootable jar docs cover what's needed for SE 16+.


On Tue, Jun 1, 2021 at 2:38 PM Brian Stansberry <[hidden email]> wrote:


On Mon, May 31, 2021 at 10:44 AM Richard Opalka <[hidden email]> wrote:
Helo,

Comments inlined:

On Thu, May 27, 2021 at 9:44 PM Brian Stansberry <[hidden email]> wrote:
For the last month we've been focusing quite a bit of energy toward seeing what it will take for WildFly to run well on the upcoming JDK 17. This post is one of two I plan. This one is an attempt to start a discussion around a couple topics; the other will be more of a status update. 

Status summary is things are progressing well, no show-stoppers so far, but plenty more to do. More on that in the other post....

WF 23 runs well on SE 13. We want to get to 17. The key barriers are:

1) SE 14 dropped the java.security.acl package.
2) SE 15 introduced hidden classes (JEP 371)
3) SE 16 strongly encapsulated JDK internals by default (JEP 396)
SE 17 will eliminate global --illegal-access command line option (JEP 403) so explicit enumeration
of --add-opens and --add-exports will be the only possibility to open JDK internal packages.

The discussion points relate to #3. WildFly does quite a lot reflection stuff, plus we have some use cases where end users may want to use internal JDK classes. SE 16 locks this down. For a good primer on the basic things SE allows us when we need things to be made available, see [1]. Richard Opalka did a lot of good analysis of what JPMS-related VM launch settings we need for WF to work properly; see [2] and [3]. It's not a huge set, which is nice.

But, it's not complete, because it doesn't account for user applications. If application code requires additional deep reflection, then additional VM launch settings will be needed. I think that's ok in general; we provide hooks for users to add things to the JAVA_OPTS flags that are passed to java. But the less users need to do that the better. Hence the discussion topics:
AFAIR there are two places where modular jdk params must be specified.
One place is shell scripts (common.sh(.bat), etc) and second place is org.jboss.as.host.controller.jvm.JvmType .
That is a minor problem. We should configure these params either in shell scripts or in config files but not in Java code.

The problematic part is also how to propagate (configure) modular jdk parameters?
1) Should we have just one global place where modular jdk params will be specified (e.g. common.sh(.bat)) and propagated ( e.g. domain -> host controller(s) -> server(s) )?
2) Or should users be able to specify modular jdk params for each entity like domain, server, host controller?
Maybe that is already supported via standard configuration?

The key requirement is that each HC (DC or otherwise) is independently configurable, and each server is as well. A domain mode server doesn't have to run on the same VM as its HC, so we cannot force HC-level settings (e.g. stuff from common.sh) onto the servers.

The various 'jvm' config settings in domain.xml/host.xml let users customize these things (or any other JVM launch setting) on a per-server basis. JvmType provides a kind of simple ease-of-use thing. Otherwise we'd have to have standard config blocks per JVM type in the standard config files we ship, and then users (and our own testsuite) would have to be taught to use them.

It is clunky that there is no way to turn off applying those defaults though.


1) The ClassReflectionIndex[4] constructor iterates over all fields and methods in a class and marks them as accessible. For any class that is used as an EE component type, *as well its superclasses*, a ClassReflectionIndex is created. This means if an application uses some JDK class as a superclass (ignore Object, which gets some special handling that makes it not a problem in this discussion), then that superclass's package is going to need to be opened. We have no way to know what superclasses our users' component might have, so we can't open them up in for them as part of our standard launch args.

My general understanding is we do this in order to allow things like injection of values into fields or wrapping calls to non-public methods with interceptors.

Is there anything we can do about this? Any intelligence we can apply to avoid doing unnecessary opening? (See [5] for a very specific example of such a thing.) 

Or is this maybe not a big problem? We already need to open the java.util package for other reasons, so EE component based on classes in that package won't have a problem.
There are many utility classes in various JDK packages, e.g.
 * Readable or Runnable in java.lang. 
 * Closeable or Flushable in java.io.
We do open all these packages already so maybe it will be sufficient for our users too without need to open other packages.

Thanks; that's a good point.


2) There are cases where our configuration allows users to specify a class to use as the impl of interface, as an instruction for the server to instantiate an instance and use it. Examples include NamingContext and java.security.Policy impls.  In some cases well known examples of those interfaces are internal JDK classes.

Should we identify likely cases of these things and proactively include those packages in our server launch --add-opens set? My general instinct is no, but there may be cases where my instincts are wrong.
I would also say no. Few of the answers will be obvious after a successful 100% TCK run.
_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His

_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
Reply | Threaded
Open this post in threaded view
|

Re: Progress toward WildFly support for JDK 17 -- Avoiding JPMS problems

Jean Francois Denise
On 6/4/21 9:30 PM, Brian Stansberry wrote:
Cody Lerum made me aware of a pain point in this area -- bootable jar.

We use our standard launch scripts (e.g. [1]) to make setting the needed JPMS settings easy for users doing traditional launches, but with bootable jar the expectation is 'java -jar' would be used.

Of course even on SE 8 people may have non-trivial args to java, and not just 'java -jar'.  But we'll need to be sure our bootable jar docs cover what's needed for SE 16+.

I have logged https://issues.redhat.com/browse/WFCORE-5451 to evolve the WildFly Bootable JAR doc.

On Tue, Jun 1, 2021 at 2:38 PM Brian Stansberry <[hidden email]> wrote:


On Mon, May 31, 2021 at 10:44 AM Richard Opalka <[hidden email]> wrote:
Helo,

Comments inlined:

On Thu, May 27, 2021 at 9:44 PM Brian Stansberry <[hidden email]> wrote:
For the last month we've been focusing quite a bit of energy toward seeing what it will take for WildFly to run well on the upcoming JDK 17. This post is one of two I plan. This one is an attempt to start a discussion around a couple topics; the other will be more of a status update. 

Status summary is things are progressing well, no show-stoppers so far, but plenty more to do. More on that in the other post....

WF 23 runs well on SE 13. We want to get to 17. The key barriers are:

1) SE 14 dropped the java.security.acl package.
2) SE 15 introduced hidden classes (JEP 371)
3) SE 16 strongly encapsulated JDK internals by default (JEP 396)
SE 17 will eliminate global --illegal-access command line option (JEP 403) so explicit enumeration
of --add-opens and --add-exports will be the only possibility to open JDK internal packages.

The discussion points relate to #3. WildFly does quite a lot reflection stuff, plus we have some use cases where end users may want to use internal JDK classes. SE 16 locks this down. For a good primer on the basic things SE allows us when we need things to be made available, see [1]. Richard Opalka did a lot of good analysis of what JPMS-related VM launch settings we need for WF to work properly; see [2] and [3]. It's not a huge set, which is nice.

But, it's not complete, because it doesn't account for user applications. If application code requires additional deep reflection, then additional VM launch settings will be needed. I think that's ok in general; we provide hooks for users to add things to the JAVA_OPTS flags that are passed to java. But the less users need to do that the better. Hence the discussion topics:
AFAIR there are two places where modular jdk params must be specified.
One place is shell scripts (common.sh(.bat), etc) and second place is org.jboss.as.host.controller.jvm.JvmType .
That is a minor problem. We should configure these params either in shell scripts or in config files but not in Java code.

The problematic part is also how to propagate (configure) modular jdk parameters?
1) Should we have just one global place where modular jdk params will be specified (e.g. common.sh(.bat)) and propagated ( e.g. domain -> host controller(s) -> server(s) )?
2) Or should users be able to specify modular jdk params for each entity like domain, server, host controller?
Maybe that is already supported via standard configuration?

The key requirement is that each HC (DC or otherwise) is independently configurable, and each server is as well. A domain mode server doesn't have to run on the same VM as its HC, so we cannot force HC-level settings (e.g. stuff from common.sh) onto the servers.

The various 'jvm' config settings in domain.xml/host.xml let users customize these things (or any other JVM launch setting) on a per-server basis. JvmType provides a kind of simple ease-of-use thing. Otherwise we'd have to have standard config blocks per JVM type in the standard config files we ship, and then users (and our own testsuite) would have to be taught to use them.

It is clunky that there is no way to turn off applying those defaults though.


1) The ClassReflectionIndex[4] constructor iterates over all fields and methods in a class and marks them as accessible. For any class that is used as an EE component type, *as well its superclasses*, a ClassReflectionIndex is created. This means if an application uses some JDK class as a superclass (ignore Object, which gets some special handling that makes it not a problem in this discussion), then that superclass's package is going to need to be opened. We have no way to know what superclasses our users' component might have, so we can't open them up in for them as part of our standard launch args.

My general understanding is we do this in order to allow things like injection of values into fields or wrapping calls to non-public methods with interceptors.

Is there anything we can do about this? Any intelligence we can apply to avoid doing unnecessary opening? (See [5] for a very specific example of such a thing.) 

Or is this maybe not a big problem? We already need to open the java.util package for other reasons, so EE component based on classes in that package won't have a problem.
There are many utility classes in various JDK packages, e.g.
 * Readable or Runnable in java.lang. 
 * Closeable or Flushable in java.io.
We do open all these packages already so maybe it will be sufficient for our users too without need to open other packages.

Thanks; that's a good point.


2) There are cases where our configuration allows users to specify a class to use as the impl of interface, as an instruction for the server to instantiate an instance and use it. Examples include NamingContext and java.security.Policy impls.  In some cases well known examples of those interfaces are internal JDK classes.

Should we identify likely cases of these things and proactively include those packages in our server launch --add-opens set? My general instinct is no, but there may be cases where my instincts are wrong.
I would also say no. Few of the answers will be obvious after a successful 100% TCK run.
_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His

_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s



_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
Reply | Threaded
Open this post in threaded view
|

Re: Progress toward WildFly support for JDK 17 -- Avoiding JPMS problems

Brian Stansberry


On Mon, Jun 7, 2021 at 5:18 AM Jean Francois Denise <[hidden email]> wrote:
On 6/4/21 9:30 PM, Brian Stansberry wrote:
Cody Lerum made me aware of a pain point in this area -- bootable jar.

We use our standard launch scripts (e.g. [1]) to make setting the needed JPMS settings easy for users doing traditional launches, but with bootable jar the expectation is 'java -jar' would be used.

Of course even on SE 8 people may have non-trivial args to java, and not just 'java -jar'.  But we'll need to be sure our bootable jar docs cover what's needed for SE 16+.

I have logged https://issues.redhat.com/browse/WFCORE-5451 to evolve the WildFly Bootable JAR doc.

Thanks!

Here and the JIRA is a good place to figure out what we want to say about these things. I somewhat do that in code comment in common.sh (see link above).

I think it's useful to document something about why various things are needed, at least for ones where a slimmed server might not need something. But any language like that would need to have a clear N.B. that it's possible and --add-exports or --add-opens may be needed for something other than what's listed. I doubt we'd start testing with anything other than the full set, so such documentation could fall out of date.

On Tue, Jun 1, 2021 at 2:38 PM Brian Stansberry <[hidden email]> wrote:


On Mon, May 31, 2021 at 10:44 AM Richard Opalka <[hidden email]> wrote:
Helo,

Comments inlined:

On Thu, May 27, 2021 at 9:44 PM Brian Stansberry <[hidden email]> wrote:
For the last month we've been focusing quite a bit of energy toward seeing what it will take for WildFly to run well on the upcoming JDK 17. This post is one of two I plan. This one is an attempt to start a discussion around a couple topics; the other will be more of a status update. 

Status summary is things are progressing well, no show-stoppers so far, but plenty more to do. More on that in the other post....

WF 23 runs well on SE 13. We want to get to 17. The key barriers are:

1) SE 14 dropped the java.security.acl package.
2) SE 15 introduced hidden classes (JEP 371)
3) SE 16 strongly encapsulated JDK internals by default (JEP 396)
SE 17 will eliminate global --illegal-access command line option (JEP 403) so explicit enumeration
of --add-opens and --add-exports will be the only possibility to open JDK internal packages.

The discussion points relate to #3. WildFly does quite a lot reflection stuff, plus we have some use cases where end users may want to use internal JDK classes. SE 16 locks this down. For a good primer on the basic things SE allows us when we need things to be made available, see [1]. Richard Opalka did a lot of good analysis of what JPMS-related VM launch settings we need for WF to work properly; see [2] and [3]. It's not a huge set, which is nice.

But, it's not complete, because it doesn't account for user applications. If application code requires additional deep reflection, then additional VM launch settings will be needed. I think that's ok in general; we provide hooks for users to add things to the JAVA_OPTS flags that are passed to java. But the less users need to do that the better. Hence the discussion topics:
AFAIR there are two places where modular jdk params must be specified.
One place is shell scripts (common.sh(.bat), etc) and second place is org.jboss.as.host.controller.jvm.JvmType .
That is a minor problem. We should configure these params either in shell scripts or in config files but not in Java code.

The problematic part is also how to propagate (configure) modular jdk parameters?
1) Should we have just one global place where modular jdk params will be specified (e.g. common.sh(.bat)) and propagated ( e.g. domain -> host controller(s) -> server(s) )?
2) Or should users be able to specify modular jdk params for each entity like domain, server, host controller?
Maybe that is already supported via standard configuration?

The key requirement is that each HC (DC or otherwise) is independently configurable, and each server is as well. A domain mode server doesn't have to run on the same VM as its HC, so we cannot force HC-level settings (e.g. stuff from common.sh) onto the servers.

The various 'jvm' config settings in domain.xml/host.xml let users customize these things (or any other JVM launch setting) on a per-server basis. JvmType provides a kind of simple ease-of-use thing. Otherwise we'd have to have standard config blocks per JVM type in the standard config files we ship, and then users (and our own testsuite) would have to be taught to use them.

It is clunky that there is no way to turn off applying those defaults though.


1) The ClassReflectionIndex[4] constructor iterates over all fields and methods in a class and marks them as accessible. For any class that is used as an EE component type, *as well its superclasses*, a ClassReflectionIndex is created. This means if an application uses some JDK class as a superclass (ignore Object, which gets some special handling that makes it not a problem in this discussion), then that superclass's package is going to need to be opened. We have no way to know what superclasses our users' component might have, so we can't open them up in for them as part of our standard launch args.

My general understanding is we do this in order to allow things like injection of values into fields or wrapping calls to non-public methods with interceptors.

Is there anything we can do about this? Any intelligence we can apply to avoid doing unnecessary opening? (See [5] for a very specific example of such a thing.) 

Or is this maybe not a big problem? We already need to open the java.util package for other reasons, so EE component based on classes in that package won't have a problem.
There are many utility classes in various JDK packages, e.g.
 * Readable or Runnable in java.lang. 
 * Closeable or Flushable in java.io.
We do open all these packages already so maybe it will be sufficient for our users too without need to open other packages.

Thanks; that's a good point.


2) There are cases where our configuration allows users to specify a class to use as the impl of interface, as an instruction for the server to instantiate an instance and use it. Examples include NamingContext and java.security.Policy impls.  In some cases well known examples of those interfaces are internal JDK classes.

Should we identify likely cases of these things and proactively include those packages in our server launch --add-opens set? My general instinct is no, but there may be cases where my instincts are wrong.
I would also say no. Few of the answers will be obvious after a successful 100% TCK run.
_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His

_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His

_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
Reply | Threaded
Open this post in threaded view
|

Re: Progress toward WildFly support for JDK 17 -- Avoiding JPMS problems

Richard Opalka
Hello Brian & Jean,

   Just FYI it is possible to assign --add-opens && --add-exports via Java's manifest. For example:

# Exporting not exported package from command line - reflection not allowed
--add-exports java.base/sun.nio.ch=ALL-UNNAMED
# Exporting not exported package from command line - reflection is allowed
--add-opens java.base/java.lang=ALL-UNNAMED

# The same like above can be achieved from MANIFEST.MF
# if JVM is executed like java -jar foo.jar, where foo.jar
# contains MANIFEST.MF with entries:
Add-Exports: java.base/sun.nio.ch=ALL-UNNAMED
Add-Opens: java.base/java.lang=ALL-UNNAMED
# These entries must be space separated. If specified like
# this there's no need to specify them on the commandline.

I hope this helps.

Rio

On Mon, Jun 7, 2021 at 4:35 PM Brian Stansberry <[hidden email]> wrote:


On Mon, Jun 7, 2021 at 5:18 AM Jean Francois Denise <[hidden email]> wrote:
On 6/4/21 9:30 PM, Brian Stansberry wrote:
Cody Lerum made me aware of a pain point in this area -- bootable jar.

We use our standard launch scripts (e.g. [1]) to make setting the needed JPMS settings easy for users doing traditional launches, but with bootable jar the expectation is 'java -jar' would be used.

Of course even on SE 8 people may have non-trivial args to java, and not just 'java -jar'.  But we'll need to be sure our bootable jar docs cover what's needed for SE 16+.

I have logged https://issues.redhat.com/browse/WFCORE-5451 to evolve the WildFly Bootable JAR doc.

Thanks!

Here and the JIRA is a good place to figure out what we want to say about these things. I somewhat do that in code comment in common.sh (see link above).

I think it's useful to document something about why various things are needed, at least for ones where a slimmed server might not need something. But any language like that would need to have a clear N.B. that it's possible and --add-exports or --add-opens may be needed for something other than what's listed. I doubt we'd start testing with anything other than the full set, so such documentation could fall out of date.

On Tue, Jun 1, 2021 at 2:38 PM Brian Stansberry <[hidden email]> wrote:


On Mon, May 31, 2021 at 10:44 AM Richard Opalka <[hidden email]> wrote:
Helo,

Comments inlined:

On Thu, May 27, 2021 at 9:44 PM Brian Stansberry <[hidden email]> wrote:
For the last month we've been focusing quite a bit of energy toward seeing what it will take for WildFly to run well on the upcoming JDK 17. This post is one of two I plan. This one is an attempt to start a discussion around a couple topics; the other will be more of a status update. 

Status summary is things are progressing well, no show-stoppers so far, but plenty more to do. More on that in the other post....

WF 23 runs well on SE 13. We want to get to 17. The key barriers are:

1) SE 14 dropped the java.security.acl package.
2) SE 15 introduced hidden classes (JEP 371)
3) SE 16 strongly encapsulated JDK internals by default (JEP 396)
SE 17 will eliminate global --illegal-access command line option (JEP 403) so explicit enumeration
of --add-opens and --add-exports will be the only possibility to open JDK internal packages.

The discussion points relate to #3. WildFly does quite a lot reflection stuff, plus we have some use cases where end users may want to use internal JDK classes. SE 16 locks this down. For a good primer on the basic things SE allows us when we need things to be made available, see [1]. Richard Opalka did a lot of good analysis of what JPMS-related VM launch settings we need for WF to work properly; see [2] and [3]. It's not a huge set, which is nice.

But, it's not complete, because it doesn't account for user applications. If application code requires additional deep reflection, then additional VM launch settings will be needed. I think that's ok in general; we provide hooks for users to add things to the JAVA_OPTS flags that are passed to java. But the less users need to do that the better. Hence the discussion topics:
AFAIR there are two places where modular jdk params must be specified.
One place is shell scripts (common.sh(.bat), etc) and second place is org.jboss.as.host.controller.jvm.JvmType .
That is a minor problem. We should configure these params either in shell scripts or in config files but not in Java code.

The problematic part is also how to propagate (configure) modular jdk parameters?
1) Should we have just one global place where modular jdk params will be specified (e.g. common.sh(.bat)) and propagated ( e.g. domain -> host controller(s) -> server(s) )?
2) Or should users be able to specify modular jdk params for each entity like domain, server, host controller?
Maybe that is already supported via standard configuration?

The key requirement is that each HC (DC or otherwise) is independently configurable, and each server is as well. A domain mode server doesn't have to run on the same VM as its HC, so we cannot force HC-level settings (e.g. stuff from common.sh) onto the servers.

The various 'jvm' config settings in domain.xml/host.xml let users customize these things (or any other JVM launch setting) on a per-server basis. JvmType provides a kind of simple ease-of-use thing. Otherwise we'd have to have standard config blocks per JVM type in the standard config files we ship, and then users (and our own testsuite) would have to be taught to use them.

It is clunky that there is no way to turn off applying those defaults though.


1) The ClassReflectionIndex[4] constructor iterates over all fields and methods in a class and marks them as accessible. For any class that is used as an EE component type, *as well its superclasses*, a ClassReflectionIndex is created. This means if an application uses some JDK class as a superclass (ignore Object, which gets some special handling that makes it not a problem in this discussion), then that superclass's package is going to need to be opened. We have no way to know what superclasses our users' component might have, so we can't open them up in for them as part of our standard launch args.

My general understanding is we do this in order to allow things like injection of values into fields or wrapping calls to non-public methods with interceptors.

Is there anything we can do about this? Any intelligence we can apply to avoid doing unnecessary opening? (See [5] for a very specific example of such a thing.) 

Or is this maybe not a big problem? We already need to open the java.util package for other reasons, so EE component based on classes in that package won't have a problem.
There are many utility classes in various JDK packages, e.g.
 * Readable or Runnable in java.lang. 
 * Closeable or Flushable in java.io.
We do open all these packages already so maybe it will be sufficient for our users too without need to open other packages.

Thanks; that's a good point.


2) There are cases where our configuration allows users to specify a class to use as the impl of interface, as an instruction for the server to instantiate an instance and use it. Examples include NamingContext and java.security.Policy impls.  In some cases well known examples of those interfaces are internal JDK classes.

Should we identify likely cases of these things and proactively include those packages in our server launch --add-opens set? My general instinct is no, but there may be cases where my instincts are wrong.
I would also say no. Few of the answers will be obvious after a successful 100% TCK run.
_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His

_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His
_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
Reply | Threaded
Open this post in threaded view
|

Re: Progress toward WildFly support for JDK 17 -- Avoiding JPMS problems

Jean Francois Denise
On 6/8/21 9:48 AM, Richard Opalka wrote:
Hello Brian & Jean,

   Just FYI it is possible to assign --add-opens && --add-exports via Java's manifest. For example:

# Exporting not exported package from command line - reflection not allowed
--add-exports java.base/sun.nio.ch=ALL-UNNAMED
# Exporting not exported package from command line - reflection is allowed
--add-opens java.base/java.lang=ALL-UNNAMED

# The same like above can be achieved from MANIFEST.MF
# if JVM is executed like java -jar foo.jar, where foo.jar
# contains MANIFEST.MF with entries:
Add-Exports: java.base/sun.nio.ch=ALL-UNNAMED
Add-Opens: java.base/java.lang=ALL-UNNAMED
# These entries must be space separated. If specified like
# this there's no need to specify them on the commandline.

I hope this helps.

Richard, Thank-you. That seems promising!

I did a quick experiment that I hope is missing something...

Add-Exports: java.naming/com.sun.jndi.ldap=ALL-UNNAMED

A simple Main class does:

import com.sun.jndi.ldap.DefaultResponseControlFactory;

DefaultResponseControlFactory factory = new DefaultResponseControlFactory();

This has no effect, I still need to add the --add-exports option.

The migration doc spec[1] is clear that it can be used.

[1]https://docs.oracle.com/en/java/javase/16/migrate/jdk-migration-guide.pdf


Rio

On Mon, Jun 7, 2021 at 4:35 PM Brian Stansberry <[hidden email]> wrote:


On Mon, Jun 7, 2021 at 5:18 AM Jean Francois Denise <[hidden email]> wrote:
On 6/4/21 9:30 PM, Brian Stansberry wrote:
Cody Lerum made me aware of a pain point in this area -- bootable jar.

We use our standard launch scripts (e.g. [1]) to make setting the needed JPMS settings easy for users doing traditional launches, but with bootable jar the expectation is 'java -jar' would be used.

Of course even on SE 8 people may have non-trivial args to java, and not just 'java -jar'.  But we'll need to be sure our bootable jar docs cover what's needed for SE 16+.

I have logged https://issues.redhat.com/browse/WFCORE-5451 to evolve the WildFly Bootable JAR doc.

Thanks!

Here and the JIRA is a good place to figure out what we want to say about these things. I somewhat do that in code comment in common.sh (see link above).

I think it's useful to document something about why various things are needed, at least for ones where a slimmed server might not need something. But any language like that would need to have a clear N.B. that it's possible and --add-exports or --add-opens may be needed for something other than what's listed. I doubt we'd start testing with anything other than the full set, so such documentation could fall out of date.

On Tue, Jun 1, 2021 at 2:38 PM Brian Stansberry <[hidden email]> wrote:


On Mon, May 31, 2021 at 10:44 AM Richard Opalka <[hidden email]> wrote:
Helo,

Comments inlined:

On Thu, May 27, 2021 at 9:44 PM Brian Stansberry <[hidden email]> wrote:
For the last month we've been focusing quite a bit of energy toward seeing what it will take for WildFly to run well on the upcoming JDK 17. This post is one of two I plan. This one is an attempt to start a discussion around a couple topics; the other will be more of a status update. 

Status summary is things are progressing well, no show-stoppers so far, but plenty more to do. More on that in the other post....

WF 23 runs well on SE 13. We want to get to 17. The key barriers are:

1) SE 14 dropped the java.security.acl package.
2) SE 15 introduced hidden classes (JEP 371)
3) SE 16 strongly encapsulated JDK internals by default (JEP 396)
SE 17 will eliminate global --illegal-access command line option (JEP 403) so explicit enumeration
of --add-opens and --add-exports will be the only possibility to open JDK internal packages.

The discussion points relate to #3. WildFly does quite a lot reflection stuff, plus we have some use cases where end users may want to use internal JDK classes. SE 16 locks this down. For a good primer on the basic things SE allows us when we need things to be made available, see [1]. Richard Opalka did a lot of good analysis of what JPMS-related VM launch settings we need for WF to work properly; see [2] and [3]. It's not a huge set, which is nice.

But, it's not complete, because it doesn't account for user applications. If application code requires additional deep reflection, then additional VM launch settings will be needed. I think that's ok in general; we provide hooks for users to add things to the JAVA_OPTS flags that are passed to java. But the less users need to do that the better. Hence the discussion topics:
AFAIR there are two places where modular jdk params must be specified.
One place is shell scripts (common.sh(.bat), etc) and second place is org.jboss.as.host.controller.jvm.JvmType .
That is a minor problem. We should configure these params either in shell scripts or in config files but not in Java code.

The problematic part is also how to propagate (configure) modular jdk parameters?
1) Should we have just one global place where modular jdk params will be specified (e.g. common.sh(.bat)) and propagated ( e.g. domain -> host controller(s) -> server(s) )?
2) Or should users be able to specify modular jdk params for each entity like domain, server, host controller?
Maybe that is already supported via standard configuration?

The key requirement is that each HC (DC or otherwise) is independently configurable, and each server is as well. A domain mode server doesn't have to run on the same VM as its HC, so we cannot force HC-level settings (e.g. stuff from common.sh) onto the servers.

The various 'jvm' config settings in domain.xml/host.xml let users customize these things (or any other JVM launch setting) on a per-server basis. JvmType provides a kind of simple ease-of-use thing. Otherwise we'd have to have standard config blocks per JVM type in the standard config files we ship, and then users (and our own testsuite) would have to be taught to use them.

It is clunky that there is no way to turn off applying those defaults though.


1) The ClassReflectionIndex[4] constructor iterates over all fields and methods in a class and marks them as accessible. For any class that is used as an EE component type, *as well its superclasses*, a ClassReflectionIndex is created. This means if an application uses some JDK class as a superclass (ignore Object, which gets some special handling that makes it not a problem in this discussion), then that superclass's package is going to need to be opened. We have no way to know what superclasses our users' component might have, so we can't open them up in for them as part of our standard launch args.

My general understanding is we do this in order to allow things like injection of values into fields or wrapping calls to non-public methods with interceptors.

Is there anything we can do about this? Any intelligence we can apply to avoid doing unnecessary opening? (See [5] for a very specific example of such a thing.) 

Or is this maybe not a big problem? We already need to open the java.util package for other reasons, so EE component based on classes in that package won't have a problem.
There are many utility classes in various JDK packages, e.g.
 * Readable or Runnable in java.lang. 
 * Closeable or Flushable in java.io.
We do open all these packages already so maybe it will be sufficient for our users too without need to open other packages.

Thanks; that's a good point.


2) There are cases where our configuration allows users to specify a class to use as the impl of interface, as an instruction for the server to instantiate an instance and use it. Examples include NamingContext and java.security.Policy impls.  In some cases well known examples of those interfaces are internal JDK classes.

Should we identify likely cases of these things and proactively include those packages in our server launch --add-opens set? My general instinct is no, but there may be cases where my instincts are wrong.
I would also say no. Few of the answers will be obvious after a successful 100% TCK run.
_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His

_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His
_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s



_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
Reply | Threaded
Open this post in threaded view
|

Re: Progress toward WildFly support for JDK 17 -- Avoiding JPMS problems

Jean Francois Denise
In reply to this post by Richard Opalka
On 6/8/21 9:48 AM, Richard Opalka wrote:
Hello Brian & Jean,

   Just FYI it is possible to assign --add-opens && --add-exports via Java's manifest. For example:

# Exporting not exported package from command line - reflection not allowed
--add-exports java.base/sun.nio.ch=ALL-UNNAMED
# Exporting not exported package from command line - reflection is allowed
--add-opens java.base/java.lang=ALL-UNNAMED

# The same like above can be achieved from MANIFEST.MF
# if JVM is executed like java -jar foo.jar, where foo.jar
# contains MANIFEST.MF with entries:
Add-Exports: java.base/sun.nio.ch=ALL-UNNAMED
Add-Opens: java.base/java.lang=ALL-UNNAMED
# These entries must be space separated. If specified like
# this there's no need to specify them on the commandline.

I hope this helps.
I found the root cause the syntax must be Add-Exports: java.naming/com.sun.jndi.ldap, no =ALL-UNNAMED. Seems to work well!

Rio

On Mon, Jun 7, 2021 at 4:35 PM Brian Stansberry <[hidden email]> wrote:


On Mon, Jun 7, 2021 at 5:18 AM Jean Francois Denise <[hidden email]> wrote:
On 6/4/21 9:30 PM, Brian Stansberry wrote:
Cody Lerum made me aware of a pain point in this area -- bootable jar.

We use our standard launch scripts (e.g. [1]) to make setting the needed JPMS settings easy for users doing traditional launches, but with bootable jar the expectation is 'java -jar' would be used.

Of course even on SE 8 people may have non-trivial args to java, and not just 'java -jar'.  But we'll need to be sure our bootable jar docs cover what's needed for SE 16+.

I have logged https://issues.redhat.com/browse/WFCORE-5451 to evolve the WildFly Bootable JAR doc.

Thanks!

Here and the JIRA is a good place to figure out what we want to say about these things. I somewhat do that in code comment in common.sh (see link above).

I think it's useful to document something about why various things are needed, at least for ones where a slimmed server might not need something. But any language like that would need to have a clear N.B. that it's possible and --add-exports or --add-opens may be needed for something other than what's listed. I doubt we'd start testing with anything other than the full set, so such documentation could fall out of date.

On Tue, Jun 1, 2021 at 2:38 PM Brian Stansberry <[hidden email]> wrote:


On Mon, May 31, 2021 at 10:44 AM Richard Opalka <[hidden email]> wrote:
Helo,

Comments inlined:

On Thu, May 27, 2021 at 9:44 PM Brian Stansberry <[hidden email]> wrote:
For the last month we've been focusing quite a bit of energy toward seeing what it will take for WildFly to run well on the upcoming JDK 17. This post is one of two I plan. This one is an attempt to start a discussion around a couple topics; the other will be more of a status update. 

Status summary is things are progressing well, no show-stoppers so far, but plenty more to do. More on that in the other post....

WF 23 runs well on SE 13. We want to get to 17. The key barriers are:

1) SE 14 dropped the java.security.acl package.
2) SE 15 introduced hidden classes (JEP 371)
3) SE 16 strongly encapsulated JDK internals by default (JEP 396)
SE 17 will eliminate global --illegal-access command line option (JEP 403) so explicit enumeration
of --add-opens and --add-exports will be the only possibility to open JDK internal packages.

The discussion points relate to #3. WildFly does quite a lot reflection stuff, plus we have some use cases where end users may want to use internal JDK classes. SE 16 locks this down. For a good primer on the basic things SE allows us when we need things to be made available, see [1]. Richard Opalka did a lot of good analysis of what JPMS-related VM launch settings we need for WF to work properly; see [2] and [3]. It's not a huge set, which is nice.

But, it's not complete, because it doesn't account for user applications. If application code requires additional deep reflection, then additional VM launch settings will be needed. I think that's ok in general; we provide hooks for users to add things to the JAVA_OPTS flags that are passed to java. But the less users need to do that the better. Hence the discussion topics:
AFAIR there are two places where modular jdk params must be specified.
One place is shell scripts (common.sh(.bat), etc) and second place is org.jboss.as.host.controller.jvm.JvmType .
That is a minor problem. We should configure these params either in shell scripts or in config files but not in Java code.

The problematic part is also how to propagate (configure) modular jdk parameters?
1) Should we have just one global place where modular jdk params will be specified (e.g. common.sh(.bat)) and propagated ( e.g. domain -> host controller(s) -> server(s) )?
2) Or should users be able to specify modular jdk params for each entity like domain, server, host controller?
Maybe that is already supported via standard configuration?

The key requirement is that each HC (DC or otherwise) is independently configurable, and each server is as well. A domain mode server doesn't have to run on the same VM as its HC, so we cannot force HC-level settings (e.g. stuff from common.sh) onto the servers.

The various 'jvm' config settings in domain.xml/host.xml let users customize these things (or any other JVM launch setting) on a per-server basis. JvmType provides a kind of simple ease-of-use thing. Otherwise we'd have to have standard config blocks per JVM type in the standard config files we ship, and then users (and our own testsuite) would have to be taught to use them.

It is clunky that there is no way to turn off applying those defaults though.


1) The ClassReflectionIndex[4] constructor iterates over all fields and methods in a class and marks them as accessible. For any class that is used as an EE component type, *as well its superclasses*, a ClassReflectionIndex is created. This means if an application uses some JDK class as a superclass (ignore Object, which gets some special handling that makes it not a problem in this discussion), then that superclass's package is going to need to be opened. We have no way to know what superclasses our users' component might have, so we can't open them up in for them as part of our standard launch args.

My general understanding is we do this in order to allow things like injection of values into fields or wrapping calls to non-public methods with interceptors.

Is there anything we can do about this? Any intelligence we can apply to avoid doing unnecessary opening? (See [5] for a very specific example of such a thing.) 

Or is this maybe not a big problem? We already need to open the java.util package for other reasons, so EE component based on classes in that package won't have a problem.
There are many utility classes in various JDK packages, e.g.
 * Readable or Runnable in java.lang. 
 * Closeable or Flushable in java.io.
We do open all these packages already so maybe it will be sufficient for our users too without need to open other packages.

Thanks; that's a good point.


2) There are cases where our configuration allows users to specify a class to use as the impl of interface, as an instruction for the server to instantiate an instance and use it. Examples include NamingContext and java.security.Policy impls.  In some cases well known examples of those interfaces are internal JDK classes.

Should we identify likely cases of these things and proactively include those packages in our server launch --add-opens set? My general instinct is no, but there may be cases where my instincts are wrong.
I would also say no. Few of the answers will be obvious after a successful 100% TCK run.
_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His

_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His
_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s



_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
Reply | Threaded
Open this post in threaded view
|

Re: Progress toward WildFly support for JDK 17 -- Avoiding JPMS problems

Richard Opalka

On Tue, Jun 8, 2021 at 11:48 AM Jean Francois Denise <[hidden email]> wrote:
On 6/8/21 9:48 AM, Richard Opalka wrote:
Hello Brian & Jean,

   Just FYI it is possible to assign --add-opens && --add-exports via Java's manifest. For example:

# Exporting not exported package from command line - reflection not allowed
--add-exports java.base/sun.nio.ch=ALL-UNNAMED
# Exporting not exported package from command line - reflection is allowed
--add-opens java.base/java.lang=ALL-UNNAMED

# The same like above can be achieved from MANIFEST.MF
# if JVM is executed like java -jar foo.jar, where foo.jar
# contains MANIFEST.MF with entries:
Add-Exports: java.base/sun.nio.ch=ALL-UNNAMED
Add-Opens: java.base/java.lang=ALL-UNNAMED
# These entries must be space separated. If specified like
# this there's no need to specify them on the commandline.

I hope this helps.
I found the root cause the syntax must be Add-Exports: java.naming/com.sun.jndi.ldap, no =ALL-UNNAMED. Seems to work well!

Rio

On Mon, Jun 7, 2021 at 4:35 PM Brian Stansberry <[hidden email]> wrote:


On Mon, Jun 7, 2021 at 5:18 AM Jean Francois Denise <[hidden email]> wrote:
On 6/4/21 9:30 PM, Brian Stansberry wrote:
Cody Lerum made me aware of a pain point in this area -- bootable jar.

We use our standard launch scripts (e.g. [1]) to make setting the needed JPMS settings easy for users doing traditional launches, but with bootable jar the expectation is 'java -jar' would be used.

Of course even on SE 8 people may have non-trivial args to java, and not just 'java -jar'.  But we'll need to be sure our bootable jar docs cover what's needed for SE 16+.

I have logged https://issues.redhat.com/browse/WFCORE-5451 to evolve the WildFly Bootable JAR doc.

Thanks!

Here and the JIRA is a good place to figure out what we want to say about these things. I somewhat do that in code comment in common.sh (see link above).

I think it's useful to document something about why various things are needed, at least for ones where a slimmed server might not need something. But any language like that would need to have a clear N.B. that it's possible and --add-exports or --add-opens may be needed for something other than what's listed. I doubt we'd start testing with anything other than the full set, so such documentation could fall out of date.

On Tue, Jun 1, 2021 at 2:38 PM Brian Stansberry <[hidden email]> wrote:


On Mon, May 31, 2021 at 10:44 AM Richard Opalka <[hidden email]> wrote:
Helo,

Comments inlined:

On Thu, May 27, 2021 at 9:44 PM Brian Stansberry <[hidden email]> wrote:
For the last month we've been focusing quite a bit of energy toward seeing what it will take for WildFly to run well on the upcoming JDK 17. This post is one of two I plan. This one is an attempt to start a discussion around a couple topics; the other will be more of a status update. 

Status summary is things are progressing well, no show-stoppers so far, but plenty more to do. More on that in the other post....

WF 23 runs well on SE 13. We want to get to 17. The key barriers are:

1) SE 14 dropped the java.security.acl package.
2) SE 15 introduced hidden classes (JEP 371)
3) SE 16 strongly encapsulated JDK internals by default (JEP 396)
SE 17 will eliminate global --illegal-access command line option (JEP 403) so explicit enumeration
of --add-opens and --add-exports will be the only possibility to open JDK internal packages.

The discussion points relate to #3. WildFly does quite a lot reflection stuff, plus we have some use cases where end users may want to use internal JDK classes. SE 16 locks this down. For a good primer on the basic things SE allows us when we need things to be made available, see [1]. Richard Opalka did a lot of good analysis of what JPMS-related VM launch settings we need for WF to work properly; see [2] and [3]. It's not a huge set, which is nice.

But, it's not complete, because it doesn't account for user applications. If application code requires additional deep reflection, then additional VM launch settings will be needed. I think that's ok in general; we provide hooks for users to add things to the JAVA_OPTS flags that are passed to java. But the less users need to do that the better. Hence the discussion topics:
AFAIR there are two places where modular jdk params must be specified.
One place is shell scripts (common.sh(.bat), etc) and second place is org.jboss.as.host.controller.jvm.JvmType .
That is a minor problem. We should configure these params either in shell scripts or in config files but not in Java code.

The problematic part is also how to propagate (configure) modular jdk parameters?
1) Should we have just one global place where modular jdk params will be specified (e.g. common.sh(.bat)) and propagated ( e.g. domain -> host controller(s) -> server(s) )?
2) Or should users be able to specify modular jdk params for each entity like domain, server, host controller?
Maybe that is already supported via standard configuration?

The key requirement is that each HC (DC or otherwise) is independently configurable, and each server is as well. A domain mode server doesn't have to run on the same VM as its HC, so we cannot force HC-level settings (e.g. stuff from common.sh) onto the servers.

The various 'jvm' config settings in domain.xml/host.xml let users customize these things (or any other JVM launch setting) on a per-server basis. JvmType provides a kind of simple ease-of-use thing. Otherwise we'd have to have standard config blocks per JVM type in the standard config files we ship, and then users (and our own testsuite) would have to be taught to use them.

It is clunky that there is no way to turn off applying those defaults though.


1) The ClassReflectionIndex[4] constructor iterates over all fields and methods in a class and marks them as accessible. For any class that is used as an EE component type, *as well its superclasses*, a ClassReflectionIndex is created. This means if an application uses some JDK class as a superclass (ignore Object, which gets some special handling that makes it not a problem in this discussion), then that superclass's package is going to need to be opened. We have no way to know what superclasses our users' component might have, so we can't open them up in for them as part of our standard launch args.

My general understanding is we do this in order to allow things like injection of values into fields or wrapping calls to non-public methods with interceptors.

Is there anything we can do about this? Any intelligence we can apply to avoid doing unnecessary opening? (See [5] for a very specific example of such a thing.) 

Or is this maybe not a big problem? We already need to open the java.util package for other reasons, so EE component based on classes in that package won't have a problem.
There are many utility classes in various JDK packages, e.g.
 * Readable or Runnable in java.lang. 
 * Closeable or Flushable in java.io.
We do open all these packages already so maybe it will be sufficient for our users too without need to open other packages.

Thanks; that's a good point.


2) There are cases where our configuration allows users to specify a class to use as the impl of interface, as an instruction for the server to instantiate an instance and use it. Examples include NamingContext and java.security.Policy impls.  In some cases well known examples of those interfaces are internal JDK classes.

Should we identify likely cases of these things and proactively include those packages in our server launch --add-opens set? My general instinct is no, but there may be cases where my instincts are wrong.
I would also say no. Few of the answers will be obvious after a successful 100% TCK run.
_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His

_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s


--
Brian Stansberry
Principal Architect, Red Hat JBoss EAP
He/Him/His
_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s



_______________________________________________
wildfly-dev mailing list -- [hidden email]
To unsubscribe send an email to [hidden email]
%(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s