a way to obtain local ModelController within javaagent?

classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

a way to obtain local ModelController within javaagent?

John Mazzitelli
OK, here's another one where I need some secret magical sauce - hoping someone knows of a technique I can use.

Suppose I have a javaagent installed in a WildFly server (using the standard -javaagent VM argument).

The javaagent would like to talk to the WildFly Server it is co-located with. Since it is in the same VM, the javaagent wants to avoid it looking like a remote call.

But I know of no way to obtain a local ModelController instance to build a client short of injecting some service or subsystem into WildFly itself (something I would like to avoid).

If the javaagent were instead a subsystem extension, it could do something like this:

   InjectedValue<ModelController> mcValue = new InjectedValue<>();
   ...
   ((ServiceBuilder) bldr).addDependency(Services.JBOSS_SERVER_CONTROLLER, ModelController.class, mcValue);
   ...
   WHAT_I_WANT = mcValue.getValue().createClient(...)

But obviously, that's no good for something running outside of the WildFly container (albeit in the same JVM).

Any hope at all? I was thinking some trickery on the order of what ByteMan does in order to figure out a way to obtain a local ModelController? But that's a last ditch effort :) Hoping there is something that uses a little less witchcraft.
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

Stuart Douglas
You can call org.jboss.as.server.CurrentServiceContainer#getServiceContainer() to do this, however it is trickey from a JavaAgent (as the module will not be available from the agent's class loader).

Unfortunately there is no getting around this, as none of the API classes you need will be available in the module. As I see it you have a few different options:

1) Use reflection to get hold of this class, and then use reflection to make the calls
2) Create a class that does this directly, and then make sure it is loaded from the server module (which has access to the classes you need).

There may be some other options, but that is all I can think of of the top of my head.

If you want more info on how to implement either of these approaches feel free to ask me on hipchat.

Stuart

On Sat, Mar 11, 2017 at 5:43 PM, John Mazzitelli <[hidden email]> wrote:
OK, here's another one where I need some secret magical sauce - hoping someone knows of a technique I can use.

Suppose I have a javaagent installed in a WildFly server (using the standard -javaagent VM argument).

The javaagent would like to talk to the WildFly Server it is co-located with. Since it is in the same VM, the javaagent wants to avoid it looking like a remote call.

But I know of no way to obtain a local ModelController instance to build a client short of injecting some service or subsystem into WildFly itself (something I would like to avoid).

If the javaagent were instead a subsystem extension, it could do something like this:

   InjectedValue<ModelController> mcValue = new InjectedValue<>();
   ...
   ((ServiceBuilder) bldr).addDependency(Services.JBOSS_SERVER_CONTROLLER, ModelController.class, mcValue);
   ...
   WHAT_I_WANT = mcValue.getValue().createClient(...)

But obviously, that's no good for something running outside of the WildFly container (albeit in the same JVM).

Any hope at all? I was thinking some trickery on the order of what ByteMan does in order to figure out a way to obtain a local ModelController? But that's a last ditch effort :) Hoping there is something that uses a little less witchcraft.
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

Brian Stansberry
Once you get access to MSC, do not use Services.JBOSS_SERVER_CONTROLLER to get a ModelController to create the client. That method is deprecated and I plan to remove it in the next release (WildFly Core 4 / WildFly 12).

Use ServiceName.parse("org.wildfly.managment.model-controller-client-factory”) to get a ModelControllerClientFactory and use it to create a client, calling createSuperUserClient.

Note that if you want that client to work properly in a SecurityManager enabled environment your code will need to have org.jboss.as.controller.security.ControllerPermission#PERFORM_IN_VM_CALL PERFORM_IN_VM_CALL. The security guys added in that requirement for WF Core 3. Note that applies even if you use the deprecated ModelController.createClient method.

> On Mar 12, 2017, at 8:32 PM, Stuart Douglas <[hidden email]> wrote:
>
> You can call org.jboss.as.server.CurrentServiceContainer#getServiceContainer() to do this, however it is trickey from a JavaAgent (as the module will not be available from the agent's class loader).
>
> Unfortunately there is no getting around this, as none of the API classes you need will be available in the module. As I see it you have a few different options:
>
> 1) Use reflection to get hold of this class, and then use reflection to make the calls
> 2) Create a class that does this directly, and then make sure it is loaded from the server module (which has access to the classes you need).
>
> There may be some other options, but that is all I can think of of the top of my head.
>
> If you want more info on how to implement either of these approaches feel free to ask me on hipchat.
>
> Stuart
>
> On Sat, Mar 11, 2017 at 5:43 PM, John Mazzitelli <[hidden email]> wrote:
> OK, here's another one where I need some secret magical sauce - hoping someone knows of a technique I can use.
>
> Suppose I have a javaagent installed in a WildFly server (using the standard -javaagent VM argument).
>
> The javaagent would like to talk to the WildFly Server it is co-located with. Since it is in the same VM, the javaagent wants to avoid it looking like a remote call.
>
> But I know of no way to obtain a local ModelController instance to build a client short of injecting some service or subsystem into WildFly itself (something I would like to avoid).
>
> If the javaagent were instead a subsystem extension, it could do something like this:
>
>    InjectedValue<ModelController> mcValue = new InjectedValue<>();
>    ...
>    ((ServiceBuilder) bldr).addDependency(Services.JBOSS_SERVER_CONTROLLER, ModelController.class, mcValue);
>    ...
>    WHAT_I_WANT = mcValue.getValue().createClient(...)
>
> But obviously, that's no good for something running outside of the WildFly container (albeit in the same JVM).
>
> Any hope at all? I was thinking some trickery on the order of what ByteMan does in order to figure out a way to obtain a local ModelController? But that's a last ditch effort :) Hoping there is something that uses a little less witchcraft.
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

--
Brian Stansberry
Manager, Senior Principal Software Engineer
JBoss by Red Hat




_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

jtgreene
Administrator
Hi John,

Before moving with non-traditional access like this I think we need to make sure the overall strategy behind what you are building is something we can support over multiple releases. Of course, a lot of that depends on your compatibility expectations.

> On Mar 13, 2017, at 10:08 AM, Brian Stansberry <[hidden email]> wrote:
>
> Once you get access to MSC, do not use Services.JBOSS_SERVER_CONTROLLER to get a ModelController to create the client. That method is deprecated and I plan to remove it in the next release (WildFly Core 4 / WildFly 12).
>
> Use ServiceName.parse("org.wildfly.managment.model-controller-client-factory”) to get a ModelControllerClientFactory and use it to create a client, calling createSuperUserClient.
>
> Note that if you want that client to work properly in a SecurityManager enabled environment your code will need to have org.jboss.as.controller.security.ControllerPermission#PERFORM_IN_VM_CALL PERFORM_IN_VM_CALL. The security guys added in that requirement for WF Core 3. Note that applies even if you use the deprecated ModelController.createClient method.
>
>> On Mar 12, 2017, at 8:32 PM, Stuart Douglas <[hidden email]> wrote:
>>
>> You can call org.jboss.as.server.CurrentServiceContainer#getServiceContainer() to do this, however it is trickey from a JavaAgent (as the module will not be available from the agent's class loader).
>>
>> Unfortunately there is no getting around this, as none of the API classes you need will be available in the module. As I see it you have a few different options:
>>
>> 1) Use reflection to get hold of this class, and then use reflection to make the calls
>> 2) Create a class that does this directly, and then make sure it is loaded from the server module (which has access to the classes you need).
>>
>> There may be some other options, but that is all I can think of of the top of my head.
>>
>> If you want more info on how to implement either of these approaches feel free to ask me on hipchat.
>>
>> Stuart
>>
>> On Sat, Mar 11, 2017 at 5:43 PM, John Mazzitelli <[hidden email]> wrote:
>> OK, here's another one where I need some secret magical sauce - hoping someone knows of a technique I can use.
>>
>> Suppose I have a javaagent installed in a WildFly server (using the standard -javaagent VM argument).
>>
>> The javaagent would like to talk to the WildFly Server it is co-located with. Since it is in the same VM, the javaagent wants to avoid it looking like a remote call.
>>
>> But I know of no way to obtain a local ModelController instance to build a client short of injecting some service or subsystem into WildFly itself (something I would like to avoid).
>>
>> If the javaagent were instead a subsystem extension, it could do something like this:
>>
>>   InjectedValue<ModelController> mcValue = new InjectedValue<>();
>>   ...
>>   ((ServiceBuilder) bldr).addDependency(Services.JBOSS_SERVER_CONTROLLER, ModelController.class, mcValue);
>>   ...
>>   WHAT_I_WANT = mcValue.getValue().createClient(...)
>>
>> But obviously, that's no good for something running outside of the WildFly container (albeit in the same JVM).
>>
>> Any hope at all? I was thinking some trickery on the order of what ByteMan does in order to figure out a way to obtain a local ModelController? But that's a last ditch effort :) Hoping there is something that uses a little less witchcraft.
>> _______________________________________________
>> wildfly-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>
>> _______________________________________________
>> wildfly-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
> --
> Brian Stansberry
> Manager, Senior Principal Software Engineer
> JBoss by Red Hat
>
>
>
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

John Mazzitelli
OK, here's the basic gist of what this is for - I'll try to keep it short.

WHAT THE AGENT DOES:

The "Hawkular WildFly Agent" looks at the tree of resources in its monitored servers (e.g. for WildFly servers, things like "/subsystem=datasources/data-source=MyH2" are resources; for JMX MBeanServers, MBeans are resources), and collects metrics from those resources. It stores that inventory and metric data to a backend Hawkular Server. Systems like ManageIQ, CloudForms, and even things like Grafana can then display in their UIs the inventory details (e.g. "here's a list of your 5 EAP servers"), and pretty graphs of all their metrics (e.g. "here's what your 5 EAP servers' heap memory usages look like").

HOW THE AGENT RUNS TODAY:

Today, Hawkular WildFly Agent MUST BE installed as a subsystem. It can be installed in EAP 6.4, EAP 7.0/7.1, and their associated WildFly versions - but it has to be hosted in one of those things because it is implemented as a subsystem extension. It can be installed in a standalone server or a host controller. It can collect inventory and metric data for the server it is running in.

WHY WE WANT TO CHANGE:

But we have received feedback that "it would be nice to support this agent in a bunch of other projects that are not EAP based." Think Karaf-based or vert-x based applications that expose inventory/metrics over JMX. We want to collect inventory and metrics from non-EAP-based products that expose JMX so ManageIQ and CloudForms can see them just like EAP-based projects can be seen today.

So I refactored the Hawkular Agent core engine so it can run EITHER as a subsystem extension OR as a VM java agent.

Then I have received feedback, "it would be nice to not have to support and maintain both versions of the agent - just support the Java Agent since it can monitor EAP and non-EAP based products. Throw away the subsystem one since it isn't needed anymore."

WHAT IS THE PROBLEM?

Right now, I have a java agent that is working well and can support both EAP and non-EAP based products. However, there are two problems:

1) Unrelated to this email thread, but I can't run this java agent in a Host controller (that's this JIRA: https://issues.jboss.org/browse/WFCORE-2526) - the workaround for this is to run the agent in its own JVM and remotely monitor the Host Controller. Not ideal because now it has to go through the remote management endpoint. Other workaround is to rip out jboss logging entirely from my agent and use some other logging framework. But again, ignore that for now. Not relevant to this discussion.

2) Even though the agent is running co-located inside the WildFly JVM as a javaagent, it can't get a local client so it can talk locally to it. It must go through the remote management endpoint. I would like this agent to get a local ModelController so it can build clients like it can today when running in a subsystem. The agent (when running as a subsystem) uses code similar to this:

    InjectedValue<ModelController> mcValue = new InjectedValue<>();
   ...
   ((ServiceBuilder) bldr).addDependency(Services.JBOSS_SERVER_CONTROLLER, ModelController.class, mcValue);
   ...
   WHAT_I_WANT = mcValue.getValue().createClient(...)

I need something similar in the javaagent - I need to get that local ModelController so the javaagent can build a local client and thus avoid having to configure the WildFly remote management interface and avoid having to make remote calls to its co-located WildFly Server.

Hopefully, that makes sense. Sorry - wasn't short :)

----- Original Message -----

> Hi John,
>
> Before moving with non-traditional access like this I think we need to make
> sure the overall strategy behind what you are building is something we can
> support over multiple releases. Of course, a lot of that depends on your
> compatibility expectations.
>
> > On Mar 13, 2017, at 10:08 AM, Brian Stansberry
> > <[hidden email]> wrote:
> >
> > Once you get access to MSC, do not use Services.JBOSS_SERVER_CONTROLLER to
> > get a ModelController to create the client. That method is deprecated and
> > I plan to remove it in the next release (WildFly Core 4 / WildFly 12).
> >
> > Use
> > ServiceName.parse("org.wildfly.managment.model-controller-client-factory”)
> > to get a ModelControllerClientFactory and use it to create a client,
> > calling createSuperUserClient.
> >
> > Note that if you want that client to work properly in a SecurityManager
> > enabled environment your code will need to have
> > org.jboss.as.controller.security.ControllerPermission#PERFORM_IN_VM_CALL
> > PERFORM_IN_VM_CALL. The security guys added in that requirement for WF
> > Core 3. Note that applies even if you use the deprecated
> > ModelController.createClient method.
> >
> >> On Mar 12, 2017, at 8:32 PM, Stuart Douglas <[hidden email]>
> >> wrote:
> >>
> >> You can call
> >> org.jboss.as.server.CurrentServiceContainer#getServiceContainer() to do
> >> this, however it is trickey from a JavaAgent (as the module will not be
> >> available from the agent's class loader).
> >>
> >> Unfortunately there is no getting around this, as none of the API classes
> >> you need will be available in the module. As I see it you have a few
> >> different options:
> >>
> >> 1) Use reflection to get hold of this class, and then use reflection to
> >> make the calls
> >> 2) Create a class that does this directly, and then make sure it is loaded
> >> from the server module (which has access to the classes you need).
> >>
> >> There may be some other options, but that is all I can think of of the top
> >> of my head.
> >>
> >> If you want more info on how to implement either of these approaches feel
> >> free to ask me on hipchat.
> >>
> >> Stuart
> >>
> >> On Sat, Mar 11, 2017 at 5:43 PM, John Mazzitelli <[hidden email]> wrote:
> >> OK, here's another one where I need some secret magical sauce - hoping
> >> someone knows of a technique I can use.
> >>
> >> Suppose I have a javaagent installed in a WildFly server (using the
> >> standard -javaagent VM argument).
> >>
> >> The javaagent would like to talk to the WildFly Server it is co-located
> >> with. Since it is in the same VM, the javaagent wants to avoid it looking
> >> like a remote call.
> >>
> >> But I know of no way to obtain a local ModelController instance to build a
> >> client short of injecting some service or subsystem into WildFly itself
> >> (something I would like to avoid).
> >>
> >> If the javaagent were instead a subsystem extension, it could do something
> >> like this:
> >>
> >>   InjectedValue<ModelController> mcValue = new InjectedValue<>();
> >>   ...
> >>   ((ServiceBuilder) bldr).addDependency(Services.JBOSS_SERVER_CONTROLLER,
> >>   ModelController.class, mcValue);
> >>   ...
> >>   WHAT_I_WANT = mcValue.getValue().createClient(...)
> >>
> >> But obviously, that's no good for something running outside of the WildFly
> >> container (albeit in the same JVM).
> >>
> >> Any hope at all? I was thinking some trickery on the order of what ByteMan
> >> does in order to figure out a way to obtain a local ModelController? But
> >> that's a last ditch effort :) Hoping there is something that uses a
> >> little less witchcraft.

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

David M. Lloyd
To me it seems like, when running in WildFly you always want it to be a
subsystem, and when running not in WildFly, you always want it to be an
agent.

In other words, you're describing two pieces of software (with some
shared/common code) that do two different things in two different
scenarios, and you're trying to make it all one.

If my understanding is correct, I don't think this is really going to
save any work compared to supporting two different things with common
code.  If anything, the combinatory approach going to cause extra work
due to trying to make a square-and-round peg try to fit in a
square-or-round hole.

On 03/13/2017 12:19 PM, John Mazzitelli wrote:

> OK, here's the basic gist of what this is for - I'll try to keep it short.
>
> WHAT THE AGENT DOES:
>
> The "Hawkular WildFly Agent" looks at the tree of resources in its monitored servers (e.g. for WildFly servers, things like "/subsystem=datasources/data-source=MyH2" are resources; for JMX MBeanServers, MBeans are resources), and collects metrics from those resources. It stores that inventory and metric data to a backend Hawkular Server. Systems like ManageIQ, CloudForms, and even things like Grafana can then display in their UIs the inventory details (e.g. "here's a list of your 5 EAP servers"), and pretty graphs of all their metrics (e.g. "here's what your 5 EAP servers' heap memory usages look like").
>
> HOW THE AGENT RUNS TODAY:
>
> Today, Hawkular WildFly Agent MUST BE installed as a subsystem. It can be installed in EAP 6.4, EAP 7.0/7.1, and their associated WildFly versions - but it has to be hosted in one of those things because it is implemented as a subsystem extension. It can be installed in a standalone server or a host controller. It can collect inventory and metric data for the server it is running in.
>
> WHY WE WANT TO CHANGE:
>
> But we have received feedback that "it would be nice to support this agent in a bunch of other projects that are not EAP based." Think Karaf-based or vert-x based applications that expose inventory/metrics over JMX. We want to collect inventory and metrics from non-EAP-based products that expose JMX so ManageIQ and CloudForms can see them just like EAP-based projects can be seen today.
>
> So I refactored the Hawkular Agent core engine so it can run EITHER as a subsystem extension OR as a VM java agent.
>
> Then I have received feedback, "it would be nice to not have to support and maintain both versions of the agent - just support the Java Agent since it can monitor EAP and non-EAP based products. Throw away the subsystem one since it isn't needed anymore."
>
> WHAT IS THE PROBLEM?
>
> Right now, I have a java agent that is working well and can support both EAP and non-EAP based products. However, there are two problems:
>
> 1) Unrelated to this email thread, but I can't run this java agent in a Host controller (that's this JIRA: https://issues.jboss.org/browse/WFCORE-2526) - the workaround for this is to run the agent in its own JVM and remotely monitor the Host Controller. Not ideal because now it has to go through the remote management endpoint. Other workaround is to rip out jboss logging entirely from my agent and use some other logging framework. But again, ignore that for now. Not relevant to this discussion.
>
> 2) Even though the agent is running co-located inside the WildFly JVM as a javaagent, it can't get a local client so it can talk locally to it. It must go through the remote management endpoint. I would like this agent to get a local ModelController so it can build clients like it can today when running in a subsystem. The agent (when running as a subsystem) uses code similar to this:
>
>     InjectedValue<ModelController> mcValue = new InjectedValue<>();
>    ...
>    ((ServiceBuilder) bldr).addDependency(Services.JBOSS_SERVER_CONTROLLER, ModelController.class, mcValue);
>    ...
>    WHAT_I_WANT = mcValue.getValue().createClient(...)
>
> I need something similar in the javaagent - I need to get that local ModelController so the javaagent can build a local client and thus avoid having to configure the WildFly remote management interface and avoid having to make remote calls to its co-located WildFly Server.
>
> Hopefully, that makes sense. Sorry - wasn't short :)
>
> ----- Original Message -----
>> Hi John,
>>
>> Before moving with non-traditional access like this I think we need to make
>> sure the overall strategy behind what you are building is something we can
>> support over multiple releases. Of course, a lot of that depends on your
>> compatibility expectations.
>>
>>> On Mar 13, 2017, at 10:08 AM, Brian Stansberry
>>> <[hidden email]> wrote:
>>>
>>> Once you get access to MSC, do not use Services.JBOSS_SERVER_CONTROLLER to
>>> get a ModelController to create the client. That method is deprecated and
>>> I plan to remove it in the next release (WildFly Core 4 / WildFly 12).
>>>
>>> Use
>>> ServiceName.parse("org.wildfly.managment.model-controller-client-factory”)
>>> to get a ModelControllerClientFactory and use it to create a client,
>>> calling createSuperUserClient.
>>>
>>> Note that if you want that client to work properly in a SecurityManager
>>> enabled environment your code will need to have
>>> org.jboss.as.controller.security.ControllerPermission#PERFORM_IN_VM_CALL
>>> PERFORM_IN_VM_CALL. The security guys added in that requirement for WF
>>> Core 3. Note that applies even if you use the deprecated
>>> ModelController.createClient method.
>>>
>>>> On Mar 12, 2017, at 8:32 PM, Stuart Douglas <[hidden email]>
>>>> wrote:
>>>>
>>>> You can call
>>>> org.jboss.as.server.CurrentServiceContainer#getServiceContainer() to do
>>>> this, however it is trickey from a JavaAgent (as the module will not be
>>>> available from the agent's class loader).
>>>>
>>>> Unfortunately there is no getting around this, as none of the API classes
>>>> you need will be available in the module. As I see it you have a few
>>>> different options:
>>>>
>>>> 1) Use reflection to get hold of this class, and then use reflection to
>>>> make the calls
>>>> 2) Create a class that does this directly, and then make sure it is loaded
>>>> from the server module (which has access to the classes you need).
>>>>
>>>> There may be some other options, but that is all I can think of of the top
>>>> of my head.
>>>>
>>>> If you want more info on how to implement either of these approaches feel
>>>> free to ask me on hipchat.
>>>>
>>>> Stuart
>>>>
>>>> On Sat, Mar 11, 2017 at 5:43 PM, John Mazzitelli <[hidden email]> wrote:
>>>> OK, here's another one where I need some secret magical sauce - hoping
>>>> someone knows of a technique I can use.
>>>>
>>>> Suppose I have a javaagent installed in a WildFly server (using the
>>>> standard -javaagent VM argument).
>>>>
>>>> The javaagent would like to talk to the WildFly Server it is co-located
>>>> with. Since it is in the same VM, the javaagent wants to avoid it looking
>>>> like a remote call.
>>>>
>>>> But I know of no way to obtain a local ModelController instance to build a
>>>> client short of injecting some service or subsystem into WildFly itself
>>>> (something I would like to avoid).
>>>>
>>>> If the javaagent were instead a subsystem extension, it could do something
>>>> like this:
>>>>
>>>>   InjectedValue<ModelController> mcValue = new InjectedValue<>();
>>>>   ...
>>>>   ((ServiceBuilder) bldr).addDependency(Services.JBOSS_SERVER_CONTROLLER,
>>>>   ModelController.class, mcValue);
>>>>   ...
>>>>   WHAT_I_WANT = mcValue.getValue().createClient(...)
>>>>
>>>> But obviously, that's no good for something running outside of the WildFly
>>>> container (albeit in the same JVM).
>>>>
>>>> Any hope at all? I was thinking some trickery on the order of what ByteMan
>>>> does in order to figure out a way to obtain a local ModelController? But
>>>> that's a last ditch effort :) Hoping there is something that uses a
>>>> little less witchcraft.
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>

--
- DML
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

jtgreene
Administrator
In reply to this post by John Mazzitelli
Thanks for the info. Sorry to be annoying and ask the same question over and over, but what is missing from the remote management protocol that WildFly already provides?  

> On Mar 13, 2017, at 12:19 PM, John Mazzitelli <[hidden email]> wrote:
>
> OK, here's the basic gist of what this is for - I'll try to keep it short.
>
> WHAT THE AGENT DOES:
>
> The "Hawkular WildFly Agent" looks at the tree of resources in its monitored servers (e.g. for WildFly servers, things like "/subsystem=datasources/data-source=MyH2" are resources; for JMX MBeanServers, MBeans are resources), and collects metrics from those resources. It stores that inventory and metric data to a backend Hawkular Server. Systems like ManageIQ, CloudForms, and even things like Grafana can then display in their UIs the inventory details (e.g. "here's a list of your 5 EAP servers"), and pretty graphs of all their metrics (e.g. "here's what your 5 EAP servers' heap memory usages look like").
>
> HOW THE AGENT RUNS TODAY:
>
> Today, Hawkular WildFly Agent MUST BE installed as a subsystem. It can be installed in EAP 6.4, EAP 7.0/7.1, and their associated WildFly versions - but it has to be hosted in one of those things because it is implemented as a subsystem extension. It can be installed in a standalone server or a host controller. It can collect inventory and metric data for the server it is running in.
>
> WHY WE WANT TO CHANGE:
>
> But we have received feedback that "it would be nice to support this agent in a bunch of other projects that are not EAP based." Think Karaf-based or vert-x based applications that expose inventory/metrics over JMX. We want to collect inventory and metrics from non-EAP-based products that expose JMX so ManageIQ and CloudForms can see them just like EAP-based projects can be seen today.
>
> So I refactored the Hawkular Agent core engine so it can run EITHER as a subsystem extension OR as a VM java agent.
>
> Then I have received feedback, "it would be nice to not have to support and maintain both versions of the agent - just support the Java Agent since it can monitor EAP and non-EAP based products. Throw away the subsystem one since it isn't needed anymore."
>
> WHAT IS THE PROBLEM?
>
> Right now, I have a java agent that is working well and can support both EAP and non-EAP based products. However, there are two problems:
>
> 1) Unrelated to this email thread, but I can't run this java agent in a Host controller (that's this JIRA: https://issues.jboss.org/browse/WFCORE-2526) - the workaround for this is to run the agent in its own JVM and remotely monitor the Host Controller. Not ideal because now it has to go through the remote management endpoint. Other workaround is to rip out jboss logging entirely from my agent and use some other logging framework. But again, ignore that for now. Not relevant to this discussion.
>
> 2) Even though the agent is running co-located inside the WildFly JVM as a javaagent, it can't get a local client so it can talk locally to it. It must go through the remote management endpoint. I would like this agent to get a local ModelController so it can build clients like it can today when running in a subsystem. The agent (when running as a subsystem) uses code similar to this:
>
>    InjectedValue<ModelController> mcValue = new InjectedValue<>();
>   ...
>   ((ServiceBuilder) bldr).addDependency(Services.JBOSS_SERVER_CONTROLLER, ModelController.class, mcValue);
>   ...
>   WHAT_I_WANT = mcValue.getValue().createClient(...)
>
> I need something similar in the javaagent - I need to get that local ModelController so the javaagent can build a local client and thus avoid having to configure the WildFly remote management interface and avoid having to make remote calls to its co-located WildFly Server.
>
> Hopefully, that makes sense. Sorry - wasn't short :)
>
> ----- Original Message -----
>> Hi John,
>>
>> Before moving with non-traditional access like this I think we need to make
>> sure the overall strategy behind what you are building is something we can
>> support over multiple releases. Of course, a lot of that depends on your
>> compatibility expectations.
>>
>>> On Mar 13, 2017, at 10:08 AM, Brian Stansberry
>>> <[hidden email]> wrote:
>>>
>>> Once you get access to MSC, do not use Services.JBOSS_SERVER_CONTROLLER to
>>> get a ModelController to create the client. That method is deprecated and
>>> I plan to remove it in the next release (WildFly Core 4 / WildFly 12).
>>>
>>> Use
>>> ServiceName.parse("org.wildfly.managment.model-controller-client-factory”)
>>> to get a ModelControllerClientFactory and use it to create a client,
>>> calling createSuperUserClient.
>>>
>>> Note that if you want that client to work properly in a SecurityManager
>>> enabled environment your code will need to have
>>> org.jboss.as.controller.security.ControllerPermission#PERFORM_IN_VM_CALL
>>> PERFORM_IN_VM_CALL. The security guys added in that requirement for WF
>>> Core 3. Note that applies even if you use the deprecated
>>> ModelController.createClient method.
>>>
>>>> On Mar 12, 2017, at 8:32 PM, Stuart Douglas <[hidden email]>
>>>> wrote:
>>>>
>>>> You can call
>>>> org.jboss.as.server.CurrentServiceContainer#getServiceContainer() to do
>>>> this, however it is trickey from a JavaAgent (as the module will not be
>>>> available from the agent's class loader).
>>>>
>>>> Unfortunately there is no getting around this, as none of the API classes
>>>> you need will be available in the module. As I see it you have a few
>>>> different options:
>>>>
>>>> 1) Use reflection to get hold of this class, and then use reflection to
>>>> make the calls
>>>> 2) Create a class that does this directly, and then make sure it is loaded
>>>> from the server module (which has access to the classes you need).
>>>>
>>>> There may be some other options, but that is all I can think of of the top
>>>> of my head.
>>>>
>>>> If you want more info on how to implement either of these approaches feel
>>>> free to ask me on hipchat.
>>>>
>>>> Stuart
>>>>
>>>> On Sat, Mar 11, 2017 at 5:43 PM, John Mazzitelli <[hidden email]> wrote:
>>>> OK, here's another one where I need some secret magical sauce - hoping
>>>> someone knows of a technique I can use.
>>>>
>>>> Suppose I have a javaagent installed in a WildFly server (using the
>>>> standard -javaagent VM argument).
>>>>
>>>> The javaagent would like to talk to the WildFly Server it is co-located
>>>> with. Since it is in the same VM, the javaagent wants to avoid it looking
>>>> like a remote call.
>>>>
>>>> But I know of no way to obtain a local ModelController instance to build a
>>>> client short of injecting some service or subsystem into WildFly itself
>>>> (something I would like to avoid).
>>>>
>>>> If the javaagent were instead a subsystem extension, it could do something
>>>> like this:
>>>>
>>>>  InjectedValue<ModelController> mcValue = new InjectedValue<>();
>>>>  ...
>>>>  ((ServiceBuilder) bldr).addDependency(Services.JBOSS_SERVER_CONTROLLER,
>>>>  ModelController.class, mcValue);
>>>>  ...
>>>>  WHAT_I_WANT = mcValue.getValue().createClient(...)
>>>>
>>>> But obviously, that's no good for something running outside of the WildFly
>>>> container (albeit in the same JVM).
>>>>
>>>> Any hope at all? I was thinking some trickery on the order of what ByteMan
>>>> does in order to figure out a way to obtain a local ModelController? But
>>>> that's a last ditch effort :) Hoping there is something that uses a
>>>> little less witchcraft.

--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

John Mazzitelli
> Thanks for the info. Sorry to be annoying and ask the same question over and
> over, but what is missing from the remote management protocol that WildFly
> already provides?

Nothing is "missing" from it per-se. As a workaround, the agent is going through this remote management protocol so it can do its thing. One feature I don't think I mentioned is the agent can monitor remote WildFly (and Jolokia) servers - I use that feature to monitor its co-located local WildFly server. Its local WildFly simply looks like a remote server and it monitors it as such. So the javaagent, even though it is running co-located in the same VM as the WildFly Server, is talking over the remote management endpoint to its WildFly server as a workaround for this inability to obtain a local client.

However:

1) It requires the user to configure the remote management endpoint simply so it can be monitored by an agent that is already running co-located in the VM. I already envision the customer asking "Tell me again why I need to configure the remote management interface so its javaagent can talk to it?". The customer should actually be able to disable the remote endpoint entirely (perhaps to lock it down and disallow jboss-cli access?) and have this agent still work - since it really doesn't need to (or at least should not have to) talk to the remote endpoint.

2) It requires the agent to be told what this endpoint is - if the user actually bound the remote endpoint to a non-default IP (not 127.0.0.1) or non-default port (not 9990), then they have to make sure they also configure the agent to tell it where the remote endpoint is. Potential place for user-error if they forget to do that. I suppose the agent could read standalone.xml to find out what the mgmt address/port is but a) what if its not standalone.xml? Maybe the user started with a different config? What if its not standalone but deployed in domain mode? Must look at a different config file. b) now the agent has to know how to parse standalone.xml. This can all be avoided if we just didn't have to go over the remote endpoint.

3) Seems inefficient to have to go over a remote endpoint (even if its over a loopback device) when the agent should just be able to make intra-VM Java API calls to do this.

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

jtgreene
Administrator

> On Mar 13, 2017, at 1:18 PM, John Mazzitelli <[hidden email]> wrote:
>
>> Thanks for the info. Sorry to be annoying and ask the same question over and
>> over, but what is missing from the remote management protocol that WildFly
>> already provides?
>
> Nothing is "missing" from it per-se. As a workaround, the agent is going through this remote management protocol so it can do its thing. One feature I don't think I mentioned is the agent can monitor remote WildFly (and Jolokia) servers - I use that feature to monitor its co-located local WildFly server. Its local WildFly simply looks like a remote server and it monitors it as such. So the javaagent, even though it is running co-located in the same VM as the WildFly Server, is talking over the remote management endpoint to its WildFly server as a workaround for this inability to obtain a local client.
>
> However:
>
> 1) It requires the user to configure the remote management endpoint simply so it can be monitored by an agent that is already running co-located in the VM. I already envision the customer asking "Tell me again why I need to configure the remote management interface so its javaagent can talk to it?". The customer should actually be able to disable the remote endpoint entirely (perhaps to lock it down and disallow jboss-cli access?) and have this agent still work - since it really doesn't need to (or at least should not have to) talk to the remote endpoint.
>
> 2) It requires the agent to be told what this endpoint is - if the user actually bound the remote endpoint to a non-default IP (not 127.0.0.1) or non-default port (not 9990), then they have to make sure they also configure the agent to tell it where the remote endpoint is. Potential place for user-error if they forget to do that. I suppose the agent could read standalone.xml to find out what the mgmt address/port is but a) what if its not standalone.xml? Maybe the user started with a different config? What if its not standalone but deployed in domain mode? Must look at a different config file. b) now the agent has to know how to parse standalone.xml. This can all be avoided if we just didn't have to go over the remote endpoint.
>
> 3) Seems inefficient to have to go over a remote endpoint (even if its over a loopback device) when the agent should just be able to make intra-VM Java API calls to do this.
>

Thanks John,

Although I am more generally asking why an agent is needed at all. Why not simply pull from the remote EAP node using a user that is setup for monitoring? That would reduce the software installation and maintenance overhead to just a configuration overhead.

-Jason


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

John Mazzitelli
In reply to this post by David M. Lloyd
> To me it seems like, when running in WildFly you always want it to be a
> subsystem, and when running not in WildFly, you always want it to be an
> agent.
>
> In other words, you're describing two pieces of software (with some
> shared/common code) that do two different things in two different
> scenarios, and you're trying to make it all one.
>
> If my understanding is correct, I don't think this is really going to
> save any work compared to supporting two different things with common
> code.  If anything, the combinatory approach going to cause extra work
> due to trying to make a square-and-round peg try to fit in a
> square-or-round hole.

Well, actually the "have two agents" is going to be more complicated than this "all-in-one agent" - certainly more complicated from a support/maintenance point of view.

I have the all-in-one "java agent" working now (aside from those two issues I mentioned). If I could just get a local ModelController, the same piece of software can monitor:

1) Local WildFly/EAP servers (standalone and host controller)
2) Local JMX servers
3) Remote WildFly/EAP servers (standalone and host controller)
4) Remote JMX Servers

Having this all-in-one javaagent (versus having two separate agents) means the amount of work this would save is quite a lot. For example:

a) We would no longer have to support and maintain the subsystem extension code itself (not to mention all the ancillary code like the feature pack mvn module).

b) We no longer would have to support/maintain/document two different configuration files (one for the <subsystem> in standalone.xml/host.xml and one for the javaagent yaml file)

c) We no longer have to support the agent installer. This agent installer is a piece of additional software that is needed because people complained about having to manually copy the binaries in the add-ons dir and to configure the <subsystem> XML to get the agent installed (along with its related stuff like any required <security-realms> and <socket bindings> that the agent needs). We ended up writing an installer that can install the add-ons binaries and inject the necessary XML in standalone/host xml so the user doesn't have to. But upgrading such an installment is still a problem - that installer doesn't support upgrading the XML.

Does WildFly today have a subsystem extension installer/upgrader that allows users to install add-ons in their own WildFly servers so a user doesn't have to configure standalone.xml/host.xml with the add-on's required <subsystem> XML? Does it modify/upgrade that XML in standalone.xml/host.xml if an older version of the add-on already exists? If such a tool exists, that would help a lot. Right now, it is a pain to have to write and maintain the agent's own installer. :) If WildFly has such a tool available already, then this issue with having to maintain/support an installer goes away for me. I did not think such a thing existed, at least not at the time when we wrote this agent installer.
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

John Mazzitelli
In reply to this post by jtgreene
> Although I am more generally asking why an agent is needed at all. Why not
> simply pull from the remote EAP node using a user that is setup for
> monitoring? That would reduce the software installation and maintenance
> overhead to just a configuration overhead.

Can you clarify? Are you asking, "Why not have a single agent running "somewhere" and have it pull from a collection of remote EAP nodes?"

In fact, this Hawkular Agent does support this kind of deployment (the Agent can be told to monitor N remote servers whether or not those remote servers are running on the same machine or somewhere else on the network).

One issue with something like that is - most customers do not want to expose their remote management endpoints past the localhost boundary. In that case, this means the agent must be at least installed on the same machine as the EAP node itself even if it isn't running in the same JVM. But now they have to install two things - the agent and the WildFly - albeit on the same machine. If that is the case, why not just bundle the agent together with WildFly so you have a single distribution we can call "instrumented WildFly" that a user has to worry about installing? You run your instrumented Wildfly server normally but now you automatically get inventory/metrics uploaded and stored to CloudForms which can then immediately begin to manage it. No need to install and run a separate agent process.

I would like to ask Thomas Heute to reply to this thread as he has more details about why we want it this way - he's dealt alot more with user requirements and the like.

Thomas?

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

jtgreene
Administrator

> On Mar 13, 2017, at 2:08 PM, John Mazzitelli <[hidden email]> wrote:
>
>> Although I am more generally asking why an agent is needed at all. Why not
>> simply pull from the remote EAP node using a user that is setup for
>> monitoring? That would reduce the software installation and maintenance
>> overhead to just a configuration overhead.
>
> Can you clarify? Are you asking, “Why not have a single agent running "somewhere" and have it pull from a collection of remote EAP nodes?"

Right thats what I was asking.

>
> In fact, this Hawkular Agent does support this kind of deployment (the Agent can be told to monitor N remote servers whether or not those remote servers are running on the same machine or somewhere else on the network).
>
> One issue with something like that is - most customers do not want to expose their remote management endpoints past the localhost boundary.

It’s very common for our users to do this (for example you can even create users which can only do monitoring operations); so I’ll take it to mean that your understand is that Hawkular’s prefer the call home approach, which is fair enough. I only ask since the approach is being revisited again.

> In that case, this means the agent must be at least installed on the same machine as the EAP node itself even if it isn't running in the same JVM. But now they have to install two things - the agent and the WildFly - albeit on the same machine.


> If that is the case, why not just bundle the agent together with WildFly so you have a single distribution we can call "instrumented WildFly" that a user has to worry about installing? You run your instrumented Wildfly server normally but now you automatically get inventory/metrics uploaded and stored to CloudForms which can then immediately begin to manage it. No need to install and run a separate agent process.

I think thats a totally reasonable approach. The issue in the past was that the lifecycles didn’t match (I can’t remember the details as to why). If things were to remain as a subsystem that has an independent lifecycle, in the future it would be installable via the wildfly provisioning tool (once available). That includes automating adding config blocks as part of installation. If the lifecycles can match, and there is a reliable compatibility contract for the protocol, then we could just merge the capability into the core code.

If you go the agent route though then there is not much we can do to improve the usability as our tooling isn’t going to be able to resolve the possible conflicts that arrive when more than one thing wants to own the bootstrap of the server.

--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

David M. Lloyd
In reply to this post by John Mazzitelli
On 03/13/2017 01:41 PM, John Mazzitelli wrote:

>> To me it seems like, when running in WildFly you always want it to be a
>> subsystem, and when running not in WildFly, you always want it to be an
>> agent.
>>
>> In other words, you're describing two pieces of software (with some
>> shared/common code) that do two different things in two different
>> scenarios, and you're trying to make it all one.
>>
>> If my understanding is correct, I don't think this is really going to
>> save any work compared to supporting two different things with common
>> code.  If anything, the combinatory approach going to cause extra work
>> due to trying to make a square-and-round peg try to fit in a
>> square-or-round hole.
>
> Well, actually the "have two agents" is going to be more complicated than this "all-in-one agent" - certainly more complicated from a support/maintenance point of view.
>
> I have the all-in-one "java agent" working now (aside from those two issues I mentioned). If I could just get a local ModelController, the same piece of software can monitor:
>
> 1) Local WildFly/EAP servers (standalone and host controller)
> 2) Local JMX servers
> 3) Remote WildFly/EAP servers (standalone and host controller)
> 4) Remote JMX Servers
>
> Having this all-in-one javaagent (versus having two separate agents) means the amount of work this would save is quite a lot. For example:
>
> a) We would no longer have to support and maintain the subsystem extension code itself (not to mention all the ancillary code like the feature pack mvn module).
>
> b) We no longer would have to support/maintain/document two different configuration files (one for the <subsystem> in standalone.xml/host.xml and one for the javaagent yaml file)
>
> c) We no longer have to support the agent installer. This agent installer is a piece of additional software that is needed because people complained about having to manually copy the binaries in the add-ons dir and to configure the <subsystem> XML to get the agent installed (along with its related stuff like any required <security-realms> and <socket bindings> that the agent needs). We ended up writing an installer that can install the add-ons binaries and inject the necessary XML in standalone/host xml so the user doesn't have to. But upgrading such an installment is still a problem - that installer doesn't support upgrading the XML.
>
> Does WildFly today have a subsystem extension installer/upgrader that allows users to install add-ons in their own WildFly servers so a user doesn't have to configure standalone.xml/host.xml with the add-on's required <subsystem> XML?

Generally speaking, no, not yet, though you can use the CLI to do and
script various things.

> Does it modify/upgrade that XML in standalone.xml/host.xml if an older version of the add-on already exists?

This is automatically done at server start in most if not all cases, not
by any tool but by the management model itself.  The XML isn't the
model; it's just a serialization of it.  Interaction with the
configuration often (usually?) is done via CLI and the console, rather
than by manipulating the XML itself.

> If such a tool exists, that would help a lot. Right now, it is a pain to have to write and maintain the agent's own installer. :) If WildFly has such a tool available already, then this issue with having to maintain/support an installer goes away for me. I did not think such a thing existed, at least not at the time when we wrote this agent installer.

If you're manipulating XML directly then it's possible that the CLI can
do most of what your installer is doing in a more resilient manner, and
possibly a simpler manner as well.  Have you looked into it?

--
- DML
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

David M. Lloyd
Also want to add that it is possible that a given JVM will have
zero-or-more WildFly instances in it.  So, the phrase "find the
container" doesn't really have meaning as there could be many (granted
it's not usually practical to do this *today*, but I guarantee that it
will be possible and practical in the not-too-distant future).

On 03/13/2017 05:00 PM, David M. Lloyd wrote:

> On 03/13/2017 01:41 PM, John Mazzitelli wrote:
>>> To me it seems like, when running in WildFly you always want it to be a
>>> subsystem, and when running not in WildFly, you always want it to be an
>>> agent.
>>>
>>> In other words, you're describing two pieces of software (with some
>>> shared/common code) that do two different things in two different
>>> scenarios, and you're trying to make it all one.
>>>
>>> If my understanding is correct, I don't think this is really going to
>>> save any work compared to supporting two different things with common
>>> code.  If anything, the combinatory approach going to cause extra work
>>> due to trying to make a square-and-round peg try to fit in a
>>> square-or-round hole.
>>
>> Well, actually the "have two agents" is going to be more complicated
>> than this "all-in-one agent" - certainly more complicated from a
>> support/maintenance point of view.
>>
>> I have the all-in-one "java agent" working now (aside from those two
>> issues I mentioned). If I could just get a local ModelController, the
>> same piece of software can monitor:
>>
>> 1) Local WildFly/EAP servers (standalone and host controller)
>> 2) Local JMX servers
>> 3) Remote WildFly/EAP servers (standalone and host controller)
>> 4) Remote JMX Servers
>>
>> Having this all-in-one javaagent (versus having two separate agents)
>> means the amount of work this would save is quite a lot. For example:
>>
>> a) We would no longer have to support and maintain the subsystem
>> extension code itself (not to mention all the ancillary code like the
>> feature pack mvn module).
>>
>> b) We no longer would have to support/maintain/document two different
>> configuration files (one for the <subsystem> in
>> standalone.xml/host.xml and one for the javaagent yaml file)
>>
>> c) We no longer have to support the agent installer. This agent
>> installer is a piece of additional software that is needed because
>> people complained about having to manually copy the binaries in the
>> add-ons dir and to configure the <subsystem> XML to get the agent
>> installed (along with its related stuff like any required
>> <security-realms> and <socket bindings> that the agent needs). We
>> ended up writing an installer that can install the add-ons binaries
>> and inject the necessary XML in standalone/host xml so the user
>> doesn't have to. But upgrading such an installment is still a problem
>> - that installer doesn't support upgrading the XML.
>>
>> Does WildFly today have a subsystem extension installer/upgrader that
>> allows users to install add-ons in their own WildFly servers so a user
>> doesn't have to configure standalone.xml/host.xml with the add-on's
>> required <subsystem> XML?
>
> Generally speaking, no, not yet, though you can use the CLI to do and
> script various things.
>
>> Does it modify/upgrade that XML in standalone.xml/host.xml if an older
>> version of the add-on already exists?
>
> This is automatically done at server start in most if not all cases, not
> by any tool but by the management model itself.  The XML isn't the
> model; it's just a serialization of it.  Interaction with the
> configuration often (usually?) is done via CLI and the console, rather
> than by manipulating the XML itself.
>
>> If such a tool exists, that would help a lot. Right now, it is a pain
>> to have to write and maintain the agent's own installer. :) If WildFly
>> has such a tool available already, then this issue with having to
>> maintain/support an installer goes away for me. I did not think such a
>> thing existed, at least not at the time when we wrote this agent
>> installer.
>
> If you're manipulating XML directly then it's possible that the CLI can
> do most of what your installer is doing in a more resilient manner, and
> possibly a simpler manner as well.  Have you looked into it?
>

--
- DML
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

Brian Stansberry
In reply to this post by David M. Lloyd

> On Mar 13, 2017, at 5:00 PM, David M. Lloyd <[hidden email]> wrote:
>
> On 03/13/2017 01:41 PM, John Mazzitelli wrote:
>>> To me it seems like, when running in WildFly you always want it to be a
>>> subsystem, and when running not in WildFly, you always want it to be an
>>> agent.
>>>
>>> In other words, you're describing two pieces of software (with some
>>> shared/common code) that do two different things in two different
>>> scenarios, and you're trying to make it all one.
>>>
>>> If my understanding is correct, I don't think this is really going to
>>> save any work compared to supporting two different things with common
>>> code.  If anything, the combinatory approach going to cause extra work
>>> due to trying to make a square-and-round peg try to fit in a
>>> square-or-round hole.
>>
>> Well, actually the "have two agents" is going to be more complicated than this "all-in-one agent" - certainly more complicated from a support/maintenance point of view.
>>
>> I have the all-in-one "java agent" working now (aside from those two issues I mentioned). If I could just get a local ModelController, the same piece of software can monitor:
>>
>> 1) Local WildFly/EAP servers (standalone and host controller)

Why do you need a ModelControllerClient to look at metrics? I believe metrics should be available over JMX. In another thread you asked about DMR->JMX and mentioned deployments. So if you want to do complex stuff like deployments, or really any writes, using a ModelControllerClient makes sense. But the "WHAT THE AGENT DOES:” section earlier in this thread just talks about metrics.

>> 2) Local JMX servers
>> 3) Remote WildFly/EAP servers (standalone and host controller)
>> 4) Remote JMX Servers
>>
>> Having this all-in-one javaagent (versus having two separate agents) means the amount of work this would save is quite a lot. For example:
>>
>> a) We would no longer have to support and maintain the subsystem extension code itself (not to mention all the ancillary code like the feature pack mvn module).
>>
>> b) We no longer would have to support/maintain/document two different configuration files (one for the <subsystem> in standalone.xml/host.xml and one for the javaagent yaml file)
>>
>> c) We no longer have to support the agent installer. This agent installer is a piece of additional software that is needed because people complained about having to manually copy the binaries in the add-ons dir and to configure the <subsystem> XML to get the agent installed (along with its related stuff like any required <security-realms> and <socket bindings> that the agent needs).

This security and socket related config that used to be part of the server config is now in a yaml file the the javaagent is configured to access?

>> We ended up writing an installer that can install the add-ons binaries and inject the necessary XML in standalone/host xml so the user doesn't have to. But upgrading such an installment is still a problem - that installer doesn't support upgrading the XML.
>>
>> Does WildFly today have a subsystem extension installer/upgrader that allows users to install add-ons in their own WildFly servers so a user doesn't have to configure standalone.xml/host.xml with the add-on's required <subsystem> XML?
>
> Generally speaking, no, not yet, though you can use the CLI to do and
> script various things.
>
>> Does it modify/upgrade that XML in standalone.xml/host.xml if an older version of the add-on already exists?
>
> This is automatically done at server start in most if not all cases, not
> by any tool but by the management model itself.  The XML isn't the
> model; it's just a serialization of it.  

This depends on what the upgrade is. The server will automatically move a subsystem config to the latest schema version, but if the new code has some new functionality we don’t automatically inject that into the config. Your parser for the old schema version could do that in theory, but generally that’s a bad practice.

> Interaction with the
> configuration often (usually?) is done via CLI and the console, rather
> than by manipulating the XML itself.
>
>> If such a tool exists, that would help a lot. Right now, it is a pain to have to write and maintain the agent's own installer. :) If WildFly has such a tool available already, then this issue with having to maintain/support an installer goes away for me. I did not think such a thing existed, at least not at the time when we wrote this agent installer.
>
> If you're manipulating XML directly then it's possible that the CLI can
> do most of what your installer is doing in a more resilient manner, and
> possibly a simpler manner as well.  Have you looked into it?
>

The EAP installer itself uses the CLI in this way.

> --
> - DML
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

--
Brian Stansberry
Manager, Senior Principal Software Engineer
JBoss by Red Hat




_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

John Mazzitelli
> >> 1) Local WildFly/EAP servers (standalone and host controller)
>
> Why do you need a ModelControllerClient to look at metrics? I believe metrics
> should be available over JMX. In another thread you asked about DMR->JMX and
> mentioned deployments. So if you want to do complex stuff like deployments,
> or really any writes, using a ModelControllerClient makes sense. But the
> "WHAT THE AGENT DOES:” section earlier in this thread just talks about
> metrics.

Two things:

1) Recall that I'm porting an existing agent subsystem that already supported all of this by going through ModelController client. So rather than re-write everything to go over JMX, I'm reusing/refactoring functionality I already have. Going this route let me have a fully functional javaagent in a couple days.

2) Plus, yes, as you state, I neglected to say it needs deployment functionality - that requires ModelControllerClient anyway.

> >> c) We no longer have to support the agent installer. This agent installer
> >> is a piece of additional software that is needed because people
> >> complained about having to manually copy the binaries in the add-ons dir
> >> and to configure the <subsystem> XML to get the agent installed (along
> >> with its related stuff like any required <security-realms> and <socket
> >> bindings> that the agent needs).
>
> This security and socket related config that used to be part of the server
> config is now in a yaml file the the javaagent is configured to access?

Yes. everything is now in yaml - nothing is in standalone.xml if using this javaagent. re: security-realms - I only need truststore definitions (not the full security-realm stuff) so that's all the yaml defines - truststore details. Of course, this means the loss of the vault, so I need a way to somehow obfuscate passwords - but if we are deploying this in OpenShift, I plan on using something like ${x} tokens in the yaml and being able to use OS secrets and somehow pass them to the agent without passwords in plaintext.

> > If you're manipulating XML directly then it's possible that the CLI can
> > do most of what your installer is doing in a more resilient manner, and
> > possibly a simpler manner as well.  Have you looked into it?
> >
>
> The EAP installer itself uses the CLI in this way.

Yes, I'm sure I could use a CLI script to do most of this. Do people find it acceptable to edit a CLI script if they want to change the default out of box config (as opposed to editing the XML)? How do you, for example, supply a CLI script that lays out a default set of config items for a subsystem but offer a way for customers to customize that out of box config (for example, let them give the installer a truststore with their certs, or let them give you IP addresses and ports they want to use for outbound socket bindings or things like that, if they want to use http skip security-realm definitions but if they want to use https, ask the user for security-realm/truststore details)? Does jboss-cli.sh provide ways to ask questions and set the values of attributes based on the answers? That would be cool. And if it has that functionality, I'm ashamed for not knowing about it :) Honestly, I never looked into whether the CLI can ask for user-input before applying a write-attribute command.

Also the agent installer also copies binaries to the add-ons directory. It also optionally pulls down the EAP-specific wf-extension.zip (includes the extension module binaries for the add-ons directory and a default xml snippet appropriate for injection into that EAP version) from the hawkular server. It also determines if its installing in EAP 6.4 and if so uses different config than EAP7/Wildfly.

So writing a set of CLI commands is most but not all it does.

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

Brian Stansberry
Thanks for clarifying about JMX.*

I’ll answer the question about offline CLI first.

The offline CLI is not a substitute for a full featured installer.  If all you need is some simple conditional logic, e.g. “if subsystem=foo doesn’t exist, add it” or “if subsystem=foo/newfeature=y doesn’t exist, add it” you could just execute the CLI with a script with that kind of logic. And people could certainly edit that kind of script to make simple tweaks. But if you want sophisticated interaction with the user you need something more. (That’s true with an approach based on changing xml too.)

The offline CLI however is something that can be embedded in a more sophisticated installer and can provide that installer client access to the server configuration in a controlled way without any remote interaction involved. This thread is about a request for a hook to get non-remote client access to the server configuration which is why we are pointing to it. It’s a supported solution whose semantics we understand as opposed to something new and unspecified. Using it does not involve needing access to server internals like the ServiceContainer.

I’m unclear what all the javaagent will do. Is it basically just an installer, something that is going to set up the existing subsystem? Or is the existing subsystem basically gone and now the only interaction with the WildFly container is via this ModelControllerClient you are trying to access, and also JMX? So all the interaction with the remote Hawkular server, including security thereof, is hidden inside the agent layer and is unknown to the WildFly server? Or are there other services being installed in or access from the WildFly container?

Extending WildFly by changing standalone.conf/domain.conf to add stuff to JAVA_OPTS isn’t great. It has the same weaknesses as needing to edit standalone.xml, but in a new place. But it is actually easier to externalize some commands to run some code at boot (which AIUI is the only point of your using —javaagent) to a well known location and have our launch scripts read that. Easier than reading and incorporating arbitrary chunks of xml config, which is the longstanding request in this area. And more powerful than reading and executing chunks of CLI script.

* Tangent: TBH I’m not clear why this agent needs to be doing deployments. If it’s a general purpose monitoring agent reusable across a variety of types of processes, deployments are an odd fit.

> On Mar 13, 2017, at 7:42 PM, John Mazzitelli <[hidden email]> wrote:
>
>>>> 1) Local WildFly/EAP servers (standalone and host controller)
>>
>> Why do you need a ModelControllerClient to look at metrics? I believe metrics
>> should be available over JMX. In another thread you asked about DMR->JMX and
>> mentioned deployments. So if you want to do complex stuff like deployments,
>> or really any writes, using a ModelControllerClient makes sense. But the
>> "WHAT THE AGENT DOES:” section earlier in this thread just talks about
>> metrics.
>
> Two things:
>
> 1) Recall that I'm porting an existing agent subsystem that already supported all of this by going through ModelController client. So rather than re-write everything to go over JMX, I'm reusing/refactoring functionality I already have. Going this route let me have a fully functional javaagent in a couple days.
>
> 2) Plus, yes, as you state, I neglected to say it needs deployment functionality - that requires ModelControllerClient anyway.
>

Ok, thanks.

>>>> c) We no longer have to support the agent installer. This agent installer
>>>> is a piece of additional software that is needed because people
>>>> complained about having to manually copy the binaries in the add-ons dir
>>>> and to configure the <subsystem> XML to get the agent installed (along
>>>> with its related stuff like any required <security-realms> and <socket
>>>> bindings> that the agent needs).
>>
>> This security and socket related config that used to be part of the server
>> config is now in a yaml file the the javaagent is configured to access?
>
> Yes. everything is now in yaml - nothing is in standalone.xml if using this javaagent. re: security-realms - I only need truststore definitions (not the full security-realm stuff) so that's all the yaml defines - truststore details. Of course, this means the loss of the vault, so I need a way to somehow obfuscate passwords - but if we are deploying this in OpenShift, I plan on using something like ${x} tokens in the yaml and being able to use OS secrets and somehow pass them to the agent without passwords in plaintext.
>
>>> If you're manipulating XML directly then it's possible that the CLI can
>>> do most of what your installer is doing in a more resilient manner, and
>>> possibly a simpler manner as well.  Have you looked into it?
>>>
>>
>> The EAP installer itself uses the CLI in this way.
>
> Yes, I'm sure I could use a CLI script to do most of this. Do people find it acceptable to edit a CLI script if they want to change the default out of box config (as opposed to editing the XML)? How do you, for example, supply a CLI script that lays out a default set of config items for a subsystem but offer a way for customers to customize that out of box config (for example, let them give the installer a truststore with their certs, or let them give you IP addresses and ports they want to use for outbound socket bindings or things like that, if they want to use http skip security-realm definitions but if they want to use https, ask the user for security-realm/truststore details)? Does jboss-cli.sh provide ways to ask questions and set the values of attributes based on the answers?

No, but neither can simple things like xslt.

> That would be cool. And if it has that functionality, I'm ashamed for not knowing about it :) Honestly, I never looked into whether the CLI can ask for user-input before applying a write-attribute command.

>
> Also the agent installer also copies binaries to the add-ons directory. It also optionally pulls down the EAP-specific wf-extension.zip (includes the extension module binaries for the add-ons directory and a default xml snippet appropriate for injection into that EAP version) from the hawkular server. It also determines if its installing in EAP 6.4 and if so uses different config than EAP7/Wildfly.
>
> So writing a set of CLI commands is most but not all it does.

--
Brian Stansberry
Manager, Senior Principal Software Engineer
JBoss by Red Hat




_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

Darran Lofthouse
In reply to this post by John Mazzitelli


On 14/03/17 00:42, John Mazzitelli wrote:
> Yes. everything is now in yaml - nothing is in standalone.xml if using this javaagent. re: security-realms - I only need truststore definitions (not the full security-realm stuff) so that's all the yaml defines - truststore details. Of course, this means the loss of the vault, so I need a way to somehow obfuscate passwords - but if we are deploying this in OpenShift, I plan on using something like ${x} tokens in the yaml and being able to use OS secrets and somehow pass them to the agent without passwords in plaintext.

If this part just talking about configuration of the client connecting
to the server or something else?
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

John Mazzitelli
> On 14/03/17 00:42, John Mazzitelli wrote:
> > Yes. everything is now in yaml - nothing is in standalone.xml if using this
> > javaagent. re: security-realms - I only need truststore definitions (not
> > the full security-realm stuff) so that's all the yaml defines - truststore
> > details. Of course, this means the loss of the vault, so I need a way to
> > somehow obfuscate passwords - but if we are deploying this in OpenShift, I
> > plan on using something like ${x} tokens in the yaml and being able to use
> > OS secrets and somehow pass them to the agent without passwords in
> > plaintext.
>
> If this part just talking about configuration of the client connecting
> to the server or something else?


something else. it is talking about the full configuration of what was the subsystem itself (which is now the javaagent)
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: a way to obtain local ModelController within javaagent?

Brian Stansberry
In reply to this post by Brian Stansberry
I was thinking about this yesterday and realized that an MCC accessed this way will be broken following a reload. An extension that obtains an internal client doesn’t have this issue as the entire subsystem goes away and is started again as part of the reload. But something that lives outside that cycle is going to need a different integration.

> On Mar 13, 2017, at 10:08 AM, Brian Stansberry <[hidden email]> wrote:
>
> Once you get access to MSC, do not use Services.JBOSS_SERVER_CONTROLLER to get a ModelController to create the client. That method is deprecated and I plan to remove it in the next release (WildFly Core 4 / WildFly 12).
>
> Use ServiceName.parse("org.wildfly.managment.model-controller-client-factory”) to get a ModelControllerClientFactory and use it to create a client, calling createSuperUserClient.
>
> Note that if you want that client to work properly in a SecurityManager enabled environment your code will need to have org.jboss.as.controller.security.ControllerPermission#PERFORM_IN_VM_CALL PERFORM_IN_VM_CALL. The security guys added in that requirement for WF Core 3. Note that applies even if you use the deprecated ModelController.createClient method.
>
>> On Mar 12, 2017, at 8:32 PM, Stuart Douglas <[hidden email]> wrote:
>>
>> You can call org.jboss.as.server.CurrentServiceContainer#getServiceContainer() to do this, however it is trickey from a JavaAgent (as the module will not be available from the agent's class loader).
>>
>> Unfortunately there is no getting around this, as none of the API classes you need will be available in the module. As I see it you have a few different options:
>>
>> 1) Use reflection to get hold of this class, and then use reflection to make the calls
>> 2) Create a class that does this directly, and then make sure it is loaded from the server module (which has access to the classes you need).
>>
>> There may be some other options, but that is all I can think of of the top of my head.
>>
>> If you want more info on how to implement either of these approaches feel free to ask me on hipchat.
>>
>> Stuart
>>
>> On Sat, Mar 11, 2017 at 5:43 PM, John Mazzitelli <[hidden email]> wrote:
>> OK, here's another one where I need some secret magical sauce - hoping someone knows of a technique I can use.
>>
>> Suppose I have a javaagent installed in a WildFly server (using the standard -javaagent VM argument).
>>
>> The javaagent would like to talk to the WildFly Server it is co-located with. Since it is in the same VM, the javaagent wants to avoid it looking like a remote call.
>>
>> But I know of no way to obtain a local ModelController instance to build a client short of injecting some service or subsystem into WildFly itself (something I would like to avoid).
>>
>> If the javaagent were instead a subsystem extension, it could do something like this:
>>
>>   InjectedValue<ModelController> mcValue = new InjectedValue<>();
>>   ...
>>   ((ServiceBuilder) bldr).addDependency(Services.JBOSS_SERVER_CONTROLLER, ModelController.class, mcValue);
>>   ...
>>   WHAT_I_WANT = mcValue.getValue().createClient(...)
>>
>> But obviously, that's no good for something running outside of the WildFly container (albeit in the same JVM).
>>
>> Any hope at all? I was thinking some trickery on the order of what ByteMan does in order to figure out a way to obtain a local ModelController? But that's a last ditch effort :) Hoping there is something that uses a little less witchcraft.
>> _______________________________________________
>> wildfly-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>
>> _______________________________________________
>> wildfly-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
> --
> Brian Stansberry
> Manager, Senior Principal Software Engineer
> JBoss by Red Hat
>
>
>
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

--
Brian Stansberry
Manager, Senior Principal Software Engineer
JBoss by Red Hat




_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
12
Loading...