Wildfly start-up as service script depends on console log to determinate start result

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Wildfly start-up as service script depends on console log to determinate start result

Chao Wang
Hi all,

The Wildfly start-up as service scripts wildfly-init-redhat.sh and wildfly-init-debian.sh currently depend on a grep action of key message 'WFLYSRV0025:' in console log to determinate whether service start is successful. The log message indication is accurate, however, it's not that robust since user can always remove console handler from logging subsystem. I have opened a WFCORE enhancement jira https://issues.jboss.org/browse/WFCORE-747 for it.

For the moment, I have tried three options, they're all not that perfect to implement

1. Stay with exact log message, users need to define their jboss log directory such as $JBOSS_HOME/standalone/log/server.log for standalone and $JBOSS_HOME/domain/log/host-controller.log for domain instead of searching in console log. This is more like another workaround since it is also
volatile once we update log message in future release.(EAP has 'JBAS015874:')

2. Use service pid, this is not precise because a long start-up can crash in the last second. It needs to wait a suitable seconds before checking pid existence. and still it can not avoid fake success in rare case just before timeout.

3. Use read-attribute server-state through CLI connection as I did in Pull Request on Jira. This is declined as it is possible that authentication is required before connection. In such case, any  non encrypted password is not advised in configuration files.

Therefore, I would like to listen for your opinions for them. Any other suggestion is certainly welcomed in mail or on jira.

Best regards,

Chao

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

Brian Stansberry
A couple thoughts:

1) Looking at wildfly-init-redhat.sh at least, I don't see how that
check is actually testing for successful startup. It looks like it's
just trying to delay start() returning for a while, max 30 secs.

So, what purpose is this fulfilling?

2) How does other software solve this problem? If it's solving a valid
problem, it seems like there would be a typical solution.

On 6/9/15 8:59 AM, Chao Wang wrote:

> Hi all,
>
> The Wildfly start-up as service scripts wildfly-init-redhat.sh and
> wildfly-init-debian.sh currently depend on a grep action of key message
> 'WFLYSRV0025:' in console log to determinate whether service start is
> successful. The log message indication is accurate, however, it's not
> that robust since user can always remove console handler from logging
> subsystem. I have opened a WFCORE enhancement jira
> https://issues.jboss.org/browse/WFCORE-747 for it.
>
> For the moment, I have tried three options, they're all not that perfect
> to implement
>
> 1. Stay with exact log message, users need to define their jboss log
> directory such as $JBOSS_HOME/standalone/log/server.log for standalone
> and $JBOSS_HOME/domain/log/host-controller.log for domain instead of
> searching in console log. This is more like another workaround since it
> is also volatile once we update log message in future release.(EAP has
> 'JBAS015874:')
>
> 2. Use service pid, this is not precise because a long start-up can
> crash in the last second. It needs to wait a suitable seconds before
> checking pid existence. and still it can not avoid fake success in rare
> case just before timeout.
>
> 3. Use read-attribute server-state through CLI connection as I did in
> Pull Request on Jira. This is declined as it is possible that
> authentication is required before connection. In such case, any  non
> encrypted password is not advised in configuration files.
>
> Therefore, I would like to listen for your opinions for them. Any other
> suggestion is certainly welcomed in mail or on jira.
>
> Best regards,
>
> Chao
>
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>


--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

Chao Wang
Thanks for your reply. Please see in-line below

On 06/11/2015 12:44 AM, Brian Stansberry wrote:
A couple thoughts:

1) Looking at wildfly-init-redhat.sh at least, I don't see how that 
check is actually testing for successful startup. It looks like it's 
just trying to delay start() returning for a while, max 30 secs.

So, what purpose is this fulfilling?
My bad about issue background. It's actually an case in EAP 6 (not yet in wildfly). EAP 6.x script has launched state like: https://github.com/jbossas/jboss-eap/blob/6.4.x/build/src/main/resources/bin/init.d/jboss-as-standalone.sh#L110 (not in wildfly's script). Also, console log handler has been removed only in EAP full-ha mode long time ago due to performance concern.(wildfly keeps it for the moment). This leads to issue in bz1224170.
That's why I try to seek an better option than current behavior from wildfly.

2) How does other software solve this problem? If it's solving a valid 
problem, it seems like there would be a typical solution.
I have checked some other application servers, most of them let users themselves to write a script to run as service for their OS. Geronimo does provide a script, Although I did damage to its configuration file to make a fatal error, terminal output still displays "Server started". In fact, process does not event exist and detail error can be seen in log file.
On 6/9/15 8:59 AM, Chao Wang wrote:
Hi all,

The Wildfly start-up as service scripts wildfly-init-redhat.sh and
wildfly-init-debian.sh currently depend on a grep action of key message
'WFLYSRV0025:' in console log to determinate whether service start is
successful. The log message indication is accurate, however, it's not
that robust since user can always remove console handler from logging
subsystem. I have opened a WFCORE enhancement jira
https://issues.jboss.org/browse/WFCORE-747 for it.

For the moment, I have tried three options, they're all not that perfect
to implement

1. Stay with exact log message, users need to define their jboss log
directory such as $JBOSS_HOME/standalone/log/server.log for standalone
and $JBOSS_HOME/domain/log/host-controller.log for domain instead of
searching in console log. This is more like another workaround since it
is also volatile once we update log message in future release.(EAP has
'JBAS015874:')

2. Use service pid, this is not precise because a long start-up can
crash in the last second. It needs to wait a suitable seconds before
checking pid existence. and still it can not avoid fake success in rare
case just before timeout.

3. Use read-attribute server-state through CLI connection as I did in
Pull Request on Jira. This is declined as it is possible that
authentication is required before connection. In such case, any  non
encrypted password is not advised in configuration files.

Therefore, I would like to listen for your opinions for them. Any other
suggestion is certainly welcomed in mail or on jira.

Best regards,

Chao


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev



[1] https://bugzilla.redhat.com/show_bug.cgi?id=1224170
-- 
Chao Wang
Software Engineer
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

Jorge Solórzano
I think there is no easy fix, we can assume for example that after 3 seconds that the PID exists then the process is up and return OK, but if the process stops latter, it will be a fake start.

I know that we should not rely on read the log as James stated, but until we can figure out a better aproach this is my proposal, I can modify wildfly-init-debian.sh to have this behavior

1. Add to logging subsystem to standalone*.xml:
            <logger category="org.jboss.as">
                <level name="INFO"/>
            </logger>

2. Remove JBOSS_CONSOLE_LOG and read JBOSS_LOG_DIR

3. If standalone read "$JBOSS_LOG_DIR/server.log"
 if domain read "$JBOSS_LOG_DIR/host-controller.log"
to find WFLYSRV0025 or the string used for EAP... this is probably volatile but it's not changed frequently, in fact it has been changed only once.

4. Always check the status of pid (if dies return immediately).

5. If server.log is not found, deactivated or not print the correct output there are two choices:
a. On wait timeout, send a warning about an unknow status (this is how works right now).
b. On wait timeout, just check if PID is still up and return OK, otherwise FAILURE.

I believe this approach avoids the problem of console log...

Best regards,


Jorge Solórzano
http://www.jorsol.com

On Thu, Jun 11, 2015 at 5:29 PM, Chao Wang <[hidden email]> wrote:
Thanks for your reply. Please see in-line below

On 06/11/2015 12:44 AM, Brian Stansberry wrote:
A couple thoughts:

1) Looking at wildfly-init-redhat.sh at least, I don't see how that 
check is actually testing for successful startup. It looks like it's 
just trying to delay start() returning for a while, max 30 secs.

So, what purpose is this fulfilling?
My bad about issue background. It's actually an case in EAP 6 (not yet in wildfly). EAP 6.x script has launched state like: https://github.com/jbossas/jboss-eap/blob/6.4.x/build/src/main/resources/bin/init.d/jboss-as-standalone.sh#L110 (not in wildfly's script). Also, console log handler has been removed only in EAP full-ha mode long time ago due to performance concern.(wildfly keeps it for the moment). This leads to issue in bz1224170.
That's why I try to seek an better option than current behavior from wildfly.
2) How does other software solve this problem? If it's solving a valid 
problem, it seems like there would be a typical solution.
I have checked some other application servers, most of them let users themselves to write a script to run as service for their OS. Geronimo does provide a script, Although I did damage to its configuration file to make a fatal error, terminal output still displays "Server started". In fact, process does not event exist and detail error can be seen in log file.
On 6/9/15 8:59 AM, Chao Wang wrote:
Hi all,

The Wildfly start-up as service scripts wildfly-init-redhat.sh and
wildfly-init-debian.sh currently depend on a grep action of key message
'WFLYSRV0025:' in console log to determinate whether service start is
successful. The log message indication is accurate, however, it's not
that robust since user can always remove console handler from logging
subsystem. I have opened a WFCORE enhancement jira
https://issues.jboss.org/browse/WFCORE-747 for it.

For the moment, I have tried three options, they're all not that perfect
to implement

1. Stay with exact log message, users need to define their jboss log
directory such as $JBOSS_HOME/standalone/log/server.log for standalone
and $JBOSS_HOME/domain/log/host-controller.log for domain instead of
searching in console log. This is more like another workaround since it
is also volatile once we update log message in future release.(EAP has
'JBAS015874:')

2. Use service pid, this is not precise because a long start-up can
crash in the last second. It needs to wait a suitable seconds before
checking pid existence. and still it can not avoid fake success in rare
case just before timeout.

3. Use read-attribute server-state through CLI connection as I did in
Pull Request on Jira. This is declined as it is possible that
authentication is required before connection. In such case, any  non
encrypted password is not advised in configuration files.

Therefore, I would like to listen for your opinions for them. Any other
suggestion is certainly welcomed in mail or on jira.
​​
Best regards, Chao _______________________________________________ wildfly-dev mailing list [hidden email] https://lists.jboss.org/mailman/listinfo/wildfly-dev

    

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1224170
-- 
Chao Wang
Software Engineer
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

jtgreene
Administrator
In reply to this post by Brian Stansberry

> On Jun 10, 2015, at 11:46 AM, Brian Stansberry <[hidden email]> wrote:
>
> So, what purpose is this fulfilling?
>
> 2) How does other software solve this problem? If it's solving a valid
> problem, it seems like there would be a typical solution.

The classic UNIX solution is that the daemon forks and returns, dropping a PID file of its child to disk, after it is done initializing and exits with an error code when there is a problem.

Systemd added other approaches, where a daemon can signal systemd directly, or it can use dbus to send a message.

The former can't be done efficiently in Java because it doesn't have a pure fork(), only an exec. Although it would be possible to emulate with an exec with an unacceptable hit to boot time. The latter options are too Linux specific.

I think the best solution would be for us to add a signaling mechanism specifically for this purpose. We could use sun.misc.Signal (potentially an issue on Java 9), or we could exec the kill command to signal a process.

We could also use a specially status file (e.g standalone.sh --start-status-file=blah) for a script to monitor.

Thoughts?
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

David Lloyd-2
This is yet another good use case for an idea I proposed at the last
couple developer meeting: the idea of configurable "availability"
services, where you add a configuration which says "when these
components and/or configured services are available, perform this
action" where the action might be to notify a load balancer, perform a
notification to humans, or even drop a file to the filesystem (which
would be directly useful to this use case).

Note (in case someone thinks this is a great idea and runs out to
implement it right away) that when I say "configured services" I do not
mean MSC service names; more like management capabilities where you have
a constrained namespace and each subsystem can contribute services to
this category.

Also note that Java 9 adds limited support for signalling unrelated
processes, though at the moment on UNIX the signals are basically
limited to TERM and KILL.

On 06/12/2015 07:06 AM, Jason T. Greene wrote:

>
>> On Jun 10, 2015, at 11:46 AM, Brian Stansberry <[hidden email]> wrote:
>>
>> So, what purpose is this fulfilling?
>>
>> 2) How does other software solve this problem? If it's solving a valid
>> problem, it seems like there would be a typical solution.
>
> The classic UNIX solution is that the daemon forks and returns, dropping a PID file of its child to disk, after it is done initializing and exits with an error code when there is a problem.
>
> Systemd added other approaches, where a daemon can signal systemd directly, or it can use dbus to send a message.
>
> The former can't be done efficiently in Java because it doesn't have a pure fork(), only an exec. Although it would be possible to emulate with an exec with an unacceptable hit to boot time. The latter options are too Linux specific.
>
> I think the best solution would be for us to add a signaling mechanism specifically for this purpose. We could use sun.misc.Signal (potentially an issue on Java 9), or we could exec the kill command to signal a process.
>
> We could also use a specially status file (e.g standalone.sh --start-status-file=blah) for a script to monitor.
>
> Thoughts?
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>

--
- DML
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

Brian Stansberry
In reply to this post by Chao Wang
How does EAP 5 deal with this? IIRC disabling console logging is not
something new in EAP 6.

On 6/11/15 6:29 PM, Chao Wang wrote:

> Thanks for your reply. Please see in-line below
>
> On 06/11/2015 12:44 AM, Brian Stansberry wrote:
>> A couple thoughts:
>>
>> 1) Looking at wildfly-init-redhat.sh at least, I don't see how that
>> check is actually testing for successful startup. It looks like it's
>> just trying to delay start() returning for a while, max 30 secs.
>>
>> So, what purpose is this fulfilling?
> My bad about issue background. It's actually an case in EAP 6 (not yet
> in wildfly). EAP 6.x script has launched state like:
> https://github.com/jbossas/jboss-eap/blob/6.4.x/build/src/main/resources/bin/init.d/jboss-as-standalone.sh#L110
> (not in wildfly's script). Also, console log handler has been removed
> only in EAP full-ha mode long time ago due to performance
> concern.(wildfly keeps it for the moment). This leads to issue in
> bz1224170 <https://bugzilla.redhat.com/show_bug.cgi?id=1224170>.
> That's why I try to seek an better option than current behavior from
> wildfly.
>>
>> 2) How does other software solve this problem? If it's solving a valid
>> problem, it seems like there would be a typical solution.
> I have checked some other application servers, most of them let users
> themselves to write a script to run as service for their OS. Geronimo
> does provide a script, Although I did damage to its configuration file
> to make a fatal error, terminal output still displays "Server started".
> In fact, process does not event exist and detail error can be seen in
> log file.
>> On 6/9/15 8:59 AM, Chao Wang wrote:
>>> Hi all,
>>>
>>> The Wildfly start-up as service scripts wildfly-init-redhat.sh and
>>> wildfly-init-debian.sh currently depend on a grep action of key message
>>> 'WFLYSRV0025:' in console log to determinate whether service start is
>>> successful. The log message indication is accurate, however, it's not
>>> that robust since user can always remove console handler from logging
>>> subsystem. I have opened a WFCORE enhancement jira
>>> https://issues.jboss.org/browse/WFCORE-747  for it.
>>>
>>> For the moment, I have tried three options, they're all not that perfect
>>> to implement
>>>
>>> 1. Stay with exact log message, users need to define their jboss log
>>> directory such as $JBOSS_HOME/standalone/log/server.log for standalone
>>> and $JBOSS_HOME/domain/log/host-controller.log for domain instead of
>>> searching in console log. This is more like another workaround since it
>>> is also volatile once we update log message in future release.(EAP has
>>> 'JBAS015874:')
>>>
>>> 2. Use service pid, this is not precise because a long start-up can
>>> crash in the last second. It needs to wait a suitable seconds before
>>> checking pid existence. and still it can not avoid fake success in rare
>>> case just before timeout.
>>>
>>> 3. Use read-attribute server-state through CLI connection as I did in
>>> Pull Request on Jira. This is declined as it is possible that
>>> authentication is required before connection. In such case, any  non
>>> encrypted password is not advised in configuration files.
>>>
>>> Therefore, I would like to listen for your opinions for them. Any other
>>> suggestion is certainly welcomed in mail or on jira.
>>>
>>> Best regards,
>>>
>>> Chao
>>>
>>>
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>
>>
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1224170
>
> --
> Chao Wang
> Software Engineer
> JBoss by Red Hat
>
>
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>


--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

Jorge Solórzano
It has been considered the use of Java Service Wrapper[1]? it depends on native binaries but almost all platforms are supported.


Jorge Solórzano
http://www.jorsol.com

On Fri, Jun 12, 2015 at 8:27 AM, Brian Stansberry <[hidden email]> wrote:
How does EAP 5 deal with this? IIRC disabling console logging is not
something new in EAP 6.

On 6/11/15 6:29 PM, Chao Wang wrote:
> Thanks for your reply. Please see in-line below
>
> On 06/11/2015 12:44 AM, Brian Stansberry wrote:
>> A couple thoughts:
>>
>> 1) Looking at wildfly-init-redhat.sh at least, I don't see how that
>> check is actually testing for successful startup. It looks like it's
>> just trying to delay start() returning for a while, max 30 secs.
>>
>> So, what purpose is this fulfilling?
> My bad about issue background. It's actually an case in EAP 6 (not yet
> in wildfly). EAP 6.x script has launched state like:
> https://github.com/jbossas/jboss-eap/blob/6.4.x/build/src/main/resources/bin/init.d/jboss-as-standalone.sh#L110
> (not in wildfly's script). Also, console log handler has been removed
> only in EAP full-ha mode long time ago due to performance
> concern.(wildfly keeps it for the moment). This leads to issue in
> bz1224170 <https://bugzilla.redhat.com/show_bug.cgi?id=1224170>.
> That's why I try to seek an better option than current behavior from
> wildfly.
>>
>> 2) How does other software solve this problem? If it's solving a valid
>> problem, it seems like there would be a typical solution.
> I have checked some other application servers, most of them let users
> themselves to write a script to run as service for their OS. Geronimo
> does provide a script, Although I did damage to its configuration file
> to make a fatal error, terminal output still displays "Server started".
> In fact, process does not event exist and detail error can be seen in
> log file.
>> On 6/9/15 8:59 AM, Chao Wang wrote:
>>> Hi all,
>>>
>>> The Wildfly start-up as service scripts wildfly-init-redhat.sh and
>>> wildfly-init-debian.sh currently depend on a grep action of key message
>>> 'WFLYSRV0025:' in console log to determinate whether service start is
>>> successful. The log message indication is accurate, however, it's not
>>> that robust since user can always remove console handler from logging
>>> subsystem. I have opened a WFCORE enhancement jira
>>> https://issues.jboss.org/browse/WFCORE-747  for it.
>>>
>>> For the moment, I have tried three options, they're all not that perfect
>>> to implement
>>>
>>> 1. Stay with exact log message, users need to define their jboss log
>>> directory such as $JBOSS_HOME/standalone/log/server.log for standalone
>>> and $JBOSS_HOME/domain/log/host-controller.log for domain instead of
>>> searching in console log. This is more like another workaround since it
>>> is also volatile once we update log message in future release.(EAP has
>>> 'JBAS015874:')
>>>
>>> 2. Use service pid, this is not precise because a long start-up can
>>> crash in the last second. It needs to wait a suitable seconds before
>>> checking pid existence. and still it can not avoid fake success in rare
>>> case just before timeout.
>>>
>>> 3. Use read-attribute server-state through CLI connection as I did in
>>> Pull Request on Jira. This is declined as it is possible that
>>> authentication is required before connection. In such case, any  non
>>> encrypted password is not advised in configuration files.
>>>
>>> Therefore, I would like to listen for your opinions for them. Any other
>>> suggestion is certainly welcomed in mail or on jira.
>>>
>>> Best regards,
>>>
>>> Chao
>>>
>>>
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>
>>
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1224170
>
> --
> Chao Wang
> Software Engineer
> JBoss by Red Hat
>
>
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>


--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

jtgreene
Administrator
That just monitors console output.

On Jun 12, 2015, at 11:32 AM, Jorge Solórzano <[hidden email]> wrote:

It has been considered the use of Java Service Wrapper[1]? it depends on native binaries but almost all platforms are supported.


Jorge Solórzano
http://www.jorsol.com

On Fri, Jun 12, 2015 at 8:27 AM, Brian Stansberry <[hidden email]> wrote:
How does EAP 5 deal with this? IIRC disabling console logging is not
something new in EAP 6.

On 6/11/15 6:29 PM, Chao Wang wrote:
> Thanks for your reply. Please see in-line below
>
> On 06/11/2015 12:44 AM, Brian Stansberry wrote:
>> A couple thoughts:
>>
>> 1) Looking at wildfly-init-redhat.sh at least, I don't see how that
>> check is actually testing for successful startup. It looks like it's
>> just trying to delay start() returning for a while, max 30 secs.
>>
>> So, what purpose is this fulfilling?
> My bad about issue background. It's actually an case in EAP 6 (not yet
> in wildfly). EAP 6.x script has launched state like:
> https://github.com/jbossas/jboss-eap/blob/6.4.x/build/src/main/resources/bin/init.d/jboss-as-standalone.sh#L110
> (not in wildfly's script). Also, console log handler has been removed
> only in EAP full-ha mode long time ago due to performance
> concern.(wildfly keeps it for the moment). This leads to issue in
> bz1224170 <https://bugzilla.redhat.com/show_bug.cgi?id=1224170>.
> That's why I try to seek an better option than current behavior from
> wildfly.
>>
>> 2) How does other software solve this problem? If it's solving a valid
>> problem, it seems like there would be a typical solution.
> I have checked some other application servers, most of them let users
> themselves to write a script to run as service for their OS. Geronimo
> does provide a script, Although I did damage to its configuration file
> to make a fatal error, terminal output still displays "Server started".
> In fact, process does not event exist and detail error can be seen in
> log file.
>> On 6/9/15 8:59 AM, Chao Wang wrote:
>>> Hi all,
>>>
>>> The Wildfly start-up as service scripts wildfly-init-redhat.sh and
>>> wildfly-init-debian.sh currently depend on a grep action of key message
>>> 'WFLYSRV0025:' in console log to determinate whether service start is
>>> successful. The log message indication is accurate, however, it's not
>>> that robust since user can always remove console handler from logging
>>> subsystem. I have opened a WFCORE enhancement jira
>>> https://issues.jboss.org/browse/WFCORE-747  for it.
>>>
>>> For the moment, I have tried three options, they're all not that perfect
>>> to implement
>>>
>>> 1. Stay with exact log message, users need to define their jboss log
>>> directory such as $JBOSS_HOME/standalone/log/server.log for standalone
>>> and $JBOSS_HOME/domain/log/host-controller.log for domain instead of
>>> searching in console log. This is more like another workaround since it
>>> is also volatile once we update log message in future release.(EAP has
>>> 'JBAS015874:')
>>>
>>> 2. Use service pid, this is not precise because a long start-up can
>>> crash in the last second. It needs to wait a suitable seconds before
>>> checking pid existence. and still it can not avoid fake success in rare
>>> case just before timeout.
>>>
>>> 3. Use read-attribute server-state through CLI connection as I did in
>>> Pull Request on Jira. This is declined as it is possible that
>>> authentication is required before connection. In such case, any  non
>>> encrypted password is not advised in configuration files.
>>>
>>> Therefore, I would like to listen for your opinions for them. Any other
>>> suggestion is certainly welcomed in mail or on jira.
>>>
>>> Best regards,
>>>
>>> Chao
>>>
>>>
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>
>>
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1224170
>
> --
> Chao Wang
> Software Engineer
> JBoss by Red Hat
>
>
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>


--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

Jorge Solórzano

On Fri, Jun 12, 2015 at 10:51 AM, Jason Greene <[hidden email]> wrote:
That just monitors console output.

​Not exacly, there are various modes of integration, posibly the most complex but flexible aproach is to use a listener[1] so it can be adapted properly to what we want.

[1] http://wrapper.tanukisoftware.com/doc/english/integrate-listener.html


 
On Jun 12, 2015, at 11:32 AM, Jorge Solórzano <[hidden email]> wrote:

It has been considered the use of Java Service Wrapper[1]? it depends on native binaries but almost all platforms are supported.


Jorge Solórzano
http://www.jorsol.com

On Fri, Jun 12, 2015 at 8:27 AM, Brian Stansberry <[hidden email]> wrote:
How does EAP 5 deal with this? IIRC disabling console logging is not
something new in EAP 6.

On 6/11/15 6:29 PM, Chao Wang wrote:
> Thanks for your reply. Please see in-line below
>
> On 06/11/2015 12:44 AM, Brian Stansberry wrote:
>> A couple thoughts:
>>
>> 1) Looking at wildfly-init-redhat.sh at least, I don't see how that
>> check is actually testing for successful startup. It looks like it's
>> just trying to delay start() returning for a while, max 30 secs.
>>
>> So, what purpose is this fulfilling?
> My bad about issue background. It's actually an case in EAP 6 (not yet
> in wildfly). EAP 6.x script has launched state like:
> https://github.com/jbossas/jboss-eap/blob/6.4.x/build/src/main/resources/bin/init.d/jboss-as-standalone.sh#L110
> (not in wildfly's script). Also, console log handler has been removed
> only in EAP full-ha mode long time ago due to performance
> concern.(wildfly keeps it for the moment). This leads to issue in
> bz1224170 <https://bugzilla.redhat.com/show_bug.cgi?id=1224170>.
> That's why I try to seek an better option than current behavior from
> wildfly.
>>
>> 2) How does other software solve this problem? If it's solving a valid
>> problem, it seems like there would be a typical solution.
> I have checked some other application servers, most of them let users
> themselves to write a script to run as service for their OS. Geronimo
> does provide a script, Although I did damage to its configuration file
> to make a fatal error, terminal output still displays "Server started".
> In fact, process does not event exist and detail error can be seen in
> log file.
>> On 6/9/15 8:59 AM, Chao Wang wrote:
>>> Hi all,
>>>
>>> The Wildfly start-up as service scripts wildfly-init-redhat.sh and
>>> wildfly-init-debian.sh currently depend on a grep action of key message
>>> 'WFLYSRV0025:' in console log to determinate whether service start is
>>> successful. The log message indication is accurate, however, it's not
>>> that robust since user can always remove console handler from logging
>>> subsystem. I have opened a WFCORE enhancement jira
>>> https://issues.jboss.org/browse/WFCORE-747  for it.
>>>
>>> For the moment, I have tried three options, they're all not that perfect
>>> to implement
>>>
>>> 1. Stay with exact log message, users need to define their jboss log
>>> directory such as $JBOSS_HOME/standalone/log/server.log for standalone
>>> and $JBOSS_HOME/domain/log/host-controller.log for domain instead of
>>> searching in console log. This is more like another workaround since it
>>> is also volatile once we update log message in future release.(EAP has
>>> 'JBAS015874:')
>>>
>>> 2. Use service pid, this is not precise because a long start-up can
>>> crash in the last second. It needs to wait a suitable seconds before
>>> checking pid existence. and still it can not avoid fake success in rare
>>> case just before timeout.
>>>
>>> 3. Use read-attribute server-state through CLI connection as I did in
>>> Pull Request on Jira. This is declined as it is possible that
>>> authentication is required before connection. In such case, any  non
>>> encrypted password is not advised in configuration files.
>>>
>>> Therefore, I would like to listen for your opinions for them. Any other
>>> suggestion is certainly welcomed in mail or on jira.
>>>
>>> Best regards,
>>>
>>> Chao
>>>
>>>
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>
>>
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1224170
>
> --
> Chao Wang
> Software Engineer
> JBoss by Red Hat
>
>
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>


--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

James Perkins
In reply to this post by Jorge Solórzano
This won't work as there's no guarantee there will be a server.log and/or that it would live in log directory. The best solution involves not reading logs at all. I don't have any great ideas on the best way to currently do that.

On 06/11/2015 10:25 PM, Jorge Solórzano wrote:
I think there is no easy fix, we can assume for example that after 3 seconds that the PID exists then the process is up and return OK, but if the process stops latter, it will be a fake start.

I know that we should not rely on read the log as James stated, but until we can figure out a better aproach this is my proposal, I can modify wildfly-init-debian.sh to have this behavior

1. Add to logging subsystem to standalone*.xml:
            <logger category="org.jboss.as">
                <level name="INFO"/>
            </logger>

2. Remove JBOSS_CONSOLE_LOG and read JBOSS_LOG_DIR

3. If standalone read "$JBOSS_LOG_DIR/server.log"
 if domain read "$JBOSS_LOG_DIR/host-controller.log"
to find WFLYSRV0025 or the string used for EAP... this is probably volatile but it's not changed frequently, in fact it has been changed only once.

4. Always check the status of pid (if dies return immediately).

5. If server.log is not found, deactivated or not print the correct output there are two choices:
a. On wait timeout, send a warning about an unknow status (this is how works right now).
b. On wait timeout, just check if PID is still up and return OK, otherwise FAILURE.

I believe this approach avoids the problem of console log...

Best regards,


Jorge Solórzano
http://www.jorsol.com

On Thu, Jun 11, 2015 at 5:29 PM, Chao Wang <[hidden email]> wrote:
Thanks for your reply. Please see in-line below

On 06/11/2015 12:44 AM, Brian Stansberry wrote:
A couple thoughts:

1) Looking at wildfly-init-redhat.sh at least, I don't see how that 
check is actually testing for successful startup. It looks like it's 
just trying to delay start() returning for a while, max 30 secs.

So, what purpose is this fulfilling?
My bad about issue background. It's actually an case in EAP 6 (not yet in wildfly). EAP 6.x script has launched state like: https://github.com/jbossas/jboss-eap/blob/6.4.x/build/src/main/resources/bin/init.d/jboss-as-standalone.sh#L110 (not in wildfly's script). Also, console log handler has been removed only in EAP full-ha mode long time ago due to performance concern.(wildfly keeps it for the moment). This leads to issue in bz1224170.
That's why I try to seek an better option than current behavior from wildfly.
2) How does other software solve this problem? If it's solving a valid 
problem, it seems like there would be a typical solution.
I have checked some other application servers, most of them let users themselves to write a script to run as service for their OS. Geronimo does provide a script, Although I did damage to its configuration file to make a fatal error, terminal output still displays "Server started". In fact, process does not event exist and detail error can be seen in log file.
On 6/9/15 8:59 AM, Chao Wang wrote:
Hi all,

The Wildfly start-up as service scripts wildfly-init-redhat.sh and
wildfly-init-debian.sh currently depend on a grep action of key message
'WFLYSRV0025:' in console log to determinate whether service start is
successful. The log message indication is accurate, however, it's not
that robust since user can always remove console handler from logging
subsystem. I have opened a WFCORE enhancement jira
https://issues.jboss.org/browse/WFCORE-747 for it.

For the moment, I have tried three options, they're all not that perfect
to implement

1. Stay with exact log message, users need to define their jboss log
directory such as $JBOSS_HOME/standalone/log/server.log for standalone
and $JBOSS_HOME/domain/log/host-controller.log for domain instead of
searching in console log. This is more like another workaround since it
is also volatile once we update log message in future release.(EAP has
'JBAS015874:')

2. Use service pid, this is not precise because a long start-up can
crash in the last second. It needs to wait a suitable seconds before
checking pid existence. and still it can not avoid fake success in rare
case just before timeout.

3. Use read-attribute server-state through CLI connection as I did in
Pull Request on Jira. This is declined as it is possible that
authentication is required before connection. In such case, any  non
encrypted password is not advised in configuration files.

Therefore, I would like to listen for your opinions for them. Any other
suggestion is certainly welcomed in mail or on jira.
​​
Best regards, Chao _______________________________________________ wildfly-dev mailing list [hidden email] https://lists.jboss.org/mailman/listinfo/wildfly-dev

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1224170
-- 
Chao Wang
Software Engineer
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

-- 
James R. Perkins
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

Jorge Solórzano

On Fri, Jun 12, 2015 at 11:32 AM, James R. Perkins <[hidden email]> wrote:
This won't work as there's no guarantee there will be a server.log and/or that it would live in log directory. The best solution involves not reading logs at all. I don't have any great ideas on the best way to currently do that.


​James, that why in step 5 i mention 2 aproach, if timeout return a warning, or if timeout check if pid is still up and return ok. It's not perfect since it return control until timeout, but it's more reliable that depend on console output.

 
On 06/11/2015 10:25 PM, Jorge Solórzano wrote:
I think there is no easy fix, we can assume for example that after 3 seconds that the PID exists then the process is up and return OK, but if the process stops latter, it will be a fake start.

I know that we should not rely on read the log as James stated, but until we can figure out a better aproach this is my proposal, I can modify wildfly-init-debian.sh to have this behavior

1. Add to logging subsystem to standalone*.xml:
            <logger category="org.jboss.as">
                <level name="INFO"/>
            </logger>

2. Remove JBOSS_CONSOLE_LOG and read JBOSS_LOG_DIR

3. If standalone read "$JBOSS_LOG_DIR/server.log"
 if domain read "$JBOSS_LOG_DIR/host-controller.log"
to find WFLYSRV0025 or the string used for EAP... this is probably volatile but it's not changed frequently, in fact it has been changed only once.

4. Always check the status of pid (if dies return immediately).

5. If server.log is not found, deactivated or not print the correct output there are two choices:
a. On wait timeout, send a warning about an unknow status (this is how works right now).
b. On wait timeout, just check if PID is still up and return OK, otherwise FAILURE.

I believe this approach avoids the problem of console log...

Best regards,


Jorge Solórzano
http://www.jorsol.com

On Thu, Jun 11, 2015 at 5:29 PM, Chao Wang <[hidden email]> wrote:
Thanks for your reply. Please see in-line below

On 06/11/2015 12:44 AM, Brian Stansberry wrote:
A couple thoughts:

1) Looking at wildfly-init-redhat.sh at least, I don't see how that 
check is actually testing for successful startup. It looks like it's 
just trying to delay start() returning for a while, max 30 secs.

So, what purpose is this fulfilling?
My bad about issue background. It's actually an case in EAP 6 (not yet in wildfly). EAP 6.x script has launched state like: https://github.com/jbossas/jboss-eap/blob/6.4.x/build/src/main/resources/bin/init.d/jboss-as-standalone.sh#L110 (not in wildfly's script). Also, console log handler has been removed only in EAP full-ha mode long time ago due to performance concern.(wildfly keeps it for the moment). This leads to issue in bz1224170.
That's why I try to seek an better option than current behavior from wildfly.
2) How does other software solve this problem? If it's solving a valid 
problem, it seems like there would be a typical solution.
I have checked some other application servers, most of them let users themselves to write a script to run as service for their OS. Geronimo does provide a script, Although I did damage to its configuration file to make a fatal error, terminal output still displays "Server started". In fact, process does not event exist and detail error can be seen in log file.
On 6/9/15 8:59 AM, Chao Wang wrote:
Hi all,

The Wildfly start-up as service scripts wildfly-init-redhat.sh and
wildfly-init-debian.sh currently depend on a grep action of key message
'WFLYSRV0025:' in console log to determinate whether service start is
successful. The log message indication is accurate, however, it's not
that robust since user can always remove console handler from logging
subsystem. I have opened a WFCORE enhancement jira
https://issues.jboss.org/browse/WFCORE-747 for it.

For the moment, I have tried three options, they're all not that perfect
to implement

1. Stay with exact log message, users need to define their jboss log
directory such as $JBOSS_HOME/standalone/log/server.log for standalone
and $JBOSS_HOME/domain/log/host-controller.log for domain instead of
searching in console log. This is more like another workaround since it
is also volatile once we update log message in future release.(EAP has
'JBAS015874:')

2. Use service pid, this is not precise because a long start-up can
crash in the last second. It needs to wait a suitable seconds before
checking pid existence. and still it can not avoid fake success in rare
case just before timeout.

3. Use read-attribute server-state through CLI connection as I did in
Pull Request on Jira. This is declined as it is possible that
authentication is required before connection. In such case, any  non
encrypted password is not advised in configuration files.

Therefore, I would like to listen for your opinions for them. Any other
suggestion is certainly welcomed in mail or on jira.
​​
Best regards, Chao _______________________________________________ wildfly-dev mailing list [hidden email] https://lists.jboss.org/mailman/listinfo/wildfly-dev

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1224170
-- 
Chao Wang
Software Engineer
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

-- 
James R. Perkins
JBoss by Red Hat


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

jtgreene
Administrator
In reply to this post by Jorge Solórzano
We wouldn’t be able to use that approach because its shipped under an incompatible viral license (GPL with additional restrictions). Theres also the other negative that we try to avoid shipping native bits if at all possible. 

On Jun 12, 2015, at 12:02 PM, Jorge Solórzano <[hidden email]> wrote:


On Fri, Jun 12, 2015 at 10:51 AM, Jason Greene <[hidden email]> wrote:
That just monitors console output.

​Not exacly, there are various modes of integration, posibly the most complex but flexible aproach is to use a listener[1] so it can be adapted properly to what we want.

[1] http://wrapper.tanukisoftware.com/doc/english/integrate-listener.html


 
On Jun 12, 2015, at 11:32 AM, Jorge Solórzano <[hidden email]> wrote:

It has been considered the use of Java Service Wrapper[1]? it depends on native binaries but almost all platforms are supported.


Jorge Solórzano
http://www.jorsol.com

On Fri, Jun 12, 2015 at 8:27 AM, Brian Stansberry <[hidden email]> wrote:
How does EAP 5 deal with this? IIRC disabling console logging is not
something new in EAP 6.

On 6/11/15 6:29 PM, Chao Wang wrote:
> Thanks for your reply. Please see in-line below
>
> On 06/11/2015 12:44 AM, Brian Stansberry wrote:
>> A couple thoughts:
>>
>> 1) Looking at wildfly-init-redhat.sh at least, I don't see how that
>> check is actually testing for successful startup. It looks like it's
>> just trying to delay start() returning for a while, max 30 secs.
>>
>> So, what purpose is this fulfilling?
> My bad about issue background. It's actually an case in EAP 6 (not yet
> in wildfly). EAP 6.x script has launched state like:
> https://github.com/jbossas/jboss-eap/blob/6.4.x/build/src/main/resources/bin/init.d/jboss-as-standalone.sh#L110
> (not in wildfly's script). Also, console log handler has been removed
> only in EAP full-ha mode long time ago due to performance
> concern.(wildfly keeps it for the moment). This leads to issue in
> bz1224170 <https://bugzilla.redhat.com/show_bug.cgi?id=1224170>.
> That's why I try to seek an better option than current behavior from
> wildfly.
>>
>> 2) How does other software solve this problem? If it's solving a valid
>> problem, it seems like there would be a typical solution.
> I have checked some other application servers, most of them let users
> themselves to write a script to run as service for their OS. Geronimo
> does provide a script, Although I did damage to its configuration file
> to make a fatal error, terminal output still displays "Server started".
> In fact, process does not event exist and detail error can be seen in
> log file.
>> On 6/9/15 8:59 AM, Chao Wang wrote:
>>> Hi all,
>>>
>>> The Wildfly start-up as service scripts wildfly-init-redhat.sh and
>>> wildfly-init-debian.sh currently depend on a grep action of key message
>>> 'WFLYSRV0025:' in console log to determinate whether service start is
>>> successful. The log message indication is accurate, however, it's not
>>> that robust since user can always remove console handler from logging
>>> subsystem. I have opened a WFCORE enhancement jira
>>> https://issues.jboss.org/browse/WFCORE-747  for it.
>>>
>>> For the moment, I have tried three options, they're all not that perfect
>>> to implement
>>>
>>> 1. Stay with exact log message, users need to define their jboss log
>>> directory such as $JBOSS_HOME/standalone/log/server.log for standalone
>>> and $JBOSS_HOME/domain/log/host-controller.log for domain instead of
>>> searching in console log. This is more like another workaround since it
>>> is also volatile once we update log message in future release.(EAP has
>>> 'JBAS015874:')
>>>
>>> 2. Use service pid, this is not precise because a long start-up can
>>> crash in the last second. It needs to wait a suitable seconds before
>>> checking pid existence. and still it can not avoid fake success in rare
>>> case just before timeout.
>>>
>>> 3. Use read-attribute server-state through CLI connection as I did in
>>> Pull Request on Jira. This is declined as it is possible that
>>> authentication is required before connection. In such case, any  non
>>> encrypted password is not advised in configuration files.
>>>
>>> Therefore, I would like to listen for your opinions for them. Any other
>>> suggestion is certainly welcomed in mail or on jira.
>>>
>>> Best regards,
>>>
>>> Chao
>>>
>>>
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>
>>
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1224170
>
> --
> Chao Wang
> Software Engineer
> JBoss by Red Hat
>
>
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>


--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat



--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

Jorge Solórzano
On Fri, Jun 12, 2015 at 1:40 PM, Jason Greene <[hidden email]> wrote:
We wouldn’t be able to use that approach because its shipped under an incompatible viral license (GPL with additional restrictions). Theres also the other negative that we try to avoid shipping native bits if at all possible. 

That a good reason​ to avoid that...

And what about Commons Daemon?, it stills depends on the native part, but wildfly already includes native binaries for windows, a separate download of the native part could be an option for both unix and windows...

 
On Jun 12, 2015, at 12:02 PM, Jorge Solórzano <[hidden email]> wrote:


On Fri, Jun 12, 2015 at 10:51 AM, Jason Greene <[hidden email]> wrote:
That just monitors console output.

​Not exacly, there are various modes of integration, posibly the most complex but flexible aproach is to use a listener[1] so it can be adapted properly to what we want.

[1] http://wrapper.tanukisoftware.com/doc/english/integrate-listener.html


 
On Jun 12, 2015, at 11:32 AM, Jorge Solórzano <[hidden email]> wrote:

It has been considered the use of Java Service Wrapper[1]? it depends on native binaries but almost all platforms are supported.


Jorge Solórzano
http://www.jorsol.com

On Fri, Jun 12, 2015 at 8:27 AM, Brian Stansberry <[hidden email]> wrote:
How does EAP 5 deal with this? IIRC disabling console logging is not
something new in EAP 6.

On 6/11/15 6:29 PM, Chao Wang wrote:
> Thanks for your reply. Please see in-line below
>
> On 06/11/2015 12:44 AM, Brian Stansberry wrote:
>> A couple thoughts:
>>
>> 1) Looking at wildfly-init-redhat.sh at least, I don't see how that
>> check is actually testing for successful startup. It looks like it's
>> just trying to delay start() returning for a while, max 30 secs.
>>
>> So, what purpose is this fulfilling?
> My bad about issue background. It's actually an case in EAP 6 (not yet
> in wildfly). EAP 6.x script has launched state like:
> https://github.com/jbossas/jboss-eap/blob/6.4.x/build/src/main/resources/bin/init.d/jboss-as-standalone.sh#L110
> (not in wildfly's script). Also, console log handler has been removed
> only in EAP full-ha mode long time ago due to performance
> concern.(wildfly keeps it for the moment). This leads to issue in
> bz1224170 <https://bugzilla.redhat.com/show_bug.cgi?id=1224170>.
> That's why I try to seek an better option than current behavior from
> wildfly.
>>
>> 2) How does other software solve this problem? If it's solving a valid
>> problem, it seems like there would be a typical solution.
> I have checked some other application servers, most of them let users
> themselves to write a script to run as service for their OS. Geronimo
> does provide a script, Although I did damage to its configuration file
> to make a fatal error, terminal output still displays "Server started".
> In fact, process does not event exist and detail error can be seen in
> log file.
>> On 6/9/15 8:59 AM, Chao Wang wrote:
>>> Hi all,
>>>
>>> The Wildfly start-up as service scripts wildfly-init-redhat.sh and
>>> wildfly-init-debian.sh currently depend on a grep action of key message
>>> 'WFLYSRV0025:' in console log to determinate whether service start is
>>> successful. The log message indication is accurate, however, it's not
>>> that robust since user can always remove console handler from logging
>>> subsystem. I have opened a WFCORE enhancement jira
>>> https://issues.jboss.org/browse/WFCORE-747  for it.
>>>
>>> For the moment, I have tried three options, they're all not that perfect
>>> to implement
>>>
>>> 1. Stay with exact log message, users need to define their jboss log
>>> directory such as $JBOSS_HOME/standalone/log/server.log for standalone
>>> and $JBOSS_HOME/domain/log/host-controller.log for domain instead of
>>> searching in console log. This is more like another workaround since it
>>> is also volatile once we update log message in future release.(EAP has
>>> 'JBAS015874:')
>>>
>>> 2. Use service pid, this is not precise because a long start-up can
>>> crash in the last second. It needs to wait a suitable seconds before
>>> checking pid existence. and still it can not avoid fake success in rare
>>> case just before timeout.
>>>
>>> 3. Use read-attribute server-state through CLI connection as I did in
>>> Pull Request on Jira. This is declined as it is possible that
>>> authentication is required before connection. In such case, any  non
>>> encrypted password is not advised in configuration files.
>>>
>>> Therefore, I would like to listen for your opinions for them. Any other
>>> suggestion is certainly welcomed in mail or on jira.
>>>
>>> Best regards,
>>>
>>> Chao
>>>
>>>
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>
>>
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1224170
>
> --
> Chao Wang
> Software Engineer
> JBoss by Red Hat
>
>
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>


--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat



--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

Chao Wang
In reply to this post by Brian Stansberry
On 06/12/2015 10:27 PM, Brian Stansberry wrote:
> How does EAP 5 deal with this? IIRC disabling console logging is not
> something new in EAP 6.
Yes, it also removed console logger for production configuration in EAP 5.
The script is at
https://anonsvn.jboss.org/repos/jbossas/branches/JBPAPP_5/server/src/bin/jboss_init_redhat.sh 
It simply calls start-up script and does not provide any result check as:

eval $JBOSS_CMD_START >${JBOSS_CONSOLE} 2>&1 &


> On 6/11/15 6:29 PM, Chao Wang wrote:
>> Thanks for your reply. Please see in-line below
>>
>> On 06/11/2015 12:44 AM, Brian Stansberry wrote:
>>> A couple thoughts:
>>>
>>> 1) Looking at wildfly-init-redhat.sh at least, I don't see how that
>>> check is actually testing for successful startup. It looks like it's
>>> just trying to delay start() returning for a while, max 30 secs.
>>>
>>> So, what purpose is this fulfilling?
>> My bad about issue background. It's actually an case in EAP 6 (not yet
>> in wildfly). EAP 6.x script has launched state like:
>> https://github.com/jbossas/jboss-eap/blob/6.4.x/build/src/main/resources/bin/init.d/jboss-as-standalone.sh#L110
>> (not in wildfly's script). Also, console log handler has been removed
>> only in EAP full-ha mode long time ago due to performance
>> concern.(wildfly keeps it for the moment). This leads to issue in
>> bz1224170 <https://bugzilla.redhat.com/show_bug.cgi?id=1224170>.
>> That's why I try to seek an better option than current behavior from
>> wildfly.
>>> 2) How does other software solve this problem? If it's solving a valid
>>> problem, it seems like there would be a typical solution.
>> I have checked some other application servers, most of them let users
>> themselves to write a script to run as service for their OS. Geronimo
>> does provide a script, Although I did damage to its configuration file
>> to make a fatal error, terminal output still displays "Server started".
>> In fact, process does not event exist and detail error can be seen in
>> log file.
>>> On 6/9/15 8:59 AM, Chao Wang wrote:
>>>> Hi all,
>>>>
>>>> The Wildfly start-up as service scripts wildfly-init-redhat.sh and
>>>> wildfly-init-debian.sh currently depend on a grep action of key message
>>>> 'WFLYSRV0025:' in console log to determinate whether service start is
>>>> successful. The log message indication is accurate, however, it's not
>>>> that robust since user can always remove console handler from logging
>>>> subsystem. I have opened a WFCORE enhancement jira
>>>> https://issues.jboss.org/browse/WFCORE-747  for it.
>>>>
>>>> For the moment, I have tried three options, they're all not that perfect
>>>> to implement
>>>>
>>>> 1. Stay with exact log message, users need to define their jboss log
>>>> directory such as $JBOSS_HOME/standalone/log/server.log for standalone
>>>> and $JBOSS_HOME/domain/log/host-controller.log for domain instead of
>>>> searching in console log. This is more like another workaround since it
>>>> is also volatile once we update log message in future release.(EAP has
>>>> 'JBAS015874:')
>>>>
>>>> 2. Use service pid, this is not precise because a long start-up can
>>>> crash in the last second. It needs to wait a suitable seconds before
>>>> checking pid existence. and still it can not avoid fake success in rare
>>>> case just before timeout.
>>>>
>>>> 3. Use read-attribute server-state through CLI connection as I did in
>>>> Pull Request on Jira. This is declined as it is possible that
>>>> authentication is required before connection. In such case, any  non
>>>> encrypted password is not advised in configuration files.
>>>>
>>>> Therefore, I would like to listen for your opinions for them. Any other
>>>> suggestion is certainly welcomed in mail or on jira.
>>>>
>>>> Best regards,
>>>>
>>>> Chao
>>>>
>>>>
>>>> _______________________________________________
>>>> wildfly-dev mailing list
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>>
>> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1224170
>>
>> --
>> Chao Wang
>> Software Engineer
>> JBoss by Red Hat
>>
>>
>>
>> _______________________________________________
>> wildfly-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>
>


--
Chao Wang
Software Engineer
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

Chao Wang
In reply to this post by Chao Wang


On 06/09/2015 09:59 PM, Chao Wang wrote:
Hi all,

The Wildfly start-up as service scripts wildfly-init-redhat.sh and wildfly-init-debian.sh currently depend on a grep action of key message 'WFLYSRV0025:' in console log to determinate whether service start is successful. The log message indication is accurate, however, it's not that robust since user can always remove console handler from logging subsystem. I have opened a WFCORE enhancement jira https://issues.jboss.org/browse/WFCORE-747 for it.

For the moment, I have tried three options, they're all not that perfect to implement

1. Stay with exact log message, users need to define their jboss log directory such as $JBOSS_HOME/standalone/log/server.log for standalone and $JBOSS_HOME/domain/log/host-controller.log for domain instead of searching in console log. This is more like another workaround since it is also
volatile once we update log message in future release.(EAP has 'JBAS015874:')

2. Use service pid, this is not precise because a long start-up can crash in the last second. It needs to wait a suitable seconds before checking pid existence. and still it can not avoid fake success in rare case just before timeout.

3. Use read-attribute server-state through CLI connection as I did in Pull Request on Jira. This is declined as it is possible that authentication is required before connection. In such case, any  non encrypted password is not advised in configuration files.

I have opened a PR to change the log key message dependency in start script https://github.com/wildfly/wildfly-core/pull/859

A jboss start marker file in temporary directory can be added to record launch result and its timestamp when it starts. wildfly-init-redhat.sh / wildfly-init-debian.sh will exam its content to get server start result (success/error, or nonexistent file means start failure). The timestamp can be used to identify this marker file's age in case of previous abnormal termination. Once server is normally shutdown, it can be removed as part of service stop.

If anyone is interested in this, please take a look at the PR, and feel free to reply this or leave a comment.

Thanks,


Therefore, I would like to listen for your opinions for them. Any other suggestion is certainly welcomed in mail or on jira.

Best regards,

Chao


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

-- 
Chao Wang
Software Engineer
JBoss by Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

Carlo de Wolf
In reply to this post by David Lloyd-2
On 06/12/2015 02:57 PM, David M. Lloyd wrote:

> This is yet another good use case for an idea I proposed at the last
> couple developer meeting: the idea of configurable "availability"
> services, where you add a configuration which says "when these
> components and/or configured services are available, perform this
> action" where the action might be to notify a load balancer, perform a
> notification to humans, or even drop a file to the filesystem (which
> would be directly useful to this use case).
>
> Note (in case someone thinks this is a great idea and runs out to
> implement it right away) that when I say "configured services" I do not
> mean MSC service names; more like management capabilities where you have
> a constrained namespace and each subsystem can contribute services to
> this category.

Wouldn't implementing a MSC service with dependencies be just the quick
fix that covers the use case? :-)

Carlo

>
> Also note that Java 9 adds limited support for signalling unrelated
> processes, though at the moment on UNIX the signals are basically
> limited to TERM and KILL.
>
> On 06/12/2015 07:06 AM, Jason T. Greene wrote:
>>> On Jun 10, 2015, at 11:46 AM, Brian Stansberry <[hidden email]> wrote:
>>>
>>> So, what purpose is this fulfilling?
>>>
>>> 2) How does other software solve this problem? If it's solving a valid
>>> problem, it seems like there would be a typical solution.
>> The classic UNIX solution is that the daemon forks and returns, dropping a PID file of its child to disk, after it is done initializing and exits with an error code when there is a problem.
>>
>> Systemd added other approaches, where a daemon can signal systemd directly, or it can use dbus to send a message.
>>
>> The former can't be done efficiently in Java because it doesn't have a pure fork(), only an exec. Although it would be possible to emulate with an exec with an unacceptable hit to boot time. The latter options are too Linux specific.
>>
>> I think the best solution would be for us to add a signaling mechanism specifically for this purpose. We could use sun.misc.Signal (potentially an issue on Java 9), or we could exec the kill command to signal a process.
>>
>> We could also use a specially status file (e.g standalone.sh --start-status-file=blah) for a script to monitor.
>>
>> Thoughts?
>> _______________________________________________
>> wildfly-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

Jorge Solórzano
In reply to this post by Chao Wang
I don't like that deleteStartupMarker() is repeated on HostControllerService.java and ApplicationServerService.java

The marker file should be just "startup-marker", or probably "wildfly-startup-marker".

And if we go that road why not add another line with "message: Error message", that way it can be parsed by startup scripts and print the error directly.


On Sun, Jul 12, 2015 at 11:55 PM Chao Wang <[hidden email]> wrote:


On 06/09/2015 09:59 PM, Chao Wang wrote:
Hi all,

The Wildfly start-up as service scripts wildfly-init-redhat.sh and wildfly-init-debian.sh currently depend on a grep action of key message 'WFLYSRV0025:' in console log to determinate whether service start is successful. The log message indication is accurate, however, it's not that robust since user can always remove console handler from logging subsystem. I have opened a WFCORE enhancement jira https://issues.jboss.org/browse/WFCORE-747 for it.

For the moment, I have tried three options, they're all not that perfect to implement

1. Stay with exact log message, users need to define their jboss log directory such as $JBOSS_HOME/standalone/log/server.log for standalone and $JBOSS_HOME/domain/log/host-controller.log for domain instead of searching in console log. This is more like another workaround since it is also
volatile once we update log message in future release.(EAP has 'JBAS015874:')

2. Use service pid, this is not precise because a long start-up can crash in the last second. It needs to wait a suitable seconds before checking pid existence. and still it can not avoid fake success in rare case just before timeout.

3. Use read-attribute server-state through CLI connection as I did in Pull Request on Jira. This is declined as it is possible that authentication is required before connection. In such case, any  non encrypted password is not advised in configuration files.

I have opened a PR to change the log key message dependency in start script https://github.com/wildfly/wildfly-core/pull/859

A jboss start marker file in temporary directory can be added to record launch result and its timestamp when it starts. wildfly-init-redhat.sh / wildfly-init-debian.sh will exam its content to get server start result (success/error, or nonexistent file means start failure). The timestamp can be used to identify this marker file's age in case of previous abnormal termination. Once server is normally shutdown, it can be removed as part of service stop.

If anyone is interested in this, please take a look at the PR, and feel free to reply this or leave a comment.

Thanks,


Therefore, I would like to listen for your opinions for them. Any other suggestion is certainly welcomed in mail or on jira.

Best regards,

Chao


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

-- 
Chao Wang
Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Wildfly start-up as service script depends on console log to determinate start result

David Lloyd-2
In reply to this post by Carlo de Wolf
On 07/13/2015 03:06 AM, Carlo de Wolf wrote:

> On 06/12/2015 02:57 PM, David M. Lloyd wrote:
>> This is yet another good use case for an idea I proposed at the last
>> couple developer meeting: the idea of configurable "availability"
>> services, where you add a configuration which says "when these
>> components and/or configured services are available, perform this
>> action" where the action might be to notify a load balancer, perform a
>> notification to humans, or even drop a file to the filesystem (which
>> would be directly useful to this use case).
>>
>> Note (in case someone thinks this is a great idea and runs out to
>> implement it right away) that when I say "configured services" I do not
>> mean MSC service names; more like management capabilities where you have
>> a constrained namespace and each subsystem can contribute services to
>> this category.
>
> Wouldn't implementing a MSC service with dependencies be just the quick
> fix that covers the use case? :-)

Of course; however there aren't MSC services that represent services
like entire WARs, and also, MSC service names are not API so if and when
we change service names for e.g. servlets in future versions, things
will break.  The idea is to establish the function in terms of EE app,
module, and component names, and subsystem/model-specific names rather
than service names (which are really an implementation detail).

For a particular example, it's very likely that we will change the way
services are assigned to EJBs.

>
> Carlo
>
>>
>> Also note that Java 9 adds limited support for signalling unrelated
>> processes, though at the moment on UNIX the signals are basically
>> limited to TERM and KILL.
>>
>> On 06/12/2015 07:06 AM, Jason T. Greene wrote:
>>>> On Jun 10, 2015, at 11:46 AM, Brian Stansberry
>>>> <[hidden email]> wrote:
>>>>
>>>> So, what purpose is this fulfilling?
>>>>
>>>> 2) How does other software solve this problem? If it's solving a valid
>>>> problem, it seems like there would be a typical solution.
>>> The classic UNIX solution is that the daemon forks and returns,
>>> dropping a PID file of its child to disk, after it is done
>>> initializing and exits with an error code when there is a problem.
>>>
>>> Systemd added other approaches, where a daemon can signal systemd
>>> directly, or it can use dbus to send a message.
>>>
>>> The former can't be done efficiently in Java because it doesn't have
>>> a pure fork(), only an exec. Although it would be possible to emulate
>>> with an exec with an unacceptable hit to boot time. The latter
>>> options are too Linux specific.
>>>
>>> I think the best solution would be for us to add a signaling
>>> mechanism specifically for this purpose. We could use sun.misc.Signal
>>> (potentially an issue on Java 9), or we could exec the kill command
>>> to signal a process.
>>>
>>> We could also use a specially status file (e.g standalone.sh
>>> --start-status-file=blah) for a script to monitor.
>>>
>>> Thoughts?
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>
>

--
- DML
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev