Using Wildfly as a load balancer

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Using Wildfly as a load balancer

Stuart Douglas
Hi everyone,

A while ago we added support to Wildfly to allow it to be used as a front end mod_cluster based load balancer. 

I am going to blog about this once it appears in a released version, but for now if anyone wants to try it out I have an example in my github at https://github.com/stuartwdouglas/modcluster-example 

The example basically contains the CLI commands need to start a domain with two backend servers and a front end load balancer (with a simple deployment that prints the server name that handles the request, and can start counting requests to demonstrate sticky sessions and failover), although the deployment path and local IP address will need to be modified appropriately. 

I'm just posting about this here in case it is interesting to anyone, as it should provide a very simple way to get started with clustering. 

Stuart

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

Jorge Solórzano

Hi Stuart,

This means is no longer necessary to use Apache httpd as front end? What should be the pros and cont. when used like this?

El ene 12, 2015 8:48 PM, "Stuart Douglas" <[hidden email]> escribió:
Hi everyone,

A while ago we added support to Wildfly to allow it to be used as a front end mod_cluster based load balancer. 

I am going to blog about this once it appears in a released version, but for now if anyone wants to try it out I have an example in my github at https://github.com/stuartwdouglas/modcluster-example 

The example basically contains the CLI commands need to start a domain with two backend servers and a front end load balancer (with a simple deployment that prints the server name that handles the request, and can start counting requests to demonstrate sticky sessions and failover), although the deployment path and local IP address will need to be modified appropriately. 

I'm just posting about this here in case it is interesting to anyone, as it should provide a very simple way to get started with clustering. 

Stuart

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

Stuart Douglas
Yes, if you use this then it is no longer necessary to use apache as a load balancer. 

There are a few advantages to using Wildfly, namely:

- Front and back end servers are all Wildfly, allowing them to all be managed uniformly through domain mode (which also means pure Java, so no native bits)
- Undertow should be able to perform better than apache as a load balancer (although we don't have any firm benchmarks for this yet), and will be able to use new protocols such as HTTP2 to communicate with backend servers which are more efficient on the wire. 

The down side is of course that apache is already widely deployed, so a lot of organisations will already have experience with it, or be using apache for other things as well. 

Stuart


On Tue Jan 13 2015 at 2:35:26 PM Jorge Solórzano <[hidden email]> wrote:

Hi Stuart,

This means is no longer necessary to use Apache httpd as front end? What should be the pros and cont. when used like this?

El ene 12, 2015 8:48 PM, "Stuart Douglas" <[hidden email]> escribió:
Hi everyone,

A while ago we added support to Wildfly to allow it to be used as a front end mod_cluster based load balancer. 

I am going to blog about this once it appears in a released version, but for now if anyone wants to try it out I have an example in my github at https://github.com/stuartwdouglas/modcluster-example 

The example basically contains the CLI commands need to start a domain with two backend servers and a front end load balancer (with a simple deployment that prints the server name that handles the request, and can start counting requests to demonstrate sticky sessions and failover), although the deployment path and local IP address will need to be modified appropriately. 

I'm just posting about this here in case it is interesting to anyone, as it should provide a very simple way to get started with clustering. 

Stuart

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

denstar
On 01/12/2015 09:55 PM, Stuart Douglas wrote:
> Yes, if you use this then it is no longer necessary to use apache as a load
> balancer.

This is fantastic news Stuart!

I was just about to roll out something that bundles apache -- mostly for
mod_cluster support, and URL rewriting -- and I'd prefer to go this
route (ha!).

I was going to ask about the latter-- that is, rewriting, but saw this:
 https://developer.jboss.org/thread/236258

Which I might give a go at, vs. tuckey... tho UrlRewriteFilter has some
stuff for being able to consume apache-style rewrite rules, which kinda
eases the minds of some potential transitioners, as it were.  I haven't
checked it in a while however, and it only covered a subset back then,
which I kinda doubt has changed, so... *shrug*

Anyways, I just wanted to take a second and give some kudos, and let you
all know some random person is excited about this.  Good timing!

:Denny

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

James Livingston
In reply to this post by Stuart Douglas
On Tue, 2015-01-13 at 04:55 +0000, Stuart Douglas wrote:
> - Front and back end servers are all Wildfly, allowing them to all be
> managed uniformly through domain mode (which also means pure Java, so no
> native bits)

Which also means you can use all Java tooling you know to work on
problems, rather than C tooling which may be less familiar (and in some
cases more difficult to do or less useful as well).

Being able to use ByteMan to investigate what's happening in the load
balancer rather than re-compiling a C module to add extra logging will
be nice :)

--
James "Doc" Livingston
JBoss Support Engineering Group
Red Hat

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

denstar
On 01/13/2015 04:47 PM, James Livingston wrote:
...
> Being able to use ByteMan to investigate what's happening in the load
> balancer rather than re-compiling a C module to add extra logging will
> be nice :)
>

For sure!  Limitless possibilities.  :)

What would be the best way of restarting the load balancer itself?

Maybe suggest say, a 2 node front-end, with X back-end nodes?

:Denny
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

filipepferraz
In reply to this post by Stuart Douglas
Stuart, what version of wildfly can be used for test the cluster?
I tried in 9.0.0.Alpha2-SNAPSHOT but don't find the mod-cluster filter inside the configuration=filter.
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

Tomaž Cerar-2

On Fri, Jan 16, 2015 at 6:30 PM, filipepferraz <[hidden email]> wrote:
Stuart, what version of wildfly can be used for test the cluster?
I tried in 9.0.0.Alpha2-SNAPSHOT but don't find the mod-cluster filter
inside the configuration=filter.



--
View this message in context: http://wildfly-development.1055759.n5.nabble.com/Using-Wildfly-as-a-load-balancer-tp5715464p5715477.html
Sent from the WildFly Development mailing list archive at Nabble.com.
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

denstar
With something this new you'll probably be wanting to build from source or use a nightly.

https://ci.jboss.org/hudson/job/WildFly-latest-master/lastBuild/

On Fri, Jan 16, 2015 at 1:02 PM, Tomaž Cerar <[hidden email]> wrote:

On Fri, Jan 16, 2015 at 6:30 PM, filipepferraz <[hidden email]> wrote:
Stuart, what version of wildfly can be used for test the cluster?
I tried in 9.0.0.Alpha2-SNAPSHOT but don't find the mod-cluster filter
inside the configuration=filter.



--
View this message in context: http://wildfly-development.1055759.n5.nabble.com/Using-Wildfly-as-a-load-balancer-tp5715464p5715477.html
Sent from the WildFly Development mailing list archive at Nabble.com.
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

Jorge Solórzano
Hi Stuart,

How will be handled the bind to low ports? will be needed to run the load-balancer as root or some user with privilege to bind in 80 or 443?


Jorge Solórzano
http://www.jorsol.com

On Fri, Jan 16, 2015 at 4:08 PM, denstar <[hidden email]> wrote:
With something this new you'll probably be wanting to build from source or use a nightly.

https://ci.jboss.org/hudson/job/WildFly-latest-master/lastBuild/

On Fri, Jan 16, 2015 at 1:02 PM, Tomaž Cerar <[hidden email]> wrote:

On Fri, Jan 16, 2015 at 6:30 PM, filipepferraz <[hidden email]> wrote:
Stuart, what version of wildfly can be used for test the cluster?
I tried in 9.0.0.Alpha2-SNAPSHOT but don't find the mod-cluster filter
inside the configuration=filter.



--
View this message in context: http://wildfly-development.1055759.n5.nabble.com/Using-Wildfly-as-a-load-balancer-tp5715464p5715477.html
Sent from the WildFly Development mailing list archive at Nabble.com.
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

denstar
On 01/16/2015 03:34 PM, Jorge Solórzano wrote:
> Hi Stuart,
>
> How will be handled the bind to low ports? will be needed to run the
> load-balancer as root or some user with privilege to bind in 80 or 443?

Running as root is generally to be avoided.

You could use software defined networking to do it (depending on your
stack) and have some failover and whatnot there, as a bonus.

Or use PF/iptables or something to forward from 80 to one above 1024.

Search for "privileged port binding" + whatever system you're trying to
do it on to find more information.

Thanks for sharing that video Tomaž, good stuff!

:den
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

jtgreene
Administrator
In reply to this post by Jorge Solórzano

> On Jan 16, 2015, at 4:34 PM, Jorge Solórzano <[hidden email]> wrote:
>
> Hi Stuart,
>
> How will be handled the bind to low ports? will be needed to run the load-balancer as root or some user with privilege to bind in 80 or 443?

What OS?

Assuming Linux there are options, and I recommend A) unless you care about the minuscule CPU cycles spent in kernel netfilter code spent rewriting the packet:

A. iptables rule or firewalld rule

sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080

-or if you use firewalld-

sudo firewall-cmd --add-forward-port=port=80:proto=tcp:toport=8080 --permanent

B. Using setcap to grant perms for java to bind lower ports:

sudo setcap cap_net_bind_service=+epi $JAVA_HOME/bin/java  
sudo setcap cap_net_bind_service=+epi $JAVA_HOME/jre/bin/java

If you get an error about libjli.so, you will need to add it to an ld config:

sudo echo $JAVA_HOME/jre/lib/amd64/jli/libjli.so > /etc/ld.so.conf.d/libjli.conf
sudo ldconfig | grep libjli

This should return:
libjli.so -> libjli.so


--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

denstar
On 01/16/2015 04:19 PM, Jason Greene wrote:
...
[snip helpful example rules]
>
> B. Using setcap to grant perms for java to bind lower ports:

FWIW, this would open things up for Java in general, so while it should
perform better, it'll also be a little more risky, which may or may not
be a concern.

> If you get an error about libjli.so, you will need to add it to an ld config:
>
> sudo echo $JAVA_HOME/jre/lib/amd64/jli/libjli.so > /etc/ld.so.conf.d/libjli.conf
> sudo ldconfig | grep libjli
>
> This should return:
> libjli.so -> libjli.so

Good to know!

-den

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

jtgreene
Administrator

> On Jan 16, 2015, at 5:37 PM, denstar <[hidden email]> wrote:
>
> On 01/16/2015 04:19 PM, Jason Greene wrote:
> ...
> [snip helpful example rules]
>>
>> B. Using setcap to grant perms for java to bind lower ports:
>
> FWIW, this would open things up for Java in general, so while it should
> perform better, it'll also be a little more risky, which may or may not
> be a concern.

Right all Java code using this JVM would have access to binding *all ports* (e.g a Java program could bind say the ssh port (assuming it's not running) and sniff passwords). So it would be a good idea to have a dedicated JVM just for WildFly and to limit the execution permission to just a dedicated WildFly user. That way you ensure only the wildfly process can bind these ports.

Alternatively, you could use something like docker which automates capability assignment and provides some extra isolation. It's overkill though if the only thing running on a box is a wildfly process.

Just a note that you will still get fantastic performance with iptables port forwarding since the particular rule is completely stateless, and the action is just to modify the packet in memory. It's only extreme scenarios where that overhead is worth avoiding.

-Jason




_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

Jorge Solórzano
Is authbind or privbind a good alternative? it probably has the same effect of setcap but with a little more security.

It seems the best choice is iptables.



Jorge Solórzano
http://www.jorsol.com

On Fri, Jan 16, 2015 at 9:31 PM, Jason T. Greene <[hidden email]> wrote:

> On Jan 16, 2015, at 5:37 PM, denstar <[hidden email]> wrote:
>
> On 01/16/2015 04:19 PM, Jason Greene wrote:
> ...
> [snip helpful example rules]
>>
>> B. Using setcap to grant perms for java to bind lower ports:
>
> FWIW, this would open things up for Java in general, so while it should
> perform better, it'll also be a little more risky, which may or may not
> be a concern.

Right all Java code using this JVM would have access to binding *all ports* (e.g a Java program could bind say the ssh port (assuming it's not running) and sniff passwords). So it would be a good idea to have a dedicated JVM just for WildFly and to limit the execution permission to just a dedicated WildFly user. That way you ensure only the wildfly process can bind these ports.

Alternatively, you could use something like docker which automates capability assignment and provides some extra isolation. It's overkill though if the only thing running on a box is a wildfly process.

Just a note that you will still get fantastic performance with iptables port forwarding since the particular rule is completely stateless, and the action is just to modify the packet in memory. It's only extreme scenarios where that overhead is worth avoiding.

-Jason





_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

jtgreene
Administrator
Sure, that should work, but would require some hacking if you are using domain mode, since it launches JVMs for server process. You would probably need to create a fake “java” which was a script which called authbind on the real java.

> On Jan 17, 2015, at 8:27 AM, Jorge Solórzano <[hidden email]> wrote:
>
> Is authbind or privbind a good alternative? it probably has the same effect of setcap but with a little more security.
>
> It seems the best choice is iptables.
>
>
>
> Jorge Solórzano
> http://www.jorsol.com
>
> On Fri, Jan 16, 2015 at 9:31 PM, Jason T. Greene <[hidden email]> wrote:
>
> > On Jan 16, 2015, at 5:37 PM, denstar <[hidden email]> wrote:
> >
> > On 01/16/2015 04:19 PM, Jason Greene wrote:
> > ...
> > [snip helpful example rules]
> >>
> >> B. Using setcap to grant perms for java to bind lower ports:
> >
> > FWIW, this would open things up for Java in general, so while it should
> > perform better, it'll also be a little more risky, which may or may not
> > be a concern.
>
> Right all Java code using this JVM would have access to binding *all ports* (e.g a Java program could bind say the ssh port (assuming it's not running) and sniff passwords). So it would be a good idea to have a dedicated JVM just for WildFly and to limit the execution permission to just a dedicated WildFly user. That way you ensure only the wildfly process can bind these ports.
>
> Alternatively, you could use something like docker which automates capability assignment and provides some extra isolation. It's overkill though if the only thing running on a box is a wildfly process.
>
> Just a note that you will still get fantastic performance with iptables port forwarding since the particular rule is completely stateless, and the action is just to modify the packet in memory. It's only extreme scenarios where that overhead is worth avoiding.
>
> -Jason
>
>
>
>

--
Jason T. Greene
WildFly Lead / JBoss EAP Platform Architect
JBoss, a division of Red Hat


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

denstar
In reply to this post by Jorge Solórzano
On 01/17/2015 07:27 AM, Jorge Solórzano wrote:
> Is authbind or privbind a good alternative? it probably has the same effect
> of setcap but with a little more security.
>
> It seems the best choice is iptables.
>

In general, probably.

As Jason said, we're talking some pretty low-level optimization here. In
99% of cases it won't make a lick of difference performance-wise.

Database calls and file reads and other IO will have a far more
observable impact most the time, and are also more popular vectors of
attack.  It's more likely for one to have something dumb in their code
opening a hole than say, the SSH port binding example given earlier--
tho the former leads to the latter, so +1 for layers. (Including code
reviews and such.)

Really it depends on what you're doing, and plan on doing in the future.
 Things vary by OS, which is something to consider if you're going to
have end-users running your application servers, but not so much if
you're offering a service, for example, or are fine specifying OS
requirements and what have you.

Den*
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

Juraci Paixão Kröhling
In reply to this post by jtgreene
On 01/17/2015 04:31 AM, Jason T. Greene wrote:
> Right all Java code using this JVM would have access to binding *all ports* (e.g a Java program could bind say the ssh port (assuming it's not running) and sniff passwords). So it would be a good idea to have a dedicated JVM just for WildFly and to limit the execution permission to just a dedicated WildFly user. That way you ensure only the wildfly process can bind these ports.

I guess selinux could help on this scenario. IIRC, selinux blocks
WildFly (the one from the repos) from binding on non default ports
(8080, ...), so, a custom rule to allow it to bind to 80 would be
enough. If WildFly tries to bind to 22, selinux will block.

- Juca.

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Using Wildfly as a load balancer

Luiz Gustavo Arruda
This post has NOT been accepted by the mailing list yet.
In reply to this post by Stuart Douglas
Hi Stuart,

I`m trying to run Undertow as a load balancer... but my environment does not allow multicasting communication... is it possible to do this using TCP?

Thanks.