EJB passivation/activation, clustering and JPA extended persistence context handling...

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

EJB passivation/activation, clustering and JPA extended persistence context handling...

Scott Marlow
Hi,

I'm looking into handling serialization of the extended persistence
context (for both clustering and EJB3.1 passivation/activation).

The (JPA subsystem) spi is still developing for this but if anyone has
feedback or wants to follow.  A sketch for the spi is at
https://github.com/scottmarlow/jboss-as/blob/d7ec51ac14afb47ced7d203f8b3696c10221e9db/jpa/spi/src/main/java/org/jboss/as/jpa/spi/XPCSerializationController.java

One question that comes up is why.  Why not just throw
NotSerializableException instead of serializing the extended persistence
context.  If anyone feels strongly that we should throw
NotSerializableException instead of serializing the extended persistence
context, I'd like to hear why.

I've made internal changes on
https://github.com/scottmarlow/jboss-as/tree/cluster1 to support
clustering extended persistence contexts.  Next is to do the actual
serialization.

Scott
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: EJB passivation/activation, clustering and JPA extended persistence context handling...

Brian Stansberry
Throwing NotSerializableException instead of serializing means removing
support for clustering SFSBs with an XPC. That IMHO would be a pretty
major feature regression. I believe we have supported that since AS 4.2
if not earlier.

On 10/24/11 9:39 AM, Scott Marlow wrote:

> Hi,
>
> I'm looking into handling serialization of the extended persistence
> context (for both clustering and EJB3.1 passivation/activation).
>
> The (JPA subsystem) spi is still developing for this but if anyone has
> feedback or wants to follow.  A sketch for the spi is at
> https://github.com/scottmarlow/jboss-as/blob/d7ec51ac14afb47ced7d203f8b3696c10221e9db/jpa/spi/src/main/java/org/jboss/as/jpa/spi/XPCSerializationController.java
>
> One question that comes up is why.  Why not just throw
> NotSerializableException instead of serializing the extended persistence
> context.  If anyone feels strongly that we should throw
> NotSerializableException instead of serializing the extended persistence
> context, I'd like to hear why.
>
> I've made internal changes on
> https://github.com/scottmarlow/jboss-as/tree/cluster1 to support
> clustering extended persistence contexts.  Next is to do the actual
> serialization.
>
> Scott
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


--
Brian Stansberry
Principal Software Engineer
JBoss by Red Hat
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: EJB passivation/activation, clustering and JPA extended persistence context handling...

Scott Marlow


On 10/24/2011 10:47 AM, Brian Stansberry wrote:
> Throwing NotSerializableException instead of serializing means removing
> support for clustering SFSBs with an XPC. That IMHO would be a pretty
> major feature regression. I believe we have supported that since AS 4.2
> if not earlier.

I don't disagree, was just interested in hearing from anyone that
thought we shouldn't do this and why.  I appreciate the feedback that we
should do it.  :)

>
> On 10/24/11 9:39 AM, Scott Marlow wrote:
>> Hi,
>>
>> I'm looking into handling serialization of the extended persistence
>> context (for both clustering and EJB3.1 passivation/activation).
>>
>> The (JPA subsystem) spi is still developing for this but if anyone has
>> feedback or wants to follow.  A sketch for the spi is at
>> https://github.com/scottmarlow/jboss-as/blob/d7ec51ac14afb47ced7d203f8b3696c10221e9db/jpa/spi/src/main/java/org/jboss/as/jpa/spi/XPCSerializationController.java
>>
>> One question that comes up is why.  Why not just throw
>> NotSerializableException instead of serializing the extended persistence
>> context.  If anyone feels strongly that we should throw
>> NotSerializableException instead of serializing the extended persistence
>> context, I'd like to hear why.
>>
>> I've made internal changes on
>> https://github.com/scottmarlow/jboss-as/tree/cluster1 to support
>> clustering extended persistence contexts.  Next is to do the actual
>> serialization.
>>
>> Scott
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>
>

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: EJB passivation/activation, clustering and JPA extended persistence context handling...

Brian Stansberry
On 10/24/11 9:54 AM, Scott Marlow wrote:

>
>
> On 10/24/2011 10:47 AM, Brian Stansberry wrote:
>> Throwing NotSerializableException instead of serializing means removing
>> support for clustering SFSBs with an XPC. That IMHO would be a pretty
>> major feature regression. I believe we have supported that since AS 4.2
>> if not earlier.
>
> I don't disagree, was just interested in hearing from anyone that
> thought we shouldn't do this and why. I appreciate the feedback that we
> should do it. :)
>

Paul had done a bunch of cool stuff around XPC serialization that he had
to put on the back burner to focus on the AS 6 Infinispan integration.

>>
>> On 10/24/11 9:39 AM, Scott Marlow wrote:
>>> Hi,
>>>
>>> I'm looking into handling serialization of the extended persistence
>>> context (for both clustering and EJB3.1 passivation/activation).
>>>
>>> The (JPA subsystem) spi is still developing for this but if anyone has
>>> feedback or wants to follow. A sketch for the spi is at
>>> https://github.com/scottmarlow/jboss-as/blob/d7ec51ac14afb47ced7d203f8b3696c10221e9db/jpa/spi/src/main/java/org/jboss/as/jpa/spi/XPCSerializationController.java
>>>
>>>
>>> One question that comes up is why. Why not just throw
>>> NotSerializableException instead of serializing the extended persistence
>>> context. If anyone feels strongly that we should throw
>>> NotSerializableException instead of serializing the extended persistence
>>> context, I'd like to hear why.
>>>
>>> I've made internal changes on
>>> https://github.com/scottmarlow/jboss-as/tree/cluster1 to support
>>> clustering extended persistence contexts. Next is to do the actual
>>> serialization.
>>>
>>> Scott
>>> _______________________________________________
>>> jboss-as7-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>>
>


--
Brian Stansberry
Principal Software Engineer
JBoss by Red Hat
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: EJB passivation/activation, clustering and JPA extended persistence context handling...

Scott Marlow
On 10/24/2011 11:17 AM, Brian Stansberry wrote:

> On 10/24/11 9:54 AM, Scott Marlow wrote:
>>
>>
>> On 10/24/2011 10:47 AM, Brian Stansberry wrote:
>>> Throwing NotSerializableException instead of serializing means removing
>>> support for clustering SFSBs with an XPC. That IMHO would be a pretty
>>> major feature regression. I believe we have supported that since AS 4.2
>>> if not earlier.
>>
>> I don't disagree, was just interested in hearing from anyone that
>> thought we shouldn't do this and why. I appreciate the feedback that we
>> should do it. :)
>>
>
> Paul had done a bunch of cool stuff around XPC serialization that he had
> to put on the back burner to focus on the AS 6 Infinispan integration.

I think that Paul has been continuing the ideas presented in
http://community.jboss.org/wiki/DevEJB3NewSFSBCache for the SFSB
clustering (Paul?), but some changes will be needed for clustering the
extended persistence context.

The knowledge of the group of SFSBs related to a single SFSB (through
its XPCs, is available in the JPA subsystem and can be exposed via the
XPCSerializationController SPI.  If the SPI is wrong, lets correct it.

If we cannot use the SPI for clustering, I'll remove the clustering
aspects from it, but I'm hoping that won't be the case.

>
>>>
>>> On 10/24/11 9:39 AM, Scott Marlow wrote:
>>>> Hi,
>>>>
>>>> I'm looking into handling serialization of the extended persistence
>>>> context (for both clustering and EJB3.1 passivation/activation).
>>>>
>>>> The (JPA subsystem) spi is still developing for this but if anyone has
>>>> feedback or wants to follow. A sketch for the spi is at
>>>> https://github.com/scottmarlow/jboss-as/blob/d7ec51ac14afb47ced7d203f8b3696c10221e9db/jpa/spi/src/main/java/org/jboss/as/jpa/spi/XPCSerializationController.java
>>>>
>>>>
>>>>
>>>> One question that comes up is why. Why not just throw
>>>> NotSerializableException instead of serializing the extended
>>>> persistence
>>>> context. If anyone feels strongly that we should throw
>>>> NotSerializableException instead of serializing the extended
>>>> persistence
>>>> context, I'd like to hear why.
>>>>
>>>> I've made internal changes on
>>>> https://github.com/scottmarlow/jboss-as/tree/cluster1 to support
>>>> clustering extended persistence contexts. Next is to do the actual
>>>> serialization.
>>>>
>>>> Scott
>>>> _______________________________________________
>>>> jboss-as7-dev mailing list
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>
>>>
>>
>
>

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: EJB passivation/activation, clustering and JPA extended persistence context handling...

Carlo de Wolf
I think its better to treat a clustered XPC as an 'individual SFSB'.
Trying to maintain a graph around one XPC can quickly become very messy.
Especially if different XPCs get involved. Conceptually the XPC can
behave similar as a SFSB and we just keep proxies as references to it.
Then it can replicate "at whim".

Carlo

On 10/24/2011 06:10 PM, Scott Marlow wrote:

> On 10/24/2011 11:17 AM, Brian Stansberry wrote:
>> On 10/24/11 9:54 AM, Scott Marlow wrote:
>>>
>>> On 10/24/2011 10:47 AM, Brian Stansberry wrote:
>>>> Throwing NotSerializableException instead of serializing means removing
>>>> support for clustering SFSBs with an XPC. That IMHO would be a pretty
>>>> major feature regression. I believe we have supported that since AS 4.2
>>>> if not earlier.
>>> I don't disagree, was just interested in hearing from anyone that
>>> thought we shouldn't do this and why. I appreciate the feedback that we
>>> should do it. :)
>>>
>> Paul had done a bunch of cool stuff around XPC serialization that he had
>> to put on the back burner to focus on the AS 6 Infinispan integration.
> I think that Paul has been continuing the ideas presented in
> http://community.jboss.org/wiki/DevEJB3NewSFSBCache for the SFSB
> clustering (Paul?), but some changes will be needed for clustering the
> extended persistence context.
>
> The knowledge of the group of SFSBs related to a single SFSB (through
> its XPCs, is available in the JPA subsystem and can be exposed via the
> XPCSerializationController SPI.  If the SPI is wrong, lets correct it.
>
> If we cannot use the SPI for clustering, I'll remove the clustering
> aspects from it, but I'm hoping that won't be the case.
>
>>>> On 10/24/11 9:39 AM, Scott Marlow wrote:
>>>>> Hi,
>>>>>
>>>>> I'm looking into handling serialization of the extended persistence
>>>>> context (for both clustering and EJB3.1 passivation/activation).
>>>>>
>>>>> The (JPA subsystem) spi is still developing for this but if anyone has
>>>>> feedback or wants to follow. A sketch for the spi is at
>>>>> https://github.com/scottmarlow/jboss-as/blob/d7ec51ac14afb47ced7d203f8b3696c10221e9db/jpa/spi/src/main/java/org/jboss/as/jpa/spi/XPCSerializationController.java
>>>>>
>>>>>
>>>>>
>>>>> One question that comes up is why. Why not just throw
>>>>> NotSerializableException instead of serializing the extended
>>>>> persistence
>>>>> context. If anyone feels strongly that we should throw
>>>>> NotSerializableException instead of serializing the extended
>>>>> persistence
>>>>> context, I'd like to hear why.
>>>>>
>>>>> I've made internal changes on
>>>>> https://github.com/scottmarlow/jboss-as/tree/cluster1 to support
>>>>> clustering extended persistence contexts. Next is to do the actual
>>>>> serialization.
>>>>>
>>>>> Scott
>>>>> _______________________________________________
>>>>> jboss-as7-dev mailing list
>>>>> [hidden email]
>>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>
>>
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: EJB passivation/activation, clustering and JPA extended persistence context handling...

Carlo de Wolf
On 10/24/2011 08:57 PM, Carlo de Wolf wrote:
> I think its better to treat a clustered XPC as an 'individual SFSB'.
> Trying to maintain a graph around one XPC can quickly become very
> messy. Especially if different XPCs get involved. Conceptually the XPC
> can behave similar as a SFSB and we just keep proxies as references to
> it. Then it can replicate "at whim".

I did some prototyping around it here:
http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbossas/projects/ejb3/trunk/testsuite/src/test/java/org/jboss/ejb3/test/xpcalt/XPCAltBean.java?view=markup 
a long time ago.

Carlo

>
> Carlo
>
> On 10/24/2011 06:10 PM, Scott Marlow wrote:
>> On 10/24/2011 11:17 AM, Brian Stansberry wrote:
>>> On 10/24/11 9:54 AM, Scott Marlow wrote:
>>>>
>>>> On 10/24/2011 10:47 AM, Brian Stansberry wrote:
>>>>> Throwing NotSerializableException instead of serializing means
>>>>> removing
>>>>> support for clustering SFSBs with an XPC. That IMHO would be a pretty
>>>>> major feature regression. I believe we have supported that since
>>>>> AS 4.2
>>>>> if not earlier.
>>>> I don't disagree, was just interested in hearing from anyone that
>>>> thought we shouldn't do this and why. I appreciate the feedback
>>>> that we
>>>> should do it. :)
>>>>
>>> Paul had done a bunch of cool stuff around XPC serialization that he
>>> had
>>> to put on the back burner to focus on the AS 6 Infinispan integration.
>> I think that Paul has been continuing the ideas presented in
>> http://community.jboss.org/wiki/DevEJB3NewSFSBCache for the SFSB
>> clustering (Paul?), but some changes will be needed for clustering the
>> extended persistence context.
>>
>> The knowledge of the group of SFSBs related to a single SFSB (through
>> its XPCs, is available in the JPA subsystem and can be exposed via the
>> XPCSerializationController SPI.  If the SPI is wrong, lets correct it.
>>
>> If we cannot use the SPI for clustering, I'll remove the clustering
>> aspects from it, but I'm hoping that won't be the case.
>>
>>>>> On 10/24/11 9:39 AM, Scott Marlow wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I'm looking into handling serialization of the extended persistence
>>>>>> context (for both clustering and EJB3.1 passivation/activation).
>>>>>>
>>>>>> The (JPA subsystem) spi is still developing for this but if
>>>>>> anyone has
>>>>>> feedback or wants to follow. A sketch for the spi is at
>>>>>> https://github.com/scottmarlow/jboss-as/blob/d7ec51ac14afb47ced7d203f8b3696c10221e9db/jpa/spi/src/main/java/org/jboss/as/jpa/spi/XPCSerializationController.java 
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> One question that comes up is why. Why not just throw
>>>>>> NotSerializableException instead of serializing the extended
>>>>>> persistence
>>>>>> context. If anyone feels strongly that we should throw
>>>>>> NotSerializableException instead of serializing the extended
>>>>>> persistence
>>>>>> context, I'd like to hear why.
>>>>>>
>>>>>> I've made internal changes on
>>>>>> https://github.com/scottmarlow/jboss-as/tree/cluster1 to support
>>>>>> clustering extended persistence contexts. Next is to do the actual
>>>>>> serialization.
>>>>>>
>>>>>> Scott
>>>>>> _______________________________________________
>>>>>> jboss-as7-dev mailing list
>>>>>> [hidden email]
>>>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>>
>>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: EJB passivation/activation, clustering and JPA extended persistence context handling...

Scott Marlow
In reply to this post by Carlo de Wolf
Yeah, the graph could be deep.  If we are replicating SFSB1 which is
sharing an extended persistence context with SFSB2 and SFSB2 shares a
separate extended persistence context with SFSB3 (and so it
continues...).  I don't think this is the likely 95% case but still
possible.

I'm open to alternative solutions.

For fail-over to work, I think that all of the SFSBs in the graph, will
also need to be available on the node that we fail over to.  If we were
to replicate them as a group, we could optimize the serialization of
shared XPCs to only be sent once.  I'm not saying that is the way we are
going to do the SFSB level clustering, just pointing out that knowledge
about the XPC use could be obtained through the proposed SPI (if we were
to do it that way).  I think we could handle replicating the SFSB graph
entries individually and I can deal with that, if needed.

Currently, we have the ExtendedEntityManager that is a proxy to the
underlying entity manager.

https://github.com/jbossas/jboss-as/blob/master/jpa/core/src/main/java/org/jboss/as/jpa/container/ExtendedEntityManager.java 
is the current implementation on AS7 master.

https://github.com/scottmarlow/jboss-as/blob/2292b8393c1f32e23d949c728022db45a1ea6bee/jpa/core/src/main/java/org/jboss/as/jpa/container/ExtendedEntityManager.java 
is an UUID based variation of the ExtendedEntityManager (on a cluster
branch).

For the passivation of SFSBs, I think the entire XPC inheritance derived
graph could be passivated.  If one of the SFSBs cannot be passivated
(because its in a transaction, in use or perhaps has an extended
persistence context entity that isn't serializeable), I wouldn't expect
any of the SFSBs to be serialized.

On 10/24/2011 02:57 PM, Carlo de Wolf wrote:

> I think its better to treat a clustered XPC as an 'individual SFSB'.
> Trying to maintain a graph around one XPC can quickly become very messy.
> Especially if different XPCs get involved. Conceptually the XPC can
> behave similar as a SFSB and we just keep proxies as references to it.
> Then it can replicate "at whim".
>
> Carlo
>
> On 10/24/2011 06:10 PM, Scott Marlow wrote:
>> On 10/24/2011 11:17 AM, Brian Stansberry wrote:
>>> On 10/24/11 9:54 AM, Scott Marlow wrote:
>>>>
>>>> On 10/24/2011 10:47 AM, Brian Stansberry wrote:
>>>>> Throwing NotSerializableException instead of serializing means removing
>>>>> support for clustering SFSBs with an XPC. That IMHO would be a pretty
>>>>> major feature regression. I believe we have supported that since AS 4.2
>>>>> if not earlier.
>>>> I don't disagree, was just interested in hearing from anyone that
>>>> thought we shouldn't do this and why. I appreciate the feedback that we
>>>> should do it. :)
>>>>
>>> Paul had done a bunch of cool stuff around XPC serialization that he had
>>> to put on the back burner to focus on the AS 6 Infinispan integration.
>> I think that Paul has been continuing the ideas presented in
>> http://community.jboss.org/wiki/DevEJB3NewSFSBCache for the SFSB
>> clustering (Paul?), but some changes will be needed for clustering the
>> extended persistence context.
>>
>> The knowledge of the group of SFSBs related to a single SFSB (through
>> its XPCs, is available in the JPA subsystem and can be exposed via the
>> XPCSerializationController SPI.  If the SPI is wrong, lets correct it.
>>
>> If we cannot use the SPI for clustering, I'll remove the clustering
>> aspects from it, but I'm hoping that won't be the case.
>>
>>>>> On 10/24/11 9:39 AM, Scott Marlow wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I'm looking into handling serialization of the extended persistence
>>>>>> context (for both clustering and EJB3.1 passivation/activation).
>>>>>>
>>>>>> The (JPA subsystem) spi is still developing for this but if anyone has
>>>>>> feedback or wants to follow. A sketch for the spi is at
>>>>>> https://github.com/scottmarlow/jboss-as/blob/d7ec51ac14afb47ced7d203f8b3696c10221e9db/jpa/spi/src/main/java/org/jboss/as/jpa/spi/XPCSerializationController.java
>>>>>>
>>>>>>
>>>>>>
>>>>>> One question that comes up is why. Why not just throw
>>>>>> NotSerializableException instead of serializing the extended
>>>>>> persistence
>>>>>> context. If anyone feels strongly that we should throw
>>>>>> NotSerializableException instead of serializing the extended
>>>>>> persistence
>>>>>> context, I'd like to hear why.
>>>>>>
>>>>>> I've made internal changes on
>>>>>> https://github.com/scottmarlow/jboss-as/tree/cluster1 to support
>>>>>> clustering extended persistence contexts. Next is to do the actual
>>>>>> serialization.
>>>>>>
>>>>>> Scott
>>>>>> _______________________________________________
>>>>>> jboss-as7-dev mailing list
>>>>>> [hidden email]
>>>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>>
>>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: EJB passivation/activation, clustering and JPA extended persistence context handling...

Scott Marlow
In reply to this post by Brian Stansberry
A related question, I'm trying to get up to speed on what internal state
needs to be replicated, for SFSBs that reference extended persistence
context.  This will help me review some clustering code that is coming
together in this area.  I think we need to replicate the following but
want to see what else needs to be covered:

- The stateful session bean attributes as represented by the EJB container.

- The extended persistence context as represented by the JPA
container/subsystem.

- The JPA SFSBXPCMap which can map a SFSB to collection of extended
persistence contexts that directly referenced.  Can also map a extended
persistence context to collection of SFSBs that reference it.

- The JPA interceptors associated with each stateful session bean, need
to be ready to fire after fail-over has occurred (e.g.
SFSBDestroyInterceptor needs to fire when SFSB @Destroy is invoked after
a fail-over).

- Are there other interceptors associated with the SFSB that also need
to be replicated?

- Any other state for stateful session beans that is held externally to
the SFSB instance?

- Which SFSB classes are to be replicated with the SFSB?  The
StatefulSessionComponentInstance?

We also know that we need to replicate the serialization group (as its
definition changes dynamically).  For example, the application has five
stateful session beans (SFSB1-SFSB5) that are associated with three
extended persistence contexts (XPC1-XPC3).  The relationships between
these five beans are as follows:

   SFSB1 uses XPC1
   SFSB2 uses XPC1 and has reference to SFSB3
   SFSB3 uses XPC2
   SFSB4 uses XPC2 and has reference to SFSB5
   SFSB5 uses XPC3

When SFSB1 fails over, we need to fail-over its working set (the
serialization group which includes SFSB1, SFSB2, SFSB3, SFSB4, SFSB5,
XPC1, XPC2, XPC3).  In case its not obvious why, SFSB2 may have pending
database updates in XPC1, so it must fail-over also to the same target
node.  Since SFSB2 has a reference to SFSB3, SFSB3 needs to fail-over
together with the group also (and so the pattern continues for SFSB4,
SFSB5, XPC3).

Thanks for any answers, this will help me review the clustering code
like I said and also think about ways to support the extended
persistence context replication.  These are some of the questions that I
am asking myself but wanted to open the process up for input from others
that probably know better. ;)

Scott

On 10/24/2011 10:47 AM, Brian Stansberry wrote:

> Throwing NotSerializableException instead of serializing means removing
> support for clustering SFSBs with an XPC. That IMHO would be a pretty
> major feature regression. I believe we have supported that since AS 4.2
> if not earlier.
>
> On 10/24/11 9:39 AM, Scott Marlow wrote:
>> Hi,
>>
>> I'm looking into handling serialization of the extended persistence
>> context (for both clustering and EJB3.1 passivation/activation).
>>
>> The (JPA subsystem) spi is still developing for this but if anyone has
>> feedback or wants to follow.  A sketch for the spi is at
>> https://github.com/scottmarlow/jboss-as/blob/d7ec51ac14afb47ced7d203f8b3696c10221e9db/jpa/spi/src/main/java/org/jboss/as/jpa/spi/XPCSerializationController.java
>>
>> One question that comes up is why.  Why not just throw
>> NotSerializableException instead of serializing the extended persistence
>> context.  If anyone feels strongly that we should throw
>> NotSerializableException instead of serializing the extended persistence
>> context, I'd like to hear why.
>>
>> I've made internal changes on
>> https://github.com/scottmarlow/jboss-as/tree/cluster1 to support
>> clustering extended persistence contexts.  Next is to do the actual
>> serialization.
>>
>> Scott
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>
>

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: EJB passivation/activation, clustering and JPA extended persistence context handling...

Scott Marlow
More about XPC clustering below.  ;)

On 11/11/2011 10:32 PM, Scott Marlow wrote:

> A related question, I'm trying to get up to speed on what internal state
> needs to be replicated, for SFSBs that reference extended persistence
> context.  This will help me review some clustering code that is coming
> together in this area.  I think we need to replicate the following but
> want to see what else needs to be covered:
>
> - The stateful session bean attributes as represented by the EJB container.
>
> - The extended persistence context as represented by the JPA
> container/subsystem.
>
> - The JPA SFSBXPCMap which can map a SFSB to collection of extended
> persistence contexts that directly referenced.  Can also map a extended
> persistence context to collection of SFSBs that reference it.
>
> - The JPA interceptors associated with each stateful session bean, need
> to be ready to fire after fail-over has occurred (e.g.
> SFSBDestroyInterceptor needs to fire when SFSB @Destroy is invoked after
> a fail-over).
>
> - Are there other interceptors associated with the SFSB that also need
> to be replicated?
>
> - Any other state for stateful session beans that is held externally to
> the SFSB instance?
>
> - Which SFSB classes are to be replicated with the SFSB?  The
> StatefulSessionComponentInstance?
>
> We also know that we need to replicate the serialization group (as its
> definition changes dynamically).  For example, the application has five
> stateful session beans (SFSB1-SFSB5) that are associated with three
> extended persistence contexts (XPC1-XPC3).  The relationships between
> these five beans are as follows:
>
>     SFSB1 uses XPC1
>     SFSB2 uses XPC1 and has reference to SFSB3
>     SFSB3 uses XPC2
>     SFSB4 uses XPC2 and has reference to SFSB5
>     SFSB5 uses XPC3
>
> When SFSB1 fails over, we need to fail-over its working set (the
> serialization group which includes SFSB1, SFSB2, SFSB3, SFSB4, SFSB5,
> XPC1, XPC2, XPC3).  In case its not obvious why, SFSB2 may have pending
> database updates in XPC1, so it must fail-over also to the same target
> node.  Since SFSB2 has a reference to SFSB3, SFSB3 needs to fail-over
> together with the group also (and so the pattern continues for SFSB4,
> SFSB5, XPC3).

The above is the application correctness issue.  If SFSB2 fails over to
a different node than SFSB4, the extended persistence context could be
used on two different nodes.  Basically, the extended persistence
context can live through several non-transactional invocations.  When a
transaction is finally started on SFSB3 or SFSB4, the XPC2 changes will
finally be committed.

We need to have a load balancing policy that knows how to fail-over the
beans in the serialization group, as a group.  So, that all requests to
any of the beans, always run on the same target node (else we can have
the same pending database changes applied from multiple nodes).

We also need the serialization group to be complete.

Scott

>
> Thanks for any answers, this will help me review the clustering code
> like I said and also think about ways to support the extended
> persistence context replication.  These are some of the questions that I
> am asking myself but wanted to open the process up for input from others
> that probably know better. ;)
>
> Scott
>
> On 10/24/2011 10:47 AM, Brian Stansberry wrote:
>> Throwing NotSerializableException instead of serializing means removing
>> support for clustering SFSBs with an XPC. That IMHO would be a pretty
>> major feature regression. I believe we have supported that since AS 4.2
>> if not earlier.
>>
>> On 10/24/11 9:39 AM, Scott Marlow wrote:
>>> Hi,
>>>
>>> I'm looking into handling serialization of the extended persistence
>>> context (for both clustering and EJB3.1 passivation/activation).
>>>
>>> The (JPA subsystem) spi is still developing for this but if anyone has
>>> feedback or wants to follow.  A sketch for the spi is at
>>> https://github.com/scottmarlow/jboss-as/blob/d7ec51ac14afb47ced7d203f8b3696c10221e9db/jpa/spi/src/main/java/org/jboss/as/jpa/spi/XPCSerializationController.java
>>>
>>> One question that comes up is why.  Why not just throw
>>> NotSerializableException instead of serializing the extended persistence
>>> context.  If anyone feels strongly that we should throw
>>> NotSerializableException instead of serializing the extended persistence
>>> context, I'd like to hear why.
>>>
>>> I've made internal changes on
>>> https://github.com/scottmarlow/jboss-as/tree/cluster1 to support
>>> clustering extended persistence contexts.  Next is to do the actual
>>> serialization.
>>>
>>> Scott
>>> _______________________________________________
>>> jboss-as7-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>>
>
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: EJB passivation/activation, clustering and JPA extended persistence context handling...

jtgreene
Administrator
On 12/2/11 2:55 PM, Scott Marlow wrote:

>> We also know that we need to replicate the serialization group (as its
>> definition changes dynamically).  For example, the application has five
>> stateful session beans (SFSB1-SFSB5) that are associated with three
>> extended persistence contexts (XPC1-XPC3).  The relationships between
>> these five beans are as follows:
>>
>>      SFSB1 uses XPC1
>>      SFSB2 uses XPC1 and has reference to SFSB3
>>      SFSB3 uses XPC2
>>      SFSB4 uses XPC2 and has reference to SFSB5
>>      SFSB5 uses XPC3
>>
>> When SFSB1 fails over, we need to fail-over its working set (the
>> serialization group which includes SFSB1, SFSB2, SFSB3, SFSB4, SFSB5,
>> XPC1, XPC2, XPC3).  In case its not obvious why, SFSB2 may have pending
>> database updates in XPC1, so it must fail-over also to the same target
>> node.  Since SFSB2 has a reference to SFSB3, SFSB3 needs to fail-over
>> together with the group also (and so the pattern continues for SFSB4,
>> SFSB5, XPC3).
>
> The above is the application correctness issue.  If SFSB2 fails over to
> a different node than SFSB4, the extended persistence context could be
> used on two different nodes.  Basically, the extended persistence
> context can live through several non-transactional invocations.  When a
> transaction is finally started on SFSB3 or SFSB4, the XPC2 changes will
> finally be committed.

I still don't this is right. those references go across a proxy, and the
proxy's backing data can be anywhere.

--
Jason T. Greene
JBoss AS Lead / EAP Platform Architect
JBoss, a division of Red Hat
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: EJB passivation/activation, clustering and JPA extended persistence context handling...

Scott Marlow
On 12/02/2011 04:38 PM, Jason T. Greene wrote:

> On 12/2/11 2:55 PM, Scott Marlow wrote:
>>> We also know that we need to replicate the serialization group (as its
>>> definition changes dynamically).  For example, the application has five
>>> stateful session beans (SFSB1-SFSB5) that are associated with three
>>> extended persistence contexts (XPC1-XPC3).  The relationships between
>>> these five beans are as follows:
>>>
>>>       SFSB1 uses XPC1
>>>       SFSB2 uses XPC1 and has reference to SFSB3
>>>       SFSB3 uses XPC2
>>>       SFSB4 uses XPC2 and has reference to SFSB5
>>>       SFSB5 uses XPC3
>>>
>>> When SFSB1 fails over, we need to fail-over its working set (the
>>> serialization group which includes SFSB1, SFSB2, SFSB3, SFSB4, SFSB5,
>>> XPC1, XPC2, XPC3).  In case its not obvious why, SFSB2 may have pending
>>> database updates in XPC1, so it must fail-over also to the same target
>>> node.  Since SFSB2 has a reference to SFSB3, SFSB3 needs to fail-over
>>> together with the group also (and so the pattern continues for SFSB4,
>>> SFSB5, XPC3).
>>
>> The above is the application correctness issue.  If SFSB2 fails over to
>> a different node than SFSB4, the extended persistence context could be
>> used on two different nodes.  Basically, the extended persistence
>> context can live through several non-transactional invocations.  When a
>> transaction is finally started on SFSB3 or SFSB4, the XPC2 changes will
>> finally be committed.
>
> I still don't this is right. those references go across a proxy, and the
> proxy's backing data can be anywhere.
>

I should of added that SFSB2 has a local reference to SFSB3 (which means
they originates on the same JVM).  I assumed that SFSB2 is migrated to
the same target node as SFSB3 to keep the reference local (during a
fail-over).  No matter, where SFSB3 goes, SFSB4 needs to be on the same
node (to share the same extended persistence context).

The SFSBs themselves, can be anywhere but the extended persistence
context, needs to stay with the SFSBs that inherit it.

Maybe there is another way to tackle this that I'm not seeing but I
wanted to mention these details to whomever is following this thread.
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: EJB passivation/activation, clustering and JPA extended persistence context handling...

Brian Stansberry
On 12/2/11 4:22 PM, Scott Marlow wrote:

> On 12/02/2011 04:38 PM, Jason T. Greene wrote:
>> On 12/2/11 2:55 PM, Scott Marlow wrote:
>>>> We also know that we need to replicate the serialization group (as its
>>>> definition changes dynamically).  For example, the application has five
>>>> stateful session beans (SFSB1-SFSB5) that are associated with three
>>>> extended persistence contexts (XPC1-XPC3).  The relationships between
>>>> these five beans are as follows:
>>>>
>>>>        SFSB1 uses XPC1
>>>>        SFSB2 uses XPC1 and has reference to SFSB3
>>>>        SFSB3 uses XPC2
>>>>        SFSB4 uses XPC2 and has reference to SFSB5
>>>>        SFSB5 uses XPC3
>>>>
>>>> When SFSB1 fails over, we need to fail-over its working set (the
>>>> serialization group which includes SFSB1, SFSB2, SFSB3, SFSB4, SFSB5,
>>>> XPC1, XPC2, XPC3).  In case its not obvious why, SFSB2 may have pending
>>>> database updates in XPC1, so it must fail-over also to the same target
>>>> node.  Since SFSB2 has a reference to SFSB3, SFSB3 needs to fail-over
>>>> together with the group also (and so the pattern continues for SFSB4,
>>>> SFSB5, XPC3).
>>>
>>> The above is the application correctness issue.  If SFSB2 fails over to
>>> a different node than SFSB4, the extended persistence context could be
>>> used on two different nodes.  Basically, the extended persistence
>>> context can live through several non-transactional invocations.  When a
>>> transaction is finally started on SFSB3 or SFSB4, the XPC2 changes will
>>> finally be committed.
>>
>> I still don't this is right. those references go across a proxy, and the
>> proxy's backing data can be anywhere.
>>
>
> I should of added that SFSB2 has a local reference to SFSB3 (which means
> they originates on the same JVM).  I assumed that SFSB2 is migrated to
> the same target node as SFSB3 to keep the reference local (during a
> fail-over).  No matter, where SFSB3 goes, SFSB4 needs to be on the same
> node (to share the same extended persistence context).
>
> The SFSBs themselves, can be anywhere but the extended persistence
> context, needs to stay with the SFSBs that inherit it.
>
> Maybe there is another way to tackle this that I'm not seeing but I
> wanted to mention these details to whomever is following this thread.

BTW, is there any spec requirement that prohibits concurrent access to
an XPC?

--
Brian Stansberry
Principal Software Engineer
JBoss by Red Hat
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: EJB passivation/activation, clustering and JPA extended persistence context handling...

Scott Marlow
On 12/02/2011 05:27 PM, Brian Stansberry wrote:

> On 12/2/11 4:22 PM, Scott Marlow wrote:
>> On 12/02/2011 04:38 PM, Jason T. Greene wrote:
>>> On 12/2/11 2:55 PM, Scott Marlow wrote:
>>>>> We also know that we need to replicate the serialization group (as its
>>>>> definition changes dynamically).  For example, the application has five
>>>>> stateful session beans (SFSB1-SFSB5) that are associated with three
>>>>> extended persistence contexts (XPC1-XPC3).  The relationships between
>>>>> these five beans are as follows:
>>>>>
>>>>>         SFSB1 uses XPC1
>>>>>         SFSB2 uses XPC1 and has reference to SFSB3
>>>>>         SFSB3 uses XPC2
>>>>>         SFSB4 uses XPC2 and has reference to SFSB5
>>>>>         SFSB5 uses XPC3
>>>>>
>>>>> When SFSB1 fails over, we need to fail-over its working set (the
>>>>> serialization group which includes SFSB1, SFSB2, SFSB3, SFSB4, SFSB5,
>>>>> XPC1, XPC2, XPC3).  In case its not obvious why, SFSB2 may have pending
>>>>> database updates in XPC1, so it must fail-over also to the same target
>>>>> node.  Since SFSB2 has a reference to SFSB3, SFSB3 needs to fail-over
>>>>> together with the group also (and so the pattern continues for SFSB4,
>>>>> SFSB5, XPC3).
>>>>
>>>> The above is the application correctness issue.  If SFSB2 fails over to
>>>> a different node than SFSB4, the extended persistence context could be
>>>> used on two different nodes.  Basically, the extended persistence
>>>> context can live through several non-transactional invocations.  When a
>>>> transaction is finally started on SFSB3 or SFSB4, the XPC2 changes will
>>>> finally be committed.
>>>
>>> I still don't this is right. those references go across a proxy, and the
>>> proxy's backing data can be anywhere.
>>>
>>
>> I should of added that SFSB2 has a local reference to SFSB3 (which means
>> they originates on the same JVM).  I assumed that SFSB2 is migrated to
>> the same target node as SFSB3 to keep the reference local (during a
>> fail-over).  No matter, where SFSB3 goes, SFSB4 needs to be on the same
>> node (to share the same extended persistence context).
>>
>> The SFSBs themselves, can be anywhere but the extended persistence
>> context, needs to stay with the SFSBs that inherit it.
>>
>> Maybe there is another way to tackle this that I'm not seeing but I
>> wanted to mention these details to whomever is following this thread.
>
> BTW, is there any spec requirement that prohibits concurrent access to
> an XPC?

Yes, section 7.2 seems to cover that (they are not expected to be
thread-safe).

JPA 2.0 specification section 7.2 Obtaining an EntityManager:
"
An entity manager must not be shared among multiple concurrently
executing threads, as the entity manager and persistence context are not
required to be threadsafe. Entity managers must only be accessed in a
single-threaded manner.
"

The XPC usage model for sharing instances between SFSBs is described by
section 7.6.2.1 Inheritance of Extended Persistence Context:
"
If a stateful session bean instantiates a stateful session bean
(executing in the same EJB container instance) which also has such an
extended persistence context, the extended persistence context of the
first stateful session bean is inherited by the second stateful session
bean and bound to it, and this rule recursively applies—independently of
whether transactions are active or not at the point of the creation of
the stateful session beans.

If the persistence context has been inherited by any stateful session
beans, the container does not close the persistence context until all
such stateful session beans have been removed or otherwise destroyed.
"


>

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: EJB passivation/activation, clustering and JPA extended persistence context handling...

Radoslaw Rodak
In reply to this post by Brian Stansberry
Let say 5 Node Cluster.

Simpel Case without references between SFSBs
First Cluster Member with 5 independen  SFSBs.

SFSB1 uses XPC1
SFSB2 uses XPC2
SFSB3 uses XPC3
SFSB4 uses XPC4

One remote Client connected to first Cluster Member keeping remote ref to all five SFSBs
First Node killed ( kill -9 )

If I understund Scott correctly this result in 4 LoadBalanced migration groups.
Worst Case State replicated over 4 Nodes.

Remote Client needs then 4 new remote connections....does this need more time ????

Assume now no LoadBalancing on failOver.

All SFSBs goest to let say second Cluster Member.

Remote Client needs 1 new remote connection...
Don't need to figure out any relations ( faster ) because all SFSBs and all XPCs are on same Cluster Node.

Cluster just defines which nodes replicates to which nodes. On FailOver One Cluster Member needs new Node for replication.


Am 02.12.2011 um 23:27 schrieb Brian Stansberry:

> On 12/2/11 4:22 PM, Scott Marlow wrote:
>> On 12/02/2011 04:38 PM, Jason T. Greene wrote:
>>> On 12/2/11 2:55 PM, Scott Marlow wrote:
>>>>> We also know that we need to replicate the serialization group (as its
>>>>> definition changes dynamically).  For example, the application has five
>>>>> stateful session beans (SFSB1-SFSB5) that are associated with three
>>>>> extended persistence contexts (XPC1-XPC3).  The relationships between
>>>>> these five beans are as follows:
>>>>>
>>>>>       SFSB1 uses XPC1
>>>>>       SFSB2 uses XPC1 and has reference to SFSB3
>>>>>       SFSB3 uses XPC2
>>>>>       SFSB4 uses XPC2 and has reference to SFSB5
>>>>>       SFSB5 uses XPC3
>>>>>
>>>>> When SFSB1 fails over, we need to fail-over its working set (the
>>>>> serialization group which includes SFSB1, SFSB2, SFSB3, SFSB4, SFSB5,
>>>>> XPC1, XPC2, XPC3).  In case its not obvious why, SFSB2 may have pending
>>>>> database updates in XPC1, so it must fail-over also to the same target
>>>>> node.  Since SFSB2 has a reference to SFSB3, SFSB3 needs to fail-over
>>>>> together with the group also (and so the pattern continues for SFSB4,
>>>>> SFSB5, XPC3).
>>>>
>>>> The above is the application correctness issue.  If SFSB2 fails over to
>>>> a different node than SFSB4, the extended persistence context could be
>>>> used on two different nodes.  Basically, the extended persistence
>>>> context can live through several non-transactional invocations.  When a
>>>> transaction is finally started on SFSB3 or SFSB4, the XPC2 changes will
>>>> finally be committed.
>>>
>>> I still don't this is right. those references go across a proxy, and the
>>> proxy's backing data can be anywhere.
>>>
>>
>> I should of added that SFSB2 has a local reference to SFSB3 (which means
>> they originates on the same JVM).  I assumed that SFSB2 is migrated to
>> the same target node as SFSB3 to keep the reference local (during a
>> fail-over).  No matter, where SFSB3 goes, SFSB4 needs to be on the same
>> node (to share the same extended persistence context).
>>
>> The SFSBs themselves, can be anywhere but the extended persistence
>> context, needs to stay with the SFSBs that inherit it.
>>
>> Maybe there is another way to tackle this that I'm not seeing but I
>> wanted to mention these details to whomever is following this thread.
>
> BTW, is there any spec requirement that prohibits concurrent access to
> an XPC?
>
> --
> Brian Stansberry
> Principal Software Engineer
> JBoss by Red Hat
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Fwd: EJB passivation/activation, clustering and JPA extended persistence context handling...

Radoslaw Rodak
Sorry, should be:

Let say 5 Node Cluster.

Simpel Case without references between SFSBs
First Cluster Member with 4 independen  SFSBs.

SFSB1 uses XPC1
SFSB2 uses XPC2
SFSB3 uses XPC3
SFSB4 uses XPC4

One remote Client connected to first Cluster Member keeping remote ref to all four SFSBs
First Node get killed ( kill -9 )

If I understund Scott correctly this result in 4 LoadBalanced migration groups.
LoadBalancing "round robin" each SFSB goes to different cluster member.

Remote Client needs then 4 new remote connections....does this need more time ????

Assume now no LoadBalancing on failOver.

All SFSBs fails over to same Cluster Member.

Remote Client needs 1 new remote connection...
Don't need to figure out any relations ( faster ) because all SFSBs and all XPCs are on same Cluster Node.

Cluster just defines which nodes replicates to which nodes. On FailOver One Cluster Member needs new Node for replication.


Anfang der weitergeleiteten E-Mail:

Von: Radoslaw Rodak <[hidden email]>
Datum: 3. Dezember 2011 01:49:28 MEZ
Betreff: Re: [jboss-as7-dev] EJB passivation/activation, clustering and JPA extended persistence context handling...

Let say 5 Node Cluster.

Simpel Case without references between SFSBs
First Cluster Member with 5 independen  SFSBs.

SFSB1 uses XPC1
SFSB2 uses XPC2
SFSB3 uses XPC3
SFSB4 uses XPC4

One remote Client connected to first Cluster Member keeping remote ref to all five SFSBs
First Node killed ( kill -9 )

If I understund Scott correctly this result in 4 LoadBalanced migration groups.
Worst Case State replicated over 4 Nodes.

Remote Client needs then 4 new remote connections....does this need more time ????

Assume now no LoadBalancing on failOver.

All SFSBs goest to let say second Cluster Member.

Remote Client needs 1 new remote connection...
Don't need to figure out any relations ( faster ) because all SFSBs and all XPCs are on same Cluster Node.

Cluster just defines which nodes replicates to which nodes. On FailOver One Cluster Member needs new Node for replication.


Am 02.12.2011 um 23:27 schrieb Brian Stansberry:

On 12/2/11 4:22 PM, Scott Marlow wrote:
On 12/02/2011 04:38 PM, Jason T. Greene wrote:
On 12/2/11 2:55 PM, Scott Marlow wrote:
We also know that we need to replicate the serialization group (as its
definition changes dynamically).  For example, the application has five
stateful session beans (SFSB1-SFSB5) that are associated with three
extended persistence contexts (XPC1-XPC3).  The relationships between
these five beans are as follows:

     SFSB1 uses XPC1
     SFSB2 uses XPC1 and has reference to SFSB3
     SFSB3 uses XPC2
     SFSB4 uses XPC2 and has reference to SFSB5
     SFSB5 uses XPC3

When SFSB1 fails over, we need to fail-over its working set (the
serialization group which includes SFSB1, SFSB2, SFSB3, SFSB4, SFSB5,
XPC1, XPC2, XPC3).  In case its not obvious why, SFSB2 may have pending
database updates in XPC1, so it must fail-over also to the same target
node.  Since SFSB2 has a reference to SFSB3, SFSB3 needs to fail-over
together with the group also (and so the pattern continues for SFSB4,
SFSB5, XPC3).

The above is the application correctness issue.  If SFSB2 fails over to
a different node than SFSB4, the extended persistence context could be
used on two different nodes.  Basically, the extended persistence
context can live through several non-transactional invocations.  When a
transaction is finally started on SFSB3 or SFSB4, the XPC2 changes will
finally be committed.

I still don't this is right. those references go across a proxy, and the
proxy's backing data can be anywhere.


I should of added that SFSB2 has a local reference to SFSB3 (which means
they originates on the same JVM).  I assumed that SFSB2 is migrated to
the same target node as SFSB3 to keep the reference local (during a
fail-over).  No matter, where SFSB3 goes, SFSB4 needs to be on the same
node (to share the same extended persistence context).

The SFSBs themselves, can be anywhere but the extended persistence
context, needs to stay with the SFSBs that inherit it.

Maybe there is another way to tackle this that I'm not seeing but I
wanted to mention these details to whomever is following this thread.

BTW, is there any spec requirement that prohibits concurrent access to
an XPC?

--
Brian Stansberry
Principal Software Engineer
JBoss by Red Hat
_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: EJB passivation/activation, clustering and JPA extended persistence context handling...

Scott Marlow
Thanks for jumping in to help contribute solutions.

I'm just looking for the edge cases that we have to support, with regard
to clustering the extended persistence context.  More cases are welcome,
as that will help flesh out more issues.

If I'm reading your suggestion correctly, I think your saying that the
stateful session bean fail-over migration path from any one node, to
another node, is pre-determined.  Such that if any node crashes, the
stateful session beans on the crashed node, all migrate to one other
predetermined node.  That would be more of a performance hit, than if we
distributed the session beans evenly across the other remaining nodes
but could work.  The cluster would probably need enough nodes to keep
the machines underutilized, in case they suddenly have to handle the
load from one or more other nodes (wouldn't want to add load to an
overloaded node that is already running 90% cpu utilization or almost
out of Java memory already).  Every load balancing policy has positives
and negatives.  This one allows extended persistence context clustering,
which is the positive.

I think its time to pull together some working examples of the edge
cases (in the form of Arquillian unit tests).  If anyone wants to
contribute to the effort, http://community.jboss.org/wiki/HackingOnAS7 
is good description of how to get starting with building AS7 (you get
your own fork on github and everything, pretty cool ;).  The tests could
be similar to the other non-clustered tests in
https://github.com/jbossas/jboss-as/tree/master/testsuite/integration/basic/src/test/java/org/jboss/as/test/integration/jpa 
and can be kept on a branch until we are ready to create clustered
versions of the tests.

On 12/02/2011 08:02 PM, Radoslaw Rodak wrote:

> Sorry, should be:
>
> Let say 5 Node Cluster.
>
> Simpel Case without references between SFSBs
> First Cluster Member with 4 independen  SFSBs.
>
> SFSB1 uses XPC1
> SFSB2 uses XPC2
> SFSB3 uses XPC3
> SFSB4 uses XPC4
>
> One remote Client connected to first Cluster Member keeping remote ref to all four SFSBs
> First Node get killed ( kill -9 )
>
> If I understund Scott correctly this result in 4 LoadBalanced migration groups.
> LoadBalancing "round robin" each SFSB goes to different cluster member.
>
> Remote Client needs then 4 new remote connections....does this need more time ????
>
> Assume now no LoadBalancing on failOver.
>
> All SFSBs fails over to same Cluster Member.
>
> Remote Client needs 1 new remote connection...
> Don't need to figure out any relations ( faster ) because all SFSBs and all XPCs are on same Cluster Node.
>
> Cluster just defines which nodes replicates to which nodes. On FailOver One Cluster Member needs new Node for replication.
>
>
> Anfang der weitergeleiteten E-Mail:
>
>> Von: Radoslaw Rodak<[hidden email]>
>> Datum: 3. Dezember 2011 01:49:28 MEZ
>> An: [hidden email]
>> Betreff: Re: [jboss-as7-dev] EJB passivation/activation, clustering and JPA extended persistence context handling...
>>
>> Let say 5 Node Cluster.
>>
>> Simpel Case without references between SFSBs
>> First Cluster Member with 5 independen  SFSBs.
>>
>> SFSB1 uses XPC1
>> SFSB2 uses XPC2
>> SFSB3 uses XPC3
>> SFSB4 uses XPC4
>>
>> One remote Client connected to first Cluster Member keeping remote ref to all five SFSBs
>> First Node killed ( kill -9 )
>>
>> If I understund Scott correctly this result in 4 LoadBalanced migration groups.
>> Worst Case State replicated over 4 Nodes.
>>
>> Remote Client needs then 4 new remote connections....does this need more time ????
>>
>> Assume now no LoadBalancing on failOver.
>>
>> All SFSBs goest to let say second Cluster Member.
>>
>> Remote Client needs 1 new remote connection...
>> Don't need to figure out any relations ( faster ) because all SFSBs and all XPCs are on same Cluster Node.
>>
>> Cluster just defines which nodes replicates to which nodes. On FailOver One Cluster Member needs new Node for replication.
>>
>>
>> Am 02.12.2011 um 23:27 schrieb Brian Stansberry:
>>
>>> On 12/2/11 4:22 PM, Scott Marlow wrote:
>>>> On 12/02/2011 04:38 PM, Jason T. Greene wrote:
>>>>> On 12/2/11 2:55 PM, Scott Marlow wrote:
>>>>>>> We also know that we need to replicate the serialization group (as its
>>>>>>> definition changes dynamically).  For example, the application has five
>>>>>>> stateful session beans (SFSB1-SFSB5) that are associated with three
>>>>>>> extended persistence contexts (XPC1-XPC3).  The relationships between
>>>>>>> these five beans are as follows:
>>>>>>>
>>>>>>>       SFSB1 uses XPC1
>>>>>>>       SFSB2 uses XPC1 and has reference to SFSB3
>>>>>>>       SFSB3 uses XPC2
>>>>>>>       SFSB4 uses XPC2 and has reference to SFSB5
>>>>>>>       SFSB5 uses XPC3
>>>>>>>
>>>>>>> When SFSB1 fails over, we need to fail-over its working set (the
>>>>>>> serialization group which includes SFSB1, SFSB2, SFSB3, SFSB4, SFSB5,
>>>>>>> XPC1, XPC2, XPC3).  In case its not obvious why, SFSB2 may have pending
>>>>>>> database updates in XPC1, so it must fail-over also to the same target
>>>>>>> node.  Since SFSB2 has a reference to SFSB3, SFSB3 needs to fail-over
>>>>>>> together with the group also (and so the pattern continues for SFSB4,
>>>>>>> SFSB5, XPC3).
>>>>>>
>>>>>> The above is the application correctness issue.  If SFSB2 fails over to
>>>>>> a different node than SFSB4, the extended persistence context could be
>>>>>> used on two different nodes.  Basically, the extended persistence
>>>>>> context can live through several non-transactional invocations.  When a
>>>>>> transaction is finally started on SFSB3 or SFSB4, the XPC2 changes will
>>>>>> finally be committed.
>>>>>
>>>>> I still don't this is right. those references go across a proxy, and the
>>>>> proxy's backing data can be anywhere.
>>>>>
>>>>
>>>> I should of added that SFSB2 has a local reference to SFSB3 (which means
>>>> they originates on the same JVM).  I assumed that SFSB2 is migrated to
>>>> the same target node as SFSB3 to keep the reference local (during a
>>>> fail-over).  No matter, where SFSB3 goes, SFSB4 needs to be on the same
>>>> node (to share the same extended persistence context).
>>>>
>>>> The SFSBs themselves, can be anywhere but the extended persistence
>>>> context, needs to stay with the SFSBs that inherit it.
>>>>
>>>> Maybe there is another way to tackle this that I'm not seeing but I
>>>> wanted to mention these details to whomever is following this thread.
>>>
>>> BTW, is there any spec requirement that prohibits concurrent access to
>>> an XPC?
>>>
>>> --
>>> Brian Stansberry
>>> Principal Software Engineer
>>> JBoss by Red Hat
>>> _______________________________________________
>>> jboss-as7-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>
>
>
>
> _______________________________________________
> jboss-as7-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: EJB passivation/activation, clustering and JPA extended persistence context handling...

Radoslaw Rodak
Hi Scott

> I think its time to pull together some working examples of the edge cases (in the form of Arquillian unit tests).  If anyone wants to contribute to the effort, http://community.jboss.org/wiki/HackingOnAS7 is good description of how to get starting with building AS7 (you get your own fork on github and everything, pretty cool ;).  The tests could be similar to the other non-clustered tests in https://github.com/jbossas/jboss-as/tree/master/testsuite/integration/basic/src/test/java/org/jboss/as/test/integration/jpa and can be kept on a branch until we are ready to create clustered versions of the tests.

Thanks for helping how to start on jboss.  I'm pretty fresh to Jboss AS project with all Sub Project modules, but I love how you guys are working.
I will be glad to help. Until end of year I'll be pretty busy, but later I will try to help to hack some stuff.
I'm hacking on App Servers since 1998 ( started with Inprise Borland Server ) and now my company decided  to migrate next year from Oracle Weblogic 10.3.x to Jboss  ....enforced with my sugestions to go for Java EE 6 so  EAP 6.1 so for now AS 7.1CR1 snapshot ...

> I think your saying that the stateful session bean fail-over migration path from any one node, to another node, is pre-determined


Yes, but pre-determination doesn't need to be static. To which node the state should be replicated could be loadBalanced depends on policy.
Cluster Node Wight,  Sever Request Queue/Backlog load usw..

>   Such that if any node crashes, the stateful session beans on the crashed node, all migrate to one other predetermined node.


Depends how to understand "migrate on crash".  For me, I would replicate the state of all SFSBs and all XPCs from one Node to another Node  all the time. Mabe using hidden ejb View on other Node until crash is detected and on crash publish the view somwhere where client proxies will find it ( in weblogic you have something called cluster jndi tree wich is shared over all cluster nodes )
Fail Over needs to be fast and clear for clients. If clusters is under load with lot of clients say 100.. if you starts to distributes a lot of SFSBs and XPCs over all Cluster Member....first you spend time where to distribute, second you get 100 proxy requests where to finde all thos SFSB in Cluster... each of survived nodes get s 100 new Requests from 100 Clients from failed node if fail over loadBalanced over all nodes... instead just one node get hit by 100 clients. And then not realy hit, if shadow SFSBs and XPCs are ready to answer :-)

>  The cluster would probably need enough nodes to keep the machines underutilized, in case they suddenly have to handle the load from one or more other nodes (wouldn't want to add load to an overloaded node that is already running 90% cpu utilization or almost out of Java memory already).  


See my last comments...
In my experiance I never needed more the 4 Cluster Members.  75% of all cluster are 2 Node Clusters. In 9 big companies with havy load. :-)
When 4 Cluster Node were not good enought, there was allways a design problem  in application :-)
Some Applikations where even slover using more cluster nodes ( you get more overhead... if you shared, synchronized ressources in cluster... )

> (wouldn't want to add load to an overloaded node that is already running 90% cpu utilization or almost out of Java memory already).  

If all nodes in cluster have 90% cpu usage, the clustering works near at perfection. Then it doesn't mother to which nodes you will replicate and later fail over if needed. If one Node in cluster has 90% cpu Usage an others are IDLE douring StressTest then you need to reimpleted your LoadBalancing... because cluster is usless... loadBalacing works if Load is distributed over all  Nodes. Nodes with more or less similar load.
 
> or almost out of Java memory already

I wonder how  memory shortage is mesured in 7.1 :-) GarbageCollection is extrymly dynamic. Sometimes oa Node without Load  can stay  at high Memory usage until some Request requests more memory which might force full GC to free Memory.
Before JVM get's out of Memory, JVM respons will be extrymly slow ( Full GC -> not enought Memory - Full GC ->  not Enough Memory ... )
If you have something which pings each cluster nodes and writes roundtripp time, then your Cluster LoadBalancing will redirect new Application Client Request to other  nodes.  Because "fail-over migration" is predeterineted, the cluster Member with "almost out of Memory" includes allready Memory for SFSB and XPCs state replication! So first you don't need additional Memory on, second if load balancing works, this node will not get new requests on it, if Cluster LoadBalancing works... and might recover.
If all Nodes are overloaded LoadBalancing of FailOver just make it wors... and then you better sugest to your Client to invest i additional RAM and/or more cluster Members    :-)
Question is, what is chiepier and helps more ... buy memory or trouble shooting and playing with failover Load Balancing policy ...

> Every load balancing policy has positives and negatives.  This one allows extended persistence context clustering, which is the positive.

Of cours. It allways good to have choice (  but less is more :-) )


Am 03.12.2011 um 05:14 schrieb Scott Marlow:

> Thanks for jumping in to help contribute solutions.
>
> I'm just looking for the edge cases that we have to support, with regard to clustering the extended persistence context.  More cases are welcome, as that will help flesh out more issues.
>
> If I'm reading your suggestion correctly, I think your saying that the stateful session bean fail-over migration path from any one node, to another node, is pre-determined.  Such that if any node crashes, the stateful session beans on the crashed node, all migrate to one other predetermined node.  That would be more of a performance hit, than if we distributed the session beans evenly across the other remaining nodes but could work.  The cluster would probably need enough nodes to keep the machines underutilized, in case they suddenly have to handle the load from one or more other nodes (wouldn't want to add load to an overloaded node that is already running 90% cpu utilization or almost out of Java memory already).  Every load balancing policy has positives and negatives.  This one allows extended persistence context clustering, which is the positive.
>
> I think its time to pull together some working examples of the edge cases (in the form of Arquillian unit tests).  If anyone wants to contribute to the effort, http://community.jboss.org/wiki/HackingOnAS7 is good description of how to get starting with building AS7 (you get your own fork on github and everything, pretty cool ;).  The tests could be similar to the other non-clustered tests in https://github.com/jbossas/jboss-as/tree/master/testsuite/integration/basic/src/test/java/org/jboss/as/test/integration/jpa and can be kept on a branch until we are ready to create clustered versions of the tests.
>
> On 12/02/2011 08:02 PM, Radoslaw Rodak wrote:
>> Sorry, should be:
>>
>> Let say 5 Node Cluster.
>>
>> Simpel Case without references between SFSBs
>> First Cluster Member with 4 independen  SFSBs.
>>
>> SFSB1 uses XPC1
>> SFSB2 uses XPC2
>> SFSB3 uses XPC3
>> SFSB4 uses XPC4
>>
>> One remote Client connected to first Cluster Member keeping remote ref to all four SFSBs
>> First Node get killed ( kill -9 )
>>
>> If I understund Scott correctly this result in 4 LoadBalanced migration groups.
>> LoadBalancing "round robin" each SFSB goes to different cluster member.
>>
>> Remote Client needs then 4 new remote connections....does this need more time ????
>>
>> Assume now no LoadBalancing on failOver.
>>
>> All SFSBs fails over to same Cluster Member.
>>
>> Remote Client needs 1 new remote connection...
>> Don't need to figure out any relations ( faster ) because all SFSBs and all XPCs are on same Cluster Node.
>>
>> Cluster just defines which nodes replicates to which nodes. On FailOver One Cluster Member needs new Node for replication.
>>
>>
>> Anfang der weitergeleiteten E-Mail:
>>
>>> Von: Radoslaw Rodak<[hidden email]>
>>> Datum: 3. Dezember 2011 01:49:28 MEZ
>>> An: [hidden email]
>>> Betreff: Re: [jboss-as7-dev] EJB passivation/activation, clustering and JPA extended persistence context handling...
>>>
>>> Let say 5 Node Cluster.
>>>
>>> Simpel Case without references between SFSBs
>>> First Cluster Member with 5 independen  SFSBs.
>>>
>>> SFSB1 uses XPC1
>>> SFSB2 uses XPC2
>>> SFSB3 uses XPC3
>>> SFSB4 uses XPC4
>>>
>>> One remote Client connected to first Cluster Member keeping remote ref to all five SFSBs
>>> First Node killed ( kill -9 )
>>>
>>> If I understund Scott correctly this result in 4 LoadBalanced migration groups.
>>> Worst Case State replicated over 4 Nodes.
>>>
>>> Remote Client needs then 4 new remote connections....does this need more time ????
>>>
>>> Assume now no LoadBalancing on failOver.
>>>
>>> All SFSBs goest to let say second Cluster Member.
>>>
>>> Remote Client needs 1 new remote connection...
>>> Don't need to figure out any relations ( faster ) because all SFSBs and all XPCs are on same Cluster Node.
>>>
>>> Cluster just defines which nodes replicates to which nodes. On FailOver One Cluster Member needs new Node for replication.
>>>
>>>
>>> Am 02.12.2011 um 23:27 schrieb Brian Stansberry:
>>>
>>>> On 12/2/11 4:22 PM, Scott Marlow wrote:
>>>>> On 12/02/2011 04:38 PM, Jason T. Greene wrote:
>>>>>> On 12/2/11 2:55 PM, Scott Marlow wrote:
>>>>>>>> We also know that we need to replicate the serialization group (as its
>>>>>>>> definition changes dynamically).  For example, the application has five
>>>>>>>> stateful session beans (SFSB1-SFSB5) that are associated with three
>>>>>>>> extended persistence contexts (XPC1-XPC3).  The relationships between
>>>>>>>> these five beans are as follows:
>>>>>>>>
>>>>>>>>      SFSB1 uses XPC1
>>>>>>>>      SFSB2 uses XPC1 and has reference to SFSB3
>>>>>>>>      SFSB3 uses XPC2
>>>>>>>>      SFSB4 uses XPC2 and has reference to SFSB5
>>>>>>>>      SFSB5 uses XPC3
>>>>>>>>
>>>>>>>> When SFSB1 fails over, we need to fail-over its working set (the
>>>>>>>> serialization group which includes SFSB1, SFSB2, SFSB3, SFSB4, SFSB5,
>>>>>>>> XPC1, XPC2, XPC3).  In case its not obvious why, SFSB2 may have pending
>>>>>>>> database updates in XPC1, so it must fail-over also to the same target
>>>>>>>> node.  Since SFSB2 has a reference to SFSB3, SFSB3 needs to fail-over
>>>>>>>> together with the group also (and so the pattern continues for SFSB4,
>>>>>>>> SFSB5, XPC3).
>>>>>>>
>>>>>>> The above is the application correctness issue.  If SFSB2 fails over to
>>>>>>> a different node than SFSB4, the extended persistence context could be
>>>>>>> used on two different nodes.  Basically, the extended persistence
>>>>>>> context can live through several non-transactional invocations.  When a
>>>>>>> transaction is finally started on SFSB3 or SFSB4, the XPC2 changes will
>>>>>>> finally be committed.
>>>>>>
>>>>>> I still don't this is right. those references go across a proxy, and the
>>>>>> proxy's backing data can be anywhere.
>>>>>>
>>>>>
>>>>> I should of added that SFSB2 has a local reference to SFSB3 (which means
>>>>> they originates on the same JVM).  I assumed that SFSB2 is migrated to
>>>>> the same target node as SFSB3 to keep the reference local (during a
>>>>> fail-over).  No matter, where SFSB3 goes, SFSB4 needs to be on the same
>>>>> node (to share the same extended persistence context).
>>>>>
>>>>> The SFSBs themselves, can be anywhere but the extended persistence
>>>>> context, needs to stay with the SFSBs that inherit it.
>>>>>
>>>>> Maybe there is another way to tackle this that I'm not seeing but I
>>>>> wanted to mention these details to whomever is following this thread.
>>>>
>>>> BTW, is there any spec requirement that prohibits concurrent access to
>>>> an XPC?
>>>>
>>>> --
>>>> Brian Stansberry
>>>> Principal Software Engineer
>>>> JBoss by Red Hat
>>>> _______________________________________________
>>>> jboss-as7-dev mailing list
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>
>>>
>>> _______________________________________________
>>> jboss-as7-dev mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>
>>
>>
>>
>> _______________________________________________
>> jboss-as7-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>


_______________________________________________
jboss-as7-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/jboss-as7-dev