Shall we limit size of the deployment in WildFly?

classic Classic list List threaded Threaded
40 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Shall we limit size of the deployment in WildFly?

Lin Gao
Hi,

  WildFly does not limit the deployment file size, users with appropriate privilege(deployer) can select any file to deploy from both CLI and web console.

  For the too big file, it may exhaust all available disk space, and in some cases, even small file can exhaust the disk space if the current disk space is not big enough.

  So shall we limit file size of the deployment in WildFly? or shall we limit the available disk space? or shall we just show a warning message to users?

  If we do, where in the management API configuration for this should be done, if it is done this way?

  Arbitrary limits will break users, so if we have an arbitrary limit it needs to be easily adjusted.

  In case of domain mode, different hosts may have different disk space, which means they are likely have different capacity to hold deployment files. For example, servers in server-group-A may have 2GB available disk space, servers in server-group-B may have 200MB available disk space. An unified limit for the whole domain seems not fair for the servers with more available disk space.

  Also, WildFly does not limit type of the deployment file, but it might need a separate discussion if necessary?

  Any thoughts?

  FYI: https://issues.jboss.org/browse/WFCORE-1057

Best Regards
--
Lin Gao
Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

David Lloyd-2
On 11/03/2015 02:41 AM, Lin Gao wrote:

> Hi,
>
>    WildFly does not limit the deployment file size, users with appropriate privilege(deployer) can select any file to deploy from both CLI and web console.
>
>    For the too big file, it may exhaust all available disk space, and in some cases, even small file can exhaust the disk space if the current disk space is not big enough.
>
>    So shall we limit file size of the deployment in WildFly? or shall we limit the available disk space? or shall we just show a warning message to users?
>
>    If we do, where in the management API configuration for this should be done, if it is done this way?
>
>    Arbitrary limits will break users, so if we have an arbitrary limit it needs to be easily adjusted.
>
>    In case of domain mode, different hosts may have different disk space, which means they are likely have different capacity to hold deployment files. For example, servers in server-group-A may have 2GB available disk space, servers in server-group-B may have 200MB available disk space. An unified limit for the whole domain seems not fair for the servers with more available disk space.
>
>    Also, WildFly does not limit type of the deployment file, but it might need a separate discussion if necessary?
>
>    Any thoughts?

Is this in response to a real observed problem?  In general, if the user
doesn't have space for a deployment, the deployment will fail with an
error and (I am fairly certain) will delete the partial deployment.  If
there is space, then the deployment will succeed regardless of size.

It's interesting that the JIRA you reference speaks in terms of
security.  If an admin wants to lock down storage space, it's probably
best to do it at the operating system level using e.g. disk quotas -
there are too many ways to get the application server to write arbitrary
amounts of data to the file system, regardless of the presence of a
security manager or any other application-level control.

I'm pretty sure that if an attacker has permission to upload deployments
to the server, they already essentially have control over the server.
This should be an OS level concern.

>    FYI: https://issues.jboss.org/browse/WFCORE-1057
>
> Best Regards
> --
> Lin Gao
> Software Engineer
> JBoss by Red Hat
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>

--
- DML
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

Heiko W.Rupp
On 3 Nov 2015, at 14:19, David M. Lloyd wrote:
> I'm pretty sure that if an attacker has permission to upload deployments
> to the server, they already essentially have control over the server.

Well, uploads can be remotely, so this can be seen as a DOS
attack vector that does not necessarily require privileges
for (physical) access like (remote) shell.

And then I recall there being the zip bombs where a very small
file would unzip to a huge one. This is probably nothing that
could be caught by limiting the size of the upload.

Do we know if WF continues to work when e.g. the partition for
log files or other data is full?


--
Reg. Adresse: Red Hat GmbH, Technopark II, Haus C,
Werner-von-Siemens-Ring 14, D-85630 Grasbrunn
Handelsregister: Amtsgericht München HRB 153243
Geschäftsführer: Charles Cachera, Michael Cunningham, Paul Hickey, Charlie Peters
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

signature.asc (210 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

David Lloyd-2
On 11/03/2015 07:30 AM, Heiko W.Rupp wrote:
> On 3 Nov 2015, at 14:19, David M. Lloyd wrote:
>> I'm pretty sure that if an attacker has permission to upload deployments
>> to the server, they already essentially have control over the server.
>
> Well, uploads can be remotely, so this can be seen as a DOS
> attack vector that does not necessarily require privileges
> for (physical) access like (remote) shell.

It does require permissions within our security framework though.  I'm
reasonably sure we're not letting anonymous users upload arbitrary data
to the server without authorization checks.

> And then I recall there being the zip bombs where a very small
> file would unzip to a huge one. This is probably nothing that
> could be caught by limiting the size of the upload.

Sure, but this is only one of many possible attacks that you can perform
if you have the ability to upload deployments to the server.  Even with
a locked down security manager I would never recommend running untrusted
Java code on a server that isn't itself isolated and/or protected at an
OS/VM level.

--
- DML
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

Darran Lofthouse
In reply to this post by Heiko W.Rupp
On 03/11/15 13:30, Heiko W.Rupp wrote:
> On 3 Nov 2015, at 14:19, David M. Lloyd wrote:
>> I'm pretty sure that if an attacker has permission to upload deployments
>> to the server, they already essentially have control over the server.
>
> Well, uploads can be remotely, so this can be seen as a DOS
> attack vector that does not necessarily require privileges
> for (physical) access like (remote) shell.

Any user performing these uploads would have an account for managing the
server in the first place - and if the user has that permission there is
nothing to stop their deployment from creating large files at runtime.

> And then I recall there being the zip bombs where a very small
> file would unzip to a huge one. This is probably nothing that
> could be caught by limiting the size of the upload.
>
> Do we know if WF continues to work when e.g. the partition for
> log files or other data is full?
>
>
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

Eduardo Sant´Ana da Silva
In reply to this post by Heiko W.Rupp
Do we know if WF continues to work when e.g. the partition for
log files or other data is full?

>> You can try to down the server with a DOS like this:  http://sourceforge.net/p/jmdns/bugs/130/
As far as I know the server continue to run, but nothing else on the server will be doing something useful since there is no space left.

This was what I report this year:

 I saw single files log with more than 2.5G of information on my standalone log directory:

2015-07-20 06:53:46,024 SEVERE [javax.jmdns.impl.constants.DNSRecordType] (SocketListener(45-55-77-19.local.)) Could not find reco
rd type for index: -1
2015-07-20 06:53:46,042 SEVERE [javax.jmdns.impl.DNSIncoming] (SocketListener(45-55-77-19.local.)) Could not find record type: dns
[query,192.99.0.161:52050, length=184, id=0x5c78, flags=0x3030]
   0: 5c7830305c783030 5c7830305c783030 5c7830305c783031 5c7830305c783030     \x00\x00 \x00\x00 \x00\x01 \x00\x00
  20: 5c7830305c783030 5c7830305c783030 5c7830395c783546 5c7837335c783635     \x00\x00 \x00\x00 \x09\x5F \x73\x65
  40: 5c7837325c783736 5c7836395c783633 5c7836355c783733 5c7830375c783546     \x72\x76 \x69\x63 \x65\x73 \x07\x5F
  60: 5c7836345c783645 5c7837335c783244 5c7837335c783634 5c7830345c783546     \x64\x6E \x73\x2D \x73\x64 \x04\x5F
  80: 5c7837355c783634 5c7837305c783035 5c7836435c783646 5c7836335c783631     \x75\x64 \x70\x05 \x6C\x6F \x63\x61
  a0: 5c7836435c783030 5c7830305c783043 5c7830305c783031                      \x6C\x00 \x00\x0C \x00\x01

2015-07-20 06:53:46,076 WARNING [javax.jmdns.impl.constants.DNSRecordClass] (SocketListener(45-55-77-19.local.)) Could not find re
cord class for index: -1
2015-07-20 06:53:46,260 SEVERE [javax.jmdns.impl.DNSIncoming$MessageInputStream] (SocketListener(45-55-77-19.local.)) bad domain n
ame: possible circular name detected. Bad offset: 0xffffffff at 0xb6
2015-07-20 06:53:46,270 SEVERE [javax.jmdns.impl.constants.DNSRecordType] (SocketListener(45-55-77-19.local.)) Could not find reco
rd type for index: -1
2015-07-20 06:53:46,296 SEVERE [javax.jmdns.impl.DNSIncoming] (SocketListener(45-55-77-19.local.)) Could not find record type: dns
[query,192.99.0.161:52050, length=184, id=0x5c78, flags=0x3030, questions=1
questions:
[DNSQuestion@<a href="tel:2024648779" value="+12024648779" target="_blank">2024648779 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 0\x00\x01\x00\x00\x00\x00\x00\x00\x
09\x5F\x73\x6.\x72\x76\x69\x63\x65\x73\x07\x5F\x64\x6E\x73\x2D\x73\.4\x04\x5F\x75\x64\x70\x05\x6C\x6F\x63\x61\x6C\x00\x00\.C\x00\x
01ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.]]
question:      [DNSQuestion@<a href="tel:2024648779" value="+12024648779" target="_blank">2024648779 type: TYPE_IGNORE index 0, class: CLASS_UNKNOWN index 0, name: 0\x00\x01\x00\x00\x0
0\x00\x00\x00\x09\x5F\x73\x6.\x72\x76\x69\x63\x65\x73\x07\x5F\x64\x6E\x73\x2D\x73\.4\x04\x5F\x75\x64\x70\x05\x6C\x6F\x63\x61\x6C\x
00\x00\.C\x00\x01ϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿϿ.]
   0: 5c7830305c783030 5c7830305c783030 5c7830305c783031 5c7830305c783030     \x00\x00 \x00\x00 \x00\x01 \x00\x00
  20: 5c7830305c783030 5c7830305c783030 5c7830395c783546 5c7837335c783635     \x00\x00 \x00\x00 \x09\x5F \x73\x65
  40: 5c7837325c783736 5c7836395c783633 5c7836355c783733 5c7830375c783546     \x72\x76 \x69\x63 \x65\x73 \x07\x5F
  60: 5c7836345c783645 5c7837335c783244 5c7837335c783634 5c7830345c783546     \x64\x6E \x73\x2D \x73\x64 \x04\x5F
  80: 5c7837355c783634 5c7837305c783035 5c7836435c783646 5c7836335c783631     \x75\x64 \x70\x05 \x6C\x6F \x63\x61
  a0: 5c7836435c783030 5c7830305c783043 5c7830305c783031                      \x6C\x00 \x00\x0C \x00\x01

Jason said me to use this:
In the CLI do:

/subsystem=logging/logger=javax.jmdns:add
/subsystem=logging/logger=javax.jmdns:write-attribute(name=level,value=OFF)

Or you can edit the XML and add:

 <logger category="javax.jmdns">
                <level name="OFF"/>
 </logger>

I think that maybe some kind of listener could be used to report on UI administration the left space when it is too small. This could be very useful, since there are a lot of masked problems that report a totally different exception since porr try/catch statements that usually report other unrelated message.

2015-11-03 11:30 GMT-02:00 Heiko W.Rupp <[hidden email]>:
On 3 Nov 2015, at 14:19, David M. Lloyd wrote:
> I'm pretty sure that if an attacker has permission to upload deployments
> to the server, they already essentially have control over the server.

Well, uploads can be remotely, so this can be seen as a DOS
attack vector that does not necessarily require privileges
for (physical) access like (remote) shell.

And then I recall there being the zip bombs where a very small
file would unzip to a huge one. This is probably nothing that
could be caught by limiting the size of the upload.

Do we know if WF continues to work when e.g. the partition for
log files or other data is full?


--
Reg. Adresse: Red Hat GmbH, Technopark II, Haus C,
Werner-von-Siemens-Ring 14, D-85630 Grasbrunn
Handelsregister: Amtsgericht München HRB 153243
Geschäftsführer: Charles Cachera, Michael Cunningham, Paul Hickey, Charlie Peters

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev



--
__________________________
Eduardo Sant'Ana da Silva - Dr.
Pesquisador / Consultor de TI


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

Lin Gao
In reply to this post by David Lloyd-2
> On 11/03/2015 02:41 AM, Lin Gao wrote:
> > Hi,
> >
> >    WildFly does not limit the deployment file size, users with appropriate
> >    privilege(deployer) can select any file to deploy from both CLI and web
> >    console.
> >
> >    For the too big file, it may exhaust all available disk space, and in
> >    some cases, even small file can exhaust the disk space if the current
> >    disk space is not big enough.
> >
> >    So shall we limit file size of the deployment in WildFly? or shall we
> >    limit the available disk space? or shall we just show a warning message
> >    to users?
> >
> >    If we do, where in the management API configuration for this should be
> >    done, if it is done this way?
> >
> >    Arbitrary limits will break users, so if we have an arbitrary limit it
> >    needs to be easily adjusted.
> >
> >    In case of domain mode, different hosts may have different disk space,
> >    which means they are likely have different capacity to hold deployment
> >    files. For example, servers in server-group-A may have 2GB available
> >    disk space, servers in server-group-B may have 200MB available disk
> >    space. An unified limit for the whole domain seems not fair for the
> >    servers with more available disk space.
> >
> >    Also, WildFly does not limit type of the deployment file, but it might
> >    need a separate discussion if necessary?
> >
> >    Any thoughts?
>
> Is this in response to a real observed problem?  

Yes it is.

> In general, if the user
> doesn't have space for a deployment, the deployment will fail with an
> error and (I am fairly certain) will delete the partial deployment.  If
> there is space, then the deployment will succeed regardless of size.

> It's interesting that the JIRA you reference speaks in terms of
> security.  If an admin wants to lock down storage space, it's probably
> best to do it at the operating system level using e.g. disk quotas -
> there are too many ways to get the application server to write arbitrary
> amounts of data to the file system, regardless of the presence of a
> security manager or any other application-level control.
>
> I'm pretty sure that if an attacker has permission to upload deployments
> to the server, they already essentially have control over the server.
> This should be an OS level concern.

The JIRA in question was a 'bug' related to security at first, after several
rounds of discussion, it is agreed that it is not a security vulnerability, but
an 'Enhancement'.

The proposed requirement for the enhancement is:

Provide an option in config to alert user that
a) File is larger than a configurable limit
b) File type is/is not valid.

but it may need more discussion in community on both the requirement and design
if it will be done, that is why this thread comes out in first place. :-)

> >    FYI: https://issues.jboss.org/browse/WFCORE-1057
> >

Best Regards
--
Lin Gao
Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

Darran Lofthouse
 From within GWT is there any option to detect the file size before
uploading?

I wonder if it is better if we had something where we ask the server
"Can you accept a file of size X?" before commencing an upload rather
than setting some arbitrary limit.  Something like that could also be
used in domain mode before deployments are distributed allowing them to
fail early.

You may have 8Gb free on your disk and have 2Gb deployments, after a
couple of deployments the space would be gone but the arbitrary value
would still allow 2Gb uploads.

Regards,
Darran Lofthouse.


On 04/11/15 01:13, Lin Gao wrote:

>> On 11/03/2015 02:41 AM, Lin Gao wrote:
>>> Hi,
>>>
>>>     WildFly does not limit the deployment file size, users with appropriate
>>>     privilege(deployer) can select any file to deploy from both CLI and web
>>>     console.
>>>
>>>     For the too big file, it may exhaust all available disk space, and in
>>>     some cases, even small file can exhaust the disk space if the current
>>>     disk space is not big enough.
>>>
>>>     So shall we limit file size of the deployment in WildFly? or shall we
>>>     limit the available disk space? or shall we just show a warning message
>>>     to users?
>>>
>>>     If we do, where in the management API configuration for this should be
>>>     done, if it is done this way?
>>>
>>>     Arbitrary limits will break users, so if we have an arbitrary limit it
>>>     needs to be easily adjusted.
>>>
>>>     In case of domain mode, different hosts may have different disk space,
>>>     which means they are likely have different capacity to hold deployment
>>>     files. For example, servers in server-group-A may have 2GB available
>>>     disk space, servers in server-group-B may have 200MB available disk
>>>     space. An unified limit for the whole domain seems not fair for the
>>>     servers with more available disk space.
>>>
>>>     Also, WildFly does not limit type of the deployment file, but it might
>>>     need a separate discussion if necessary?
>>>
>>>     Any thoughts?
>>
>> Is this in response to a real observed problem?
>
> Yes it is.
>
>> In general, if the user
>> doesn't have space for a deployment, the deployment will fail with an
>> error and (I am fairly certain) will delete the partial deployment.  If
>> there is space, then the deployment will succeed regardless of size.
>
>> It's interesting that the JIRA you reference speaks in terms of
>> security.  If an admin wants to lock down storage space, it's probably
>> best to do it at the operating system level using e.g. disk quotas -
>> there are too many ways to get the application server to write arbitrary
>> amounts of data to the file system, regardless of the presence of a
>> security manager or any other application-level control.
>>
>> I'm pretty sure that if an attacker has permission to upload deployments
>> to the server, they already essentially have control over the server.
>> This should be an OS level concern.
>
> The JIRA in question was a 'bug' related to security at first, after several
> rounds of discussion, it is agreed that it is not a security vulnerability, but
> an 'Enhancement'.
>
> The proposed requirement for the enhancement is:
>
> Provide an option in config to alert user that
> a) File is larger than a configurable limit
> b) File type is/is not valid.
>
> but it may need more discussion in community on both the requirement and design
> if it will be done, that is why this thread comes out in first place. :-)
>
>>>     FYI: https://issues.jboss.org/browse/WFCORE-1057
>>>
>
> Best Regards
> --
> Lin Gao
> Software Engineer
> JBoss by Red Hat
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

Stuart Douglas
Is this actually a real world problem that users are running into?

If not then adding any kind of limit seems like it has the potential to cause more problems than it will solve.

As others have pointed out I think the security angle is moot, as if you can deploy to the server your deployment can DOS the server in any number of ways, security manager or no security manager (the simplest way is just to put while(true){} in a Servlet then send some requests, after a while you will lock up every web thread and use 100% CPU, and there is no way to configure the security manager to stop this).

Stuart

On Wed, 4 Nov 2015 at 20:12 Darran Lofthouse <[hidden email]> wrote:
 From within GWT is there any option to detect the file size before
uploading?

I wonder if it is better if we had something where we ask the server
"Can you accept a file of size X?" before commencing an upload rather
than setting some arbitrary limit.  Something like that could also be
used in domain mode before deployments are distributed allowing them to
fail early.

You may have 8Gb free on your disk and have 2Gb deployments, after a
couple of deployments the space would be gone but the arbitrary value
would still allow 2Gb uploads.

Regards,
Darran Lofthouse.


On 04/11/15 01:13, Lin Gao wrote:
>> On 11/03/2015 02:41 AM, Lin Gao wrote:
>>> Hi,
>>>
>>>     WildFly does not limit the deployment file size, users with appropriate
>>>     privilege(deployer) can select any file to deploy from both CLI and web
>>>     console.
>>>
>>>     For the too big file, it may exhaust all available disk space, and in
>>>     some cases, even small file can exhaust the disk space if the current
>>>     disk space is not big enough.
>>>
>>>     So shall we limit file size of the deployment in WildFly? or shall we
>>>     limit the available disk space? or shall we just show a warning message
>>>     to users?
>>>
>>>     If we do, where in the management API configuration for this should be
>>>     done, if it is done this way?
>>>
>>>     Arbitrary limits will break users, so if we have an arbitrary limit it
>>>     needs to be easily adjusted.
>>>
>>>     In case of domain mode, different hosts may have different disk space,
>>>     which means they are likely have different capacity to hold deployment
>>>     files. For example, servers in server-group-A may have 2GB available
>>>     disk space, servers in server-group-B may have 200MB available disk
>>>     space. An unified limit for the whole domain seems not fair for the
>>>     servers with more available disk space.
>>>
>>>     Also, WildFly does not limit type of the deployment file, but it might
>>>     need a separate discussion if necessary?
>>>
>>>     Any thoughts?
>>
>> Is this in response to a real observed problem?
>
> Yes it is.
>
>> In general, if the user
>> doesn't have space for a deployment, the deployment will fail with an
>> error and (I am fairly certain) will delete the partial deployment.  If
>> there is space, then the deployment will succeed regardless of size.
>
>> It's interesting that the JIRA you reference speaks in terms of
>> security.  If an admin wants to lock down storage space, it's probably
>> best to do it at the operating system level using e.g. disk quotas -
>> there are too many ways to get the application server to write arbitrary
>> amounts of data to the file system, regardless of the presence of a
>> security manager or any other application-level control.
>>
>> I'm pretty sure that if an attacker has permission to upload deployments
>> to the server, they already essentially have control over the server.
>> This should be an OS level concern.
>
> The JIRA in question was a 'bug' related to security at first, after several
> rounds of discussion, it is agreed that it is not a security vulnerability, but
> an 'Enhancement'.
>
> The proposed requirement for the enhancement is:
>
> Provide an option in config to alert user that
> a) File is larger than a configurable limit
> b) File type is/is not valid.
>
> but it may need more discussion in community on both the requirement and design
> if it will be done, that is why this thread comes out in first place. :-)
>
>>>     FYI: https://issues.jboss.org/browse/WFCORE-1057
>>>
>
> Best Regards
> --
> Lin Gao
> Software Engineer
> JBoss by Red Hat
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

Heiko Braun
In reply to this post by Darran Lofthouse
GWT boils down to plain web technologies.  And no, AFAIKT there is no way to  check the size of an uploaded file across web browsers. Most browsers do however have an implicit limitation on the file size, which for majority is about 2 GB.

On 04 Nov 2015, at 10:11, Darran Lofthouse <[hidden email]> wrote:

From within GWT is there any option to detect the file size before 
uploading?


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

Darran Lofthouse
Haven't checked uploads for a while, do we support chunked encoding or
always send a content length?

On 04/11/15 10:35, Heiko Braun wrote:

> GWT boils down to plain web technologies.  And no, AFAIKT there is no
> way to  check the size of an uploaded file across web browsers. Most
> browsers do however have an implicit limitation on the file size, which
> for majority is about 2 GB.
>
>> On 04 Nov 2015, at 10:11, Darran Lofthouse <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>> From within GWT is there any option to detect the file size before
>> uploading?
>
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

Stuart Douglas
Undertow supports chunked request encoding, so if a client sends a chunked request it will handle it, however AFAIK all browsers will set a content length when uploading a file.

Stuart

On Wed, 4 Nov 2015 at 21:40 Darran Lofthouse <[hidden email]> wrote:
Haven't checked uploads for a while, do we support chunked encoding or
always send a content length?

On 04/11/15 10:35, Heiko Braun wrote:
> GWT boils down to plain web technologies.  And no, AFAIKT there is no
> way to  check the size of an uploaded file across web browsers. Most
> browsers do however have an implicit limitation on the file size, which
> for majority is about 2 GB.
>
>> On 04 Nov 2015, at 10:11, Darran Lofthouse <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>> From within GWT is there any option to detect the file size before
>> uploading?
>
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

jtgreene
Administrator
In reply to this post by Heiko Braun

On Nov 4, 2015, at 4:36 AM, Heiko Braun <[hidden email]> wrote:

GWT boils down to plain web technologies.  And no, AFAIKT there is no way to  check the size of an uploaded file across web browsers. Most browsers do however have an implicit limitation on the file size, which for majority is about 2 GB.

You can now that we have switched to XHR to do the upload. The File object returned from a file list has a size property that you can use.

So you could in theory check that there is available space before posting, however, since as Stuart mentions, browsers set a content length, the server knows it anyway so it's less racy checking at the server level, if one was to check.



On 04 Nov 2015, at 10:11, Darran Lofthouse <[hidden email]> wrote:

From within GWT is there any option to detect the file size before 
uploading?

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

jtgreene
Administrator
In reply to this post by Lin Gao
On Nov 3, 2015, at 7:14 PM, Lin Gao <[hidden email]> wrote:

>>> On 11/03/2015 02:41 AM, Lin Gao wrote:
>>> Hi,
>>>
>>>   WildFly does not limit the deployment file size, users with appropriate
>>>   privilege(deployer) can select any file to deploy from both CLI and web
>>>   console.
>>>
>>>   For the too big file, it may exhaust all available disk space, and in
>>>   some cases, even small file can exhaust the disk space if the current
>>>   disk space is not big enough.
>>>
>>>   So shall we limit file size of the deployment in WildFly? or shall we
>>>   limit the available disk space? or shall we just show a warning message
>>>   to users?
>>>
>>>   If we do, where in the management API configuration for this should be
>>>   done, if it is done this way?
>>>
>>>   Arbitrary limits will break users, so if we have an arbitrary limit it
>>>   needs to be easily adjusted.
>>>
>>>   In case of domain mode, different hosts may have different disk space,
>>>   which means they are likely have different capacity to hold deployment
>>>   files. For example, servers in server-group-A may have 2GB available
>>>   disk space, servers in server-group-B may have 200MB available disk
>>>   space. An unified limit for the whole domain seems not fair for the
>>>   servers with more available disk space.
>>>
>>>   Also, WildFly does not limit type of the deployment file, but it might
>>>   need a separate discussion if necessary?
>>>
>>>   Any thoughts?
>>
>> Is this in response to a real observed problem?  
>
> Yes it is.

My understanding was that this was reported by a user but it was reported as something they were concerned with, and not actually something that was occurring in their environment.

I agree with David that disk quotas on the processing running the app server is the most appropriate solution to prevent interference with other processes.

I do not think we should have a hard coded limit as a constant because that will prevent legit deployments from working, and you can still run out of disk space (e.g you have 20k left and you upload a 21k deployment, which is under 1G)

We could have an extra check that verifies enough disk space in the add content management op, however Java does not give us access to quota information so the best we could do is only verify available disk space (unless we exec a command for every platform we support).

IMO this one is pretty low priority, but perhaps those that monitor this list see it as a need. If so can you speak up?



>
>> In general, if the user
>> doesn't have space for a deployment, the deployment will fail with an
>> error and (I am fairly certain) will delete the partial deployment.  If
>> there is space, then the deployment will succeed regardless of size.
>
>> It's interesting that the JIRA you reference speaks in terms of
>> security.  If an admin wants to lock down storage space, it's probably
>> best to do it at the operating system level using e.g. disk quotas -
>> there are too many ways to get the application server to write arbitrary
>> amounts of data to the file system, regardless of the presence of a
>> security manager or any other application-level control.
>>
>> I'm pretty sure that if an attacker has permission to upload deployments
>> to the server, they already essentially have control over the server.
>> This should be an OS level concern.
>
> The JIRA in question was a 'bug' related to security at first, after several
> rounds of discussion, it is agreed that it is not a security vulnerability, but
> an 'Enhancement'.
>
> The proposed requirement for the enhancement is:
>
> Provide an option in config to alert user that
> a) File is larger than a configurable limit
> b) File type is/is not valid.
>
> but it may need more discussion in community on both the requirement and design
> if it will be done, that is why this thread comes out in first place. :-)
>
>>>   FYI: https://issues.jboss.org/browse/WFCORE-1057
>
> Best Regards
> --
> Lin Gao
> Software Engineer
> JBoss by Red Hat
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Limit type of the deployment in WildFly?

Brian Stansberry
In reply to this post by Lin Gao
On 11/3/15 2:41 AM, Lin Gao wrote:
>
>    Also, WildFly does not limit type of the deployment file, but it might need a separate discussion if necessary?

It's a very different thing, so a separate branch of the thread is
appropriate.

I don't see the file type thing as being a security issue, since
deployments are just bits to WildFly unless one of our deployment unit
processors recognizes the deployment type. Enforcing file types just
helps prevent users trying to deploy the wrong file.

Doing this would require forcing all extensions to register expected
file types with the kernel. This probably wouldn't be that big of a deal
for extensions that are part of WildFly itself, but it would be a
breaking change for any externally developed extensions that use
different file extensions.

It doesn't seem worth it to me, given the long list of things we have to
work on.

By "file types" here, I mean file extension. If checking for a
particular file structure is meant (e.g. the file structures named by
media type suffixes in https://tools.ietf.org/html/rfc6839) then that's
a significantly different requirement that needs clarification.

--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

jtgreene
Administrator
In reply to this post by jtgreene

> On Nov 4, 2015, at 6:48 AM, Jason T. Greene <[hidden email]> wrote:
>
> I do not think we should have a hard coded limit as a constant because that will prevent legit deployments from working, and you can still run out of disk space (e.g you have 20k left and you upload a 21k deployment, which is under 1G)
>
> We could have an extra check that verifies enough disk space in the add content management op, however Java does not give us access to quota information so the best we could do is only verify available disk space (unless we exec a command for every platform we support).

One thing I forgot to get into was domain mode. Runtime verification is straight forward on standalone, but on domain mode the deployment is pulled by arbitrary hosts, which each may have varying disk capacity.

So in the host case we can only error later on and after the fact. Hosts try to pull down content in the content repository (where deployments are stored) only when they need them, and this can happen well after a deployment is added. As an example, 3 weeks later you could add a server to a host that's in a server group that requires a download of a deployment, and at that point go over capacity.
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

Darran Lofthouse
In reply to this post by Stuart Douglas
+1 That is why I am thinking if any check is needed can it be a
meaningful check such as "Is there room to store the file I am about to
upload" rather than some arbitrary value which will be wrong for someone
and as I point out even if you allow a large value that still does not
actually solve the problem.

On 04/11/15 09:26, Stuart Douglas wrote:

> Is this actually a real world problem that users are running into?
>
> If not then adding any kind of limit seems like it has the potential to
> cause more problems than it will solve.
>
> As others have pointed out I think the security angle is moot, as if you
> can deploy to the server your deployment can DOS the server in any
> number of ways, security manager or no security manager (the simplest
> way is just to put while(true){} in a Servlet then send some requests,
> after a while you will lock up every web thread and use 100% CPU, and
> there is no way to configure the security manager to stop this).
>
> Stuart
>
> On Wed, 4 Nov 2015 at 20:12 Darran Lofthouse <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>       From within GWT is there any option to detect the file size before
>     uploading?
>
>     I wonder if it is better if we had something where we ask the server
>     "Can you accept a file of size X?" before commencing an upload rather
>     than setting some arbitrary limit.  Something like that could also be
>     used in domain mode before deployments are distributed allowing them to
>     fail early.
>
>     You may have 8Gb free on your disk and have 2Gb deployments, after a
>     couple of deployments the space would be gone but the arbitrary value
>     would still allow 2Gb uploads.
>
>     Regards,
>     Darran Lofthouse.
>
>
>     On 04/11/15 01:13, Lin Gao wrote:
>      >> On 11/03/2015 02:41 AM, Lin Gao wrote:
>      >>> Hi,
>      >>>
>      >>>     WildFly does not limit the deployment file size, users with
>     appropriate
>      >>>     privilege(deployer) can select any file to deploy from both
>     CLI and web
>      >>>     console.
>      >>>
>      >>>     For the too big file, it may exhaust all available disk
>     space, and in
>      >>>     some cases, even small file can exhaust the disk space if
>     the current
>      >>>     disk space is not big enough.
>      >>>
>      >>>     So shall we limit file size of the deployment in WildFly?
>     or shall we
>      >>>     limit the available disk space? or shall we just show a
>     warning message
>      >>>     to users?
>      >>>
>      >>>     If we do, where in the management API configuration for
>     this should be
>      >>>     done, if it is done this way?
>      >>>
>      >>>     Arbitrary limits will break users, so if we have an
>     arbitrary limit it
>      >>>     needs to be easily adjusted.
>      >>>
>      >>>     In case of domain mode, different hosts may have different
>     disk space,
>      >>>     which means they are likely have different capacity to hold
>     deployment
>      >>>     files. For example, servers in server-group-A may have 2GB
>     available
>      >>>     disk space, servers in server-group-B may have 200MB
>     available disk
>      >>>     space. An unified limit for the whole domain seems not fair
>     for the
>      >>>     servers with more available disk space.
>      >>>
>      >>>     Also, WildFly does not limit type of the deployment file,
>     but it might
>      >>>     need a separate discussion if necessary?
>      >>>
>      >>>     Any thoughts?
>      >>
>      >> Is this in response to a real observed problem?
>      >
>      > Yes it is.
>      >
>      >> In general, if the user
>      >> doesn't have space for a deployment, the deployment will fail
>     with an
>      >> error and (I am fairly certain) will delete the partial
>     deployment.  If
>      >> there is space, then the deployment will succeed regardless of size.
>      >
>      >> It's interesting that the JIRA you reference speaks in terms of
>      >> security.  If an admin wants to lock down storage space, it's
>     probably
>      >> best to do it at the operating system level using e.g. disk quotas -
>      >> there are too many ways to get the application server to write
>     arbitrary
>      >> amounts of data to the file system, regardless of the presence of a
>      >> security manager or any other application-level control.
>      >>
>      >> I'm pretty sure that if an attacker has permission to upload
>     deployments
>      >> to the server, they already essentially have control over the
>     server.
>      >> This should be an OS level concern.
>      >
>      > The JIRA in question was a 'bug' related to security at first,
>     after several
>      > rounds of discussion, it is agreed that it is not a security
>     vulnerability, but
>      > an 'Enhancement'.
>      >
>      > The proposed requirement for the enhancement is:
>      >
>      > Provide an option in config to alert user that
>      > a) File is larger than a configurable limit
>      > b) File type is/is not valid.
>      >
>      > but it may need more discussion in community on both the
>     requirement and design
>      > if it will be done, that is why this thread comes out in first
>     place. :-)
>      >
>      >>>     FYI: https://issues.jboss.org/browse/WFCORE-1057
>      >>>
>      >
>      > Best Regards
>      > --
>      > Lin Gao
>      > Software Engineer
>      > JBoss by Red Hat
>      > _______________________________________________
>      > wildfly-dev mailing list
>      > [hidden email] <mailto:[hidden email]>
>      > https://lists.jboss.org/mailman/listinfo/wildfly-dev
>      >
>     _______________________________________________
>     wildfly-dev mailing list
>     [hidden email] <mailto:[hidden email]>
>     https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

Darran Lofthouse
In reply to this post by jtgreene
Does the op in the CLI send any length information?  That would help the
two be consistent.

On 04/11/15 12:29, Jason T. Greene wrote:

>
> On Nov 4, 2015, at 4:36 AM, Heiko Braun <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>> GWT boils down to plain web technologies.  And no, AFAIKT there is no
>> way to  check the size of an uploaded file across web browsers. Most
>> browsers do however have an implicit limitation on the file size,
>> which for majority is about 2 GB.
>
> You can now that we have switched to XHR to do the upload. The File
> object returned from a file list has a size property that you can use.
>
> So you could in theory check that there is available space before
> posting, however, since as Stuart mentions, browsers set a content
> length, the server knows it anyway so it's less racy checking at the
> server level, if one was to check.
>
>
>>
>>> On 04 Nov 2015, at 10:11, Darran Lofthouse
>>> <[hidden email] <mailto:[hidden email]>> wrote:
>>>
>>> From within GWT is there any option to detect the file size before
>>> uploading?
>>
>> _______________________________________________
>> wildfly-dev mailing list
>> [hidden email] <mailto:[hidden email]>
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

Brian Stansberry
It shouldn't as part of the management API, no.[1] Any total size
information that gets exchanged is part of the underlying management
communication protocol used for propagating streams and isn't exposed
outside that layer. I'd like to see even that bit go away too, as it's
unnecessary and wastes resources on the client.

[1] In some cases the CLI actually sends the deployment bytes base 64
encoded as a param to the op, in which case the size info is available
to the operation step handler. I consider that CLI behavior to be a bug
though.

On 11/4/15 7:52 AM, Darran Lofthouse wrote:

> Does the op in the CLI send any length information?  That would help the
> two be consistent.
>
> On 04/11/15 12:29, Jason T. Greene wrote:
>>
>> On Nov 4, 2015, at 4:36 AM, Heiko Braun <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>>> GWT boils down to plain web technologies.  And no, AFAIKT there is no
>>> way to  check the size of an uploaded file across web browsers. Most
>>> browsers do however have an implicit limitation on the file size,
>>> which for majority is about 2 GB.
>>
>> You can now that we have switched to XHR to do the upload. The File
>> object returned from a file list has a size property that you can use.
>>
>> So you could in theory check that there is available space before
>> posting, however, since as Stuart mentions, browsers set a content
>> length, the server knows it anyway so it's less racy checking at the
>> server level, if one was to check.
>>
>>
>>>
>>>> On 04 Nov 2015, at 10:11, Darran Lofthouse
>>>> <[hidden email] <mailto:[hidden email]>> wrote:
>>>>
>>>>  From within GWT is there any option to detect the file size before
>>>> uploading?
>>>
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> [hidden email] <mailto:[hidden email]>
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>


--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: Shall we limit size of the deployment in WildFly?

Darran Lofthouse
Actually - this makes me think 'Content Length' on a HTTP upload is not
a suitable measure of the file size - this is the length being sent to
the server, not necessarily the actual size of the resulting file.

On 04/11/15 14:06, Brian Stansberry wrote:

> It shouldn't as part of the management API, no.[1] Any total size
> information that gets exchanged is part of the underlying management
> communication protocol used for propagating streams and isn't exposed
> outside that layer. I'd like to see even that bit go away too, as it's
> unnecessary and wastes resources on the client.
>
> [1] In some cases the CLI actually sends the deployment bytes base 64
> encoded as a param to the op, in which case the size info is available
> to the operation step handler. I consider that CLI behavior to be a bug
> though.
>
> On 11/4/15 7:52 AM, Darran Lofthouse wrote:
>> Does the op in the CLI send any length information?  That would help the
>> two be consistent.
>>
>> On 04/11/15 12:29, Jason T. Greene wrote:
>>>
>>> On Nov 4, 2015, at 4:36 AM, Heiko Braun <[hidden email]
>>> <mailto:[hidden email]>> wrote:
>>>
>>>> GWT boils down to plain web technologies.  And no, AFAIKT there is no
>>>> way to  check the size of an uploaded file across web browsers. Most
>>>> browsers do however have an implicit limitation on the file size,
>>>> which for majority is about 2 GB.
>>>
>>> You can now that we have switched to XHR to do the upload. The File
>>> object returned from a file list has a size property that you can use.
>>>
>>> So you could in theory check that there is available space before
>>> posting, however, since as Stuart mentions, browsers set a content
>>> length, the server knows it anyway so it's less racy checking at the
>>> server level, if one was to check.
>>>
>>>
>>>>
>>>>> On 04 Nov 2015, at 10:11, Darran Lofthouse
>>>>> <[hidden email] <mailto:[hidden email]>> wrote:
>>>>>
>>>>>   From within GWT is there any option to detect the file size before
>>>>> uploading?
>>>>
>>>> _______________________________________________
>>>> wildfly-dev mailing list
>>>> [hidden email] <mailto:[hidden email]>
>>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>> _______________________________________________
>> wildfly-dev mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>
>
>
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
12