WildFly nightly builds

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

WildFly nightly builds

Rostislav Svoboda
Hi.

https://developer.jboss.org/thread/224262 announced availability of WildFly nightly builds.

Current status is that you must know the URL based on current version in master
At this moment it is https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/wildfly-11.0.0.Final-SNAPSHOT.zip, root dir of the zip is wildfly-11.0.0.Final-SNAPSHOT

'11.0.0.Final-SNAPSHOT' is the changing part and when you want to consume nightly build in CI (e.g. Travis) you end up doing unnecessary changes to .travis.yml (Alpha1-SNAPSHOT ==> Beta1-SNAPSHOT ==> Final-SNAPSHOT) when PRs fail because of unavailable nightly bits

I would like to make automated consumption of WildFly nightly builds easier

Proposal 1 - expose latest bits under the same URL + keep the same root dir name
  URL:                  https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/wildfly-latest-SNAPSHOT.zip
  root dir of the zip:  wildfly-latest-SNAPSHOT

Automation would be prepared just once and "work forever".


Proposal 2 - keep current zip & root dir name + add txt file with version information
  introduce https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/version.txt
  Currently it would contain '11.0.0.Final-SNAPSHOT'
    Example to get version: mvn help:evaluate -Dexpression=project.version | grep -v "^\["

Automation could consume content of version.txt and construct the download URL + root dir name
  VERSION=`curl https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/version.txt`
  https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/wildfly-${VERSION}.zip
  root dir of the zip is wildfly-${VERSION}


WDYT?

Regards.
Rostislav

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: WildFly nightly builds

Josef Cacek
You can also use a workaround through artifacts.zip. We use something like this in our automation:

wget --user=guest --password=guest https://ci.wildfly.org/httpAuth/repository/downloadAll/WF_Nightly/.lastSuccessful/artifacts.zip
unzip -q artifacts.zip
export WILDFLY_DIST_ZIP=$(ls wildfly-*-SNAPSHOT.zip)
unzip -q $WILDFLY_DIST_ZIP
export WILDFLY_HOME=`pwd`/${WILDFLY_DIST_ZIP%.zip}


Regards,

-- Josef

----- Original Message -----

> From: "Rostislav Svoboda" <[hidden email]>
> To: "WildFly Dev" <[hidden email]>
> Sent: Wednesday, September 6, 2017 2:42:21 PM
> Subject: [wildfly-dev] WildFly nightly builds
>
> Hi.
>
> https://developer.jboss.org/thread/224262 announced availability of WildFly
> nightly builds.
>
> Current status is that you must know the URL based on current version in
> master
> At this moment it is
> https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/wildfly-11.0.0.Final-SNAPSHOT.zip,
> root dir of the zip is wildfly-11.0.0.Final-SNAPSHOT
>
> '11.0.0.Final-SNAPSHOT' is the changing part and when you want to consume
> nightly build in CI (e.g. Travis) you end up doing unnecessary changes to
> .travis.yml (Alpha1-SNAPSHOT ==> Beta1-SNAPSHOT ==> Final-SNAPSHOT) when PRs
> fail because of unavailable nightly bits
>
> I would like to make automated consumption of WildFly nightly builds easier
>
> Proposal 1 - expose latest bits under the same URL + keep the same root dir
> name
>   URL:
>   https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/wildfly-latest-SNAPSHOT.zip
>   root dir of the zip:  wildfly-latest-SNAPSHOT
>
> Automation would be prepared just once and "work forever".
>
>
> Proposal 2 - keep current zip & root dir name + add txt file with version
> information
>   introduce
>   https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/version.txt
>   Currently it would contain '11.0.0.Final-SNAPSHOT'
>     Example to get version: mvn help:evaluate -Dexpression=project.version |
>     grep -v "^\["
>
> Automation could consume content of version.txt and construct the download
> URL + root dir name
>   VERSION=`curl
>   https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/version.txt`
>   https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/wildfly-${VERSION}.zip
>   root dir of the zip is wildfly-${VERSION}
>
>
> WDYT?
>
> Regards.
> Rostislav
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: WildFly nightly builds

jtgreene
Administrator
In reply to this post by Rostislav Svoboda
I like Proposal #1.  

On Sep 6, 2017, at 7:42 AM, Rostislav Svoboda <[hidden email]> wrote:

Hi.

https://developer.jboss.org/thread/224262 announced availability of WildFly nightly builds.

Current status is that you must know the URL based on current version in master
At this moment it is https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/wildfly-11.0.0.Final-SNAPSHOT.zip, root dir of the zip is wildfly-11.0.0.Final-SNAPSHOT

'11.0.0.Final-SNAPSHOT' is the changing part and when you want to consume nightly build in CI (e.g. Travis) you end up doing unnecessary changes to .travis.yml (Alpha1-SNAPSHOT ==> Beta1-SNAPSHOT ==> Final-SNAPSHOT) when PRs fail because of unavailable nightly bits

I would like to make automated consumption of WildFly nightly builds easier

Proposal 1 - expose latest bits under the same URL + keep the same root dir name
 URL:                  https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/wildfly-latest-SNAPSHOT.zip
 root dir of the zip:  wildfly-latest-SNAPSHOT

Automation would be prepared just once and "work forever".


Proposal 2 - keep current zip & root dir name + add txt file with version information
 introduce https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/version.txt
 Currently it would contain '11.0.0.Final-SNAPSHOT'
   Example to get version: mvn help:evaluate -Dexpression=project.version | grep -v "^\["

Automation could consume content of version.txt and construct the download URL + root dir name
 VERSION=`curl <a href="https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/version.txt`" class="">https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/version.txt`
 <a href="https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/wildfly-${VERSION}.zip" class="">https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/wildfly-${VERSION}.zip
 root dir of the zip is wildfly-${VERSION}


WDYT?

Regards.
Rostislav

_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

--
Jason T. Greene
Chief Architect, JBoss EAP
Red Hat


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: WildFly nightly builds

Emmanuel Hugonnet
In reply to this post by Rostislav Svoboda
Hi,

Why not jut have a root dir named latest and put the date (aka resolve the snapshot) in the zip name as I think the date of production is an
important information for a nightly build thus it should be part of the artifact name:

https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest/wildfly-nightly-201709070001.zip

Maybe we could also put a "symlink" to it when upload is finished like
https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest/wildfly-nightly-latest.zip

Emmanuel



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev

signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: WildFly nightly builds

Tomaž Cerar-2
Whatever we decide to do, we need to update WildFly build to do so, not just CI job.

probably best thing to add some new profile to dist module that would provide what we need.

On Thu, Sep 7, 2017 at 9:57 AM, Emmanuel Hugonnet <[hidden email]> wrote:
Hi,

Why not jut have a root dir named latest and put the date (aka resolve the snapshot) in the zip name as I think the date of production is an
important information for a nightly build thus it should be part of the artifact name:

https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest/wildfly-nightly-201709070001.zip

Maybe we could also put a "symlink" to it when upload is finished like
https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest/wildfly-nightly-latest.zip

Emmanuel



_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: WildFly nightly builds

Rostislav Svoboda
Hi.

I prepared https://github.com/rsvoboda/wildfly/commit/7c48c50f8b054c4f7e06a78de6bea7f5292171eb to fix dist module so it can leverage already existing 'jboss.as.release.version' property.

mvn clean install -DskipTests -Denforcer.skip=true -Dcheckstyle.skip=true -Djboss.as.release.version=latest -Prelease
...
ls dist/target | grep wildfly
  wildfly-latest
  wildfly-latest.jar
  wildfly-latest-src.tar.gz
  wildfly-latest-src.zip
  wildfly-latest.tar.gz
  wildfly-latest.zip

So WF_Nightly job would just use '-Djboss.as.release.version=latest' and we would have Option #1 solved.


Can I proceed with PR? Is it something I should propose also for product branch ?


Rostislav

----- Original Message -----

> Whatever we decide to do, we need to update WildFly build to do so, not just
> CI job.
>
> probably best thing to add some new profile to dist module that would provide
> what we need.
>
> On Thu, Sep 7, 2017 at 9:57 AM, Emmanuel Hugonnet < [hidden email] >
> wrote:
>
>
> Hi,
>
> Why not jut have a root dir named latest and put the date (aka resolve the
> snapshot) in the zip name as I think the date of production is an
> important information for a nightly build thus it should be part of the
> artifact name:
>
> https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest/wildfly-nightly-201709070001.zip
>
> Maybe we could also put a "symlink" to it when upload is finished like
> https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest/wildfly-nightly-latest.zip
>
> Emmanuel
>
>
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: WildFly nightly builds

Tomaž Cerar-2
In reply to this post by Josef Cacek
Yeah we can always bash around it, on our CI or when consuming it,
but that is not a clean solution in any case.

Having build updated to easier allow for what we need is a way to go.

--
tomaz

On Wed, Sep 6, 2017 at 4:09 PM, Josef Cacek <[hidden email]> wrote:
You can also use a workaround through artifacts.zip. We use something like this in our automation:

wget --user=guest --password=guest https://ci.wildfly.org/httpAuth/repository/downloadAll/WF_Nightly/.lastSuccessful/artifacts.zip
unzip -q artifacts.zip
export WILDFLY_DIST_ZIP=$(ls wildfly-*-SNAPSHOT.zip)
unzip -q $WILDFLY_DIST_ZIP
export WILDFLY_HOME=`pwd`/${WILDFLY_DIST_ZIP%.zip}


Regards,

-- Josef

----- Original Message -----
> From: "Rostislav Svoboda" <[hidden email]>
> To: "WildFly Dev" <[hidden email]>
> Sent: Wednesday, September 6, 2017 2:42:21 PM
> Subject: [wildfly-dev] WildFly nightly builds
>
> Hi.
>
> https://developer.jboss.org/thread/224262 announced availability of WildFly
> nightly builds.
>
> Current status is that you must know the URL based on current version in
> master
> At this moment it is
> https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/wildfly-11.0.0.Final-SNAPSHOT.zip,
> root dir of the zip is wildfly-11.0.0.Final-SNAPSHOT
>
> '11.0.0.Final-SNAPSHOT' is the changing part and when you want to consume
> nightly build in CI (e.g. Travis) you end up doing unnecessary changes to
> .travis.yml (Alpha1-SNAPSHOT ==> Beta1-SNAPSHOT ==> Final-SNAPSHOT) when PRs
> fail because of unavailable nightly bits
>
> I would like to make automated consumption of WildFly nightly builds easier
>
> Proposal 1 - expose latest bits under the same URL + keep the same root dir
> name
>   URL:
>   https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/wildfly-latest-SNAPSHOT.zip
>   root dir of the zip:  wildfly-latest-SNAPSHOT
>
> Automation would be prepared just once and "work forever".
>
>
> Proposal 2 - keep current zip & root dir name + add txt file with version
> information
>   introduce
>   https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/version.txt
>   Currently it would contain '11.0.0.Final-SNAPSHOT'
>     Example to get version: mvn help:evaluate -Dexpression=project.version |
>     grep -v "^\["
>
> Automation could consume content of version.txt and construct the download
> URL + root dir name
>   VERSION=`curl
>   <a href="https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/version.txt`" target="_blank" rel="noreferrer">https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/version.txt`
>   <a href="https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/wildfly-${VERSION}.zip" target="_blank" rel="noreferrer">https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/wildfly-${VERSION}.zip
>   root dir of the zip is wildfly-${VERSION}
>
>
> WDYT?
>
> Regards.
> Rostislav
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: WildFly nightly builds

Bob McWhirter
I haven’t followed entirely, but back in the day of TorqueBox, we’d use a normal CI build, without modifying the pom.xml or other bits.

All we’d do is prior to building the -SNAPSHOT version, we’d

    mvn versions:set -DnewVersion=x.y.z.incremental.$BUILD_NO

e.g.

    mvn versions:set -DnewVersion=11.0.0.incremental.$BUILD_NO

We also actually did that to WildFly proper, since TorqueBox built against WF.  This allowed us to build against a non-SNAPSHOT release of WF without having to wait for you guys to produce a release, because -SNAPSHOTs are terrible and WF 8 took quite a while to arrive.

I called them “incremental” because they happened many times a day, and thus not “nightly” builds in our case. Using the CI-provided builder number ensured they were sequential, and using the base version number helped know what the build roughly was.

-Bob

On Thu, Sep 7, 2017 at 9:27 AM, Tomaž Cerar <[hidden email]> wrote:
Yeah we can always bash around it, on our CI or when consuming it,
but that is not a clean solution in any case.

Having build updated to easier allow for what we need is a way to go.

--
tomaz

On Wed, Sep 6, 2017 at 4:09 PM, Josef Cacek <[hidden email]> wrote:
You can also use a workaround through artifacts.zip. We use something like this in our automation:

wget --user=guest --password=guest https://ci.wildfly.org/httpAuth/repository/downloadAll/WF_Nightly/.lastSuccessful/artifacts.zip
unzip -q artifacts.zip
export WILDFLY_DIST_ZIP=$(ls wildfly-*-SNAPSHOT.zip)
unzip -q $WILDFLY_DIST_ZIP
export WILDFLY_HOME=`pwd`/${WILDFLY_DIST_ZIP%.zip}


Regards,

-- Josef

----- Original Message -----
> From: "Rostislav Svoboda" <[hidden email]>
> To: "WildFly Dev" <[hidden email]>
> Sent: Wednesday, September 6, 2017 2:42:21 PM
> Subject: [wildfly-dev] WildFly nightly builds
>
> Hi.
>
> https://developer.jboss.org/thread/224262 announced availability of WildFly
> nightly builds.
>
> Current status is that you must know the URL based on current version in
> master
> At this moment it is
> https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/wildfly-11.0.0.Final-SNAPSHOT.zip,
> root dir of the zip is wildfly-11.0.0.Final-SNAPSHOT
>
> '11.0.0.Final-SNAPSHOT' is the changing part and when you want to consume
> nightly build in CI (e.g. Travis) you end up doing unnecessary changes to
> .travis.yml (Alpha1-SNAPSHOT ==> Beta1-SNAPSHOT ==> Final-SNAPSHOT) when PRs
> fail because of unavailable nightly bits
>
> I would like to make automated consumption of WildFly nightly builds easier
>
> Proposal 1 - expose latest bits under the same URL + keep the same root dir
> name
>   URL:
>   https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/wildfly-latest-SNAPSHOT.zip
>   root dir of the zip:  wildfly-latest-SNAPSHOT
>
> Automation would be prepared just once and "work forever".
>
>
> Proposal 2 - keep current zip & root dir name + add txt file with version
> information
>   introduce
>   https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/version.txt
>   Currently it would contain '11.0.0.Final-SNAPSHOT'
>     Example to get version: mvn help:evaluate -Dexpression=project.version |
>     grep -v "^\["
>
> Automation could consume content of version.txt and construct the download
> URL + root dir name
>   VERSION=`curl
>   https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/version.txt`
>   https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest.lastFinished/wildfly-${VERSION}.zip
>   root dir of the zip is wildfly-${VERSION}
>
>
> WDYT?
>
> Regards.
> Rostislav
>
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev


_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev
Reply | Threaded
Open this post in threaded view
|

Re: WildFly nightly builds

Rostislav Svoboda
In reply to this post by Rostislav Svoboda
Hi.

PR: https://github.com/wildfly/wildfly/pull/10487

I adjusted also servlet-dist module to be in sync with dist module and to make TS pass.

Rostislav

----- Original Message -----

> Hi.
>
> I prepared
> https://github.com/rsvoboda/wildfly/commit/7c48c50f8b054c4f7e06a78de6bea7f5292171eb
> to fix dist module so it can leverage already existing
> 'jboss.as.release.version' property.
>
> mvn clean install -DskipTests -Denforcer.skip=true -Dcheckstyle.skip=true
> -Djboss.as.release.version=latest -Prelease
> ...
> ls dist/target | grep wildfly
>   wildfly-latest
>   wildfly-latest.jar
>   wildfly-latest-src.tar.gz
>   wildfly-latest-src.zip
>   wildfly-latest.tar.gz
>   wildfly-latest.zip
>
> So WF_Nightly job would just use '-Djboss.as.release.version=latest' and we
> would have Option #1 solved.
>
>
> Can I proceed with PR? Is it something I should propose also for product
> branch ?
>
>
> Rostislav
>
> ----- Original Message -----
> > Whatever we decide to do, we need to update WildFly build to do so, not
> > just
> > CI job.
> >
> > probably best thing to add some new profile to dist module that would
> > provide
> > what we need.
> >
> > On Thu, Sep 7, 2017 at 9:57 AM, Emmanuel Hugonnet < [hidden email] >
> > wrote:
> >
> >
> > Hi,
> >
> > Why not jut have a root dir named latest and put the date (aka resolve the
> > snapshot) in the zip name as I think the date of production is an
> > important information for a nightly build thus it should be part of the
> > artifact name:
> >
> > https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest/wildfly-nightly-201709070001.zip
> >
> > Maybe we could also put a "symlink" to it when upload is finished like
> > https://ci.wildfly.org/guestAuth/repository/download/WF_Nightly/latest/wildfly-nightly-latest.zip
> >
> > Emmanuel
> >
> >
> >
> > _______________________________________________
> > wildfly-dev mailing list
> > [hidden email]
> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> >
> >
> > _______________________________________________
> > wildfly-dev mailing list
> > [hidden email]
> > https://lists.jboss.org/mailman/listinfo/wildfly-dev
> _______________________________________________
> wildfly-dev mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
_______________________________________________
wildfly-dev mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/wildfly-dev