[ACCEPTED]-Continuous Integration vs. Nightly Builds-continuous-integration

If you're really doing continuous integration 14 with all available tests, nightly builds 13 would be redundant, since the last thing 12 checked in that day would already have been 11 tested.

On the other hand, if your CI regime 10 only involves running a subset of all available 9 tests, for example because some of your 8 tests take a long time to run, then you 7 can use nightlies additionally to run all tests. This'll 6 let you catch many bugs early, and if you 5 can't catch them early, you can at least 4 catch them overnight.

I don't know, though, if 3 that's technically still CI, since you're 2 only doing a "partial" build each time, by 1 ignoring some of the tests.

In our organization, nightly builds and 18 CI builds have two distinct purposes. The 17 CI build is a 'latest code' build in which 16 the unit tests are run against the last 15 check in as you would expect. We also run 14 several code metrics on the CI build.

For 13 nightly builds, however, we only incorporate 12 source code that has been through the peer 11 review process and is deemed ready for testing.

This 10 way, the nightly build always contains build 9 that is 'feature ready' for testing, while 8 the CI build contains features that while 7 functional (to the extent that the unit 6 tests pass) may not be ready to send the 5 to the test group.

The test groups writes 4 new CRs exclusively from one of the nightly 3 builds as opposed to the CI build, although 2 those are also available for informal exploratory 1 type testing.

Yes, if you have a process you want attached 4 to a build, but it is resource heavy. For 3 example, on my team we run JTest during 2 the nightly build. We can't run it during 1 the day because:

1. It requires a lot of resources, which may not be available
2. It takes 4 hours to complete each time

If you have a nice robust CI process in 2 place a "nightly" is still useful.

1. As mentioned, a "nightly" build can do exhaustive tests and perhaps some high-level system tests. End-to-end stuff.
2. The concept of a "nightly" build is easily understood by everyone in the organization. If you have trouble communicating CI builds out to other groups (for example, a QA group that doesn't have the same handle on Agile that the Dev group might) a "nightly" is a powerful and simple concept.
3. If your nightly is a separate set of resources, it can be managed separately and used to cut "gold" images with some claims to software integrity. For example, developer writes code, some trusted build system that dev can't touch builds it, QA tests the gold build and signs it. In such a situation, the nightly build functions like a production build system.

Just some 1 thoughts.

In my professional opinion, the only reason 8 to use nightly builds is when the build 7 process takes so long that it can't complete 6 in a "reasonable" amount of time.

For example, if 5 your build process takes 5 hours to complete 4 there is really no reason to do a build 3 on check in.

Beyond that, there is so much 2 value in knowing as soon as possible when 1 a build fails that it overrides other concerns.

It depends on the purpose and length of 41 each of your builds. Basically, you should 40 identify what you are trying to learn from 39 CI and decide if it is worth while spending 38 the resources on running multiple builds.

We 37 used continuous integration at my last job 36 for a few different purposes.

First, we used 35 it to make sure that the repository and 34 thus the developers always had a version 33 of the code that compiled. Few things are 32 worse for team members than having to manage 31 another person's broken changes through 30 commenting, uncommenting, reverting and 29 merging, because one person checked in bad 28 code. For this, we had a build that ran 27 instantaneously with no tests or other validation 26 so we knew as soon as possible if the code 25 was safe to update. Builds usually took 24 about ten minutes and the machine was probably 23 running around 50% on a normal workday. No 22 documentation was generated here, just a 21 quiet pass or a loud fail siren.

Second, we 20 wanted to know as soon as possible whether 19 or not any rules were broken. The quicker 18 that you find a broken rule, the easier 17 it is to fix. For this purpose, we had 16 a separate machine that ran a full build 15 and validation of the code. This machine 14 was running 12-14 hours a day continuously 13 on a normal workday. Email status of the 12 build was sent out describing broken unit 11 tests, code compliance, etc.

We stopped there 10 as far as automatically triggered builds 9 go. A nightly build on top of that seemed 8 a bit extreme for us. But I suppose if 7 you wanted to have a snapshot build archived 6 daily, you may want to schedule a third 5 build with the extra steps required for 4 that. Though, we did have another build 3 that wrapped up and archived our QA deployment 2 artifacts for quick and easy deployment, but 1 we only ever manually triggered that one.

I think the other posts cover the common 12 reasons, like having a build process that 11 takes "too long" or having to run only a 10 subset of tests during the CI build. But 9 there's another reason which is political.

In 8 some organizations the official builds are 7 handled by a minimally responsive build/infrastructure/release 6 management/SCM team. In these cases you 5 might put them in charge of the nightly 4 build and then run the CI build out of development. This 3 avoids a fight because their build remains 2 "the official build" and your CI build give 1 you the feedback you need.

We have both continuous integration and 11 nightly builds in place. They serve two 10 different purposes.

Our continuous integration 9 mechanism builds the software and runs unit 8 tests under the continuous integration suite.

Our 7 nightly build tags the source under version 6 control, builds the software, runs the unit 5 tests under nightly build suite. The software 4 built here is then used in various system 3 tests and stress tests.

I think that one 2 of the main differentiator for nightly build 1 is system tests.