ccache in CI

LogicParrot · 02-05-2016, 06:03 AM

(02-05-2016, 05:29 AM)bearbin Wrote: ccache is great when a clearout is as simple as `make clean`, it's not so great when something changes and you have to remote into the server and clear it out. (If I could I would spin up a new VM every time like travis, but that's not easy to do)

The server isn't running out of capacity, and although fast builds are nice it's a lot more effort.

I fail to see what exactly is the extra effort required. I did not understand your first point about make clean and accessing the remote server. You can easily clear ccache, but I fail to see why you'd need to. Mind elaborating?

bearbin · 02-05-2016, 07:32 AM

Can you guarantee that ccache will not fail when major changes are made to the code, and it won't carry over any state from old compilations?

If it won't then it's probably fine, but any type of caching adds opportunity for statefullness to creep in. The same build should be produced from fresh source as from the state in the buildserver.

LogicParrot · 02-05-2016, 08:20 AM

(02-05-2016, 07:32 AM)bearbin Wrote: Can you guarantee that ccache will not fail when major changes are made to the code, and it won't carry over any state from old compilations?

If it won't then it's probably fine, but any type of caching adds opportunity for statefullness to creep in. The same build should be produced from fresh source as from the state in the buildserver.

Yes. This is guaranteed by design, ccache guarantees that the code will compile just as if ccache was not there at all. In other words, you should not worry about statefullness creeping in.

Quote:ccache has been carefully written to always produce exactly the same compiler output that you would get without the cache. The only way you should be able to tell that you are using ccache is the speed. Currently known exceptions to this goal are listed under BUGS. If you ever discover an undocumented case where ccache changes the output of your compiler, please let us know.

The exception to this "as-if-no-statefullness" principle are bugs. Currently there are two known bugs:

1. "ccache doesn’t handle the GNU Assembler’s .incbin directive correctly. This directive can be embedded in the source code inside an asm statement in order to include a file verbatim in the object file. If the included file is modified, ccache doesn’t pick up the change since the inclusion isn’t done by the preprocessor. A workaround of this problem is to set extra_files_to_hash to the path of the included file."

2. "There is a catch with the direct mode: header files that were used by the compiler are recorded, but header files that were not used, but would have been used if they existed, are not. So, when ccache checks if a result can be taken from the cache, it currently can’t check if the existence of a new header file should invalidate the result. In practice, the direct mode is safe to use in the absolute majority of cases."

LogicParrot · (This post was last modified: 02-06-2016, 04:22 AM by LogicParrot.)

I might roll out a temporary build server soon. It will be far less ambitious yours: Just one build type on one slave. I just want to tinker with Jenkins and learn it. I'll try it with ccache enabled and report back if this causes trouble.

bearbin · 02-06-2016, 02:32 AM

Well, if it's guaranteed by design I should try it out. It isn't too much effort to just enable it on one build.

bearbin · 02-06-2016, 02:56 AM

I enabled ccache, and there are errors: https://builds.cuberite.org/job/Cuberite...84/console

I fixed one typo in the docs that fixed on error, but there are more problems.

LogicParrot · 02-06-2016, 04:02 AM

(02-06-2016, 02:56 AM)bearbin Wrote: I enabled ccache, and there are errors: https://builds.cuberite.org/job/Cuberite...84/console

I fixed one typo in the docs that fixed on error, but there are more problems.

Are you sure this is a ccache thing? Appveyor failed too for that particular Pull Request.

bearbin · 02-06-2016, 04:58 AM

That build is not for the PR, it's from before the PR.

Before I got ccache, there was no error.

LogicParrot · 02-06-2016, 08:26 AM

(02-06-2016, 04:58 AM)bearbin Wrote: That build is not for the PR, it's from before the PR.

Before I got ccache, there was no error.

How did you configure ccache? I could help pinpoint the issue.
I think the symlink method is the most hassle free.

bearbin · 02-06-2016, 07:07 PM

I changed the compilers from "clang" to "ccache clang" and "clang++" to "ccache clang++".