OpenFOAM and Subversion

1 Local

We assume you have already created a subversion repository named "openfoam". To check out a working copy using linux you must have subversion installed and your machine must be able to resolve an IP address for the server on which it is located:

File: /etc/hosts

...
aaa.bbb.ccc.ddd <nameServer>
...

Create a directory such as REPO=$HOME/svn-openfoam and if you are using the svnserve server:

svn co svn://machserv/openfoam $HOME/svn-openfoam

1.1 Advantages

OpenFOAM creates compiled binary files interdispersed within its source tree. It is also not "svn-aware". You normally don't want all those binary files and associated make-related files to be versioned within the repository. There are a few reasons for this.

When there is a recompile, for example, wmake doesn't use svn commands when it deletes or adds files and directories, requiring you to make manual corrections to remove conflicts with the repository.
An svn repository is something like a "black hole". You really should be particular about what you put into it - you can never delete anything from it (of course you can always export stuff out of it into a new repo, but you don't want to be doing that all the time!). If you start storing stuff like compiled code and non-critical simulation data you'll find your bloated repo taking up unnecessary space on your server, interactions with the repo taking longer and longer and checkouts burning bandwidth needlessly over remote connections.
Because it works by seeking out and remembering diffs, svn works most efficiently when you use it with text files.

So the idea is that you want your repo only to contain source code, scripts and configuration files, where possible. Subversion ignore patterns permit you the convenience of using your svn working copy as your build tree.

Workflow goes something like:

Check out a working copy of the repository. If you set up your repo as suggested here, this will give you access to the OpenFOAM source tree of your choice (be it an official release or local development version).
Set up your local ~/OpenFOAM directory to use this source tree, including the necessary symlinks,
If the ignore patterns on the source tree are properly set, performing full or partial compilation of executable and library code will create create or modify files and directories (e.g. dependency files *.dep and code containers such as linux64DPOpt/) that can live happily in your subversion working copy without being versioned and thereby synced with the repository,
Changes to the source code will require a local full or partial recompile, but the difference is that now the source tree is now under full version control whilst simultaneously acting as your local build tree.

This guide goes through a typical set up process for one subversion/OpenFOAM arrangement. Much of this is a matter of personal preference.

1.2 Loading official releases into the repository

Official releases go into the $REPO/branches/ directory. Subversion can attach an svn:ignore property to a repository directory. However, the functionality is not entirely intuitive, at least for new users.

1.2.1 Initial upload

A text file was created:

File: $REPO/admin/ignore-patterns.txt

linux*
lnInclude*
*.dep
bin
*log
*logs
processor*

which was intended to ensure that directories with these name patterns would be ignored during svn add.

The source-only tarball OpenFOAM-1.4.1.General.gtgz was downloaded from the OpenCFD website and expanded into $REPO/branches. The tree was then added to the working copy:

$REPO/branches > svn add OpenFOAM-1.4.1

Everything gets added. Then the latest ignore patterns were propagated throughout the working copy:

$REPO/ > svn propset -R svn:ignore -F admin/ignore-patterns.txt .

When the big commit was made, however,

$REPO/ > svn ci -m 'Added OpenFOAM-1.4.1 official release'

even though the svn:ignore property had been properly propagated across all tree directories, evidently the svn commit did not consult it when adding directories that were not meant to be added. Two possible ways around this are outlined below.

Before compiling stuff in the svn tree, its a good idea to update your ignore patterns, e.g. using this script:

File: $REPO/update-ignores.sh

#!/bin/bash
echo "Propagating ignore patterns across entire tree..."
svn propset -R svn:ignore -F admin/ignore-patterns.txt .

1.2.2 Manually remove unwanted bits first

Manually remove all unwanted directories and files before the commit. This was the preferred method here. Anything that was redundant, likely to be rarely used, or bloated was culled, including:

.../lam-a.b.c/ (not needed)
openmpi-a.b.c/ (use native installation)
zlib-a.b.c/ (use native)
ccm26ToFoam/ (might be needed very occasionally but includes a bloated Star-CCM I/O binary set)
Doxygen/ (huge automated documentation file, can be regenerated if needed, available online)

GUI components
mico-a.b.c/ (needed only by FoamX)
FoamX/
Java/
patchTool/

Some of these, such as openmpi were needed but a single system-wide installation of these packages was preferred.

One of the main problems for svn later during compilation are lnInclude directories. If such a directory is needed but already present during a wmake, it will be deleted without using svn, leading to a problem when updating from the repo. These directories are not present in the source-only tarball.

1.2.3 Automate addition with a script

Have a script go through each directory in the tree $TREE/ and first add just the directory (not its contents) using svn add -N, then attach the svn:ignore property as

svn propset svn:ignore ADD <dir>

Once committed, then re-traverse the directories and within each execute:

svn propset svn:ignore ADD "*" .

Then finally return one level up from the top directory $TREE/ and

svn add $TREE

This should (hopefully) only add those files we do not want to ignore, but has not been tested and yes, might not actually work.

1.3 Setup on local machine

This example uses the bash shell. Lets identify the three main directory trees:

The repository (or "repo"), which is the subversion database located either (usually) remotely or locally,
Your working copy of the repo, which is a snapshot of that archive on first checkout, and thereafter can be updated to reflect the repo. Changes to files and directories made in the working copy are only sent to the repo with an explicit svn commit or svn ci command,
Your OpenFOAM root directory, generally $HOME/OpenFOAM/, which can contain multiple versions of OpenFOAM in installation directories. In the example here, these are $WM_PROJECT_DIR =
- $HOME/OpenFOAM/OpenFOAM-dev (the local development version)
- $HOME/OpenFOAM/OpenFOAM-1.4.1 (an official OpenCFD release)

Choosing between these can be done easily by manually changing the environment call in your local shell config file

File: $HOME/.bashrc

OPENFOAM_VER="dev" # ( "dev" | "1.4.1" )
source $HOME/OpenFOAM/OpenFOAM-${OPENFOAM_VER}/.OpenFOAM$-{OPENFOAM_VER}/bashrc

To ensure this change takes effect, issue source $HOME/.bashrc at the command line. These two versions just reflect what we have in the repository. The the repo the value of $$WM_PROJECT_VERSION in $REPO/trunk/OpenFOAM-dev/.OpenFOAM-dev/.bashrc has to something like "dev" in which case the local development directory would be $HOME/OpenFOAM/OpenFOAM-dev.

It should be emphasised that the repo and working directory are "alive" and change over time. The installation directories are relatively static - you set them up initially and then the only changes are associated with compilation. The primary role of the installation directories is:

Set OpenFOAM environment variables and global compilation rules
Contain build scripts (i.e. wmake) and some files resulting from the compile process
Point to working copy directories (e.g. src etc)

We will set up an installation directory for an OpenCFD release, from the repo. First, create the installation directory and export the tree:

svn export $REPO_WC/branches/opencfd/OpenFOAM-1.4.1 $HOME/OpenFOAM/OpenFOAM-1.4.1

This is basically the same as cp -r, except the .svn directories are removed. Similarly to export the local development version:

svn export $REPO_WC/trunk/OpenFOAM-dev $HOME/OpenFOAM/OpenFOAM-dev

Copy into $HOME/OpenFOAM/ the directory $REPO/branches/OpenFOAM-1.4.1/. Make local adjustments to .OpenFOAM-1.4.1/bashrc, for example. Then replace each of the major subdirectories (e.g. src/) with a symbol link, e.g.:

hoogland@onceler ~/OpenFOAM/OpenFOAM-1.4.1 $ rm -fr src
hoogland@onceler ~/OpenFOAM/OpenFOAM-1.4.1 $ ln -s ~/svn-openfoam/trunk/openfoam/src src

except lib which contains only compiled code, until you have something like:

hoogland@onceler ~/OpenFOAM/OpenFOAM-1.4.1 $ la                             
total 348K
drwxr-xr-x 5 hoogland hoogland 4.0K Oct  2 15:29 .
drwxr-xr-x 4 hoogland hoogland 4.0K Oct  2 13:07 ..
drwxr-xr-x 4 hoogland hoogland 4.0K Oct  2 12:16 .OpenFOAM-1.4.1
-rw-r----- 1 hoogland hoogland 7.1K Aug 13 21:00 .bashrc
-rw-r----- 1 hoogland hoogland 7.1K Aug  1 12:40 .cshrc
-rw-r----- 1 hoogland hoogland    9 Aug  2 21:31 .timeStamp
-rwxr-x--- 1 hoogland hoogland  163 Apr  5  2006 Allwmake
-rw-r----- 1 hoogland hoogland  18K Jun 21  2005 COPYING
-rw-r----- 1 hoogland hoogland 8.6K Jul 31 13:28 README
-rw-r----- 1 hoogland hoogland 6.1K Aug  2 21:28 ReleaseNotes-1.4.1
lrwxrwxrwx 1 hoogland hoogland   55 Oct  2 12:33 applications -> /home/hoogland/svn-openfoam/trunk/openfoam/applications
lrwxrwxrwx 1 hoogland hoogland   46 Oct  2 12:53 bin -> /home/hoogland/svn-openfoam/trunk/openfoam/bin
lrwxrwxrwx 1 hoogland hoogland   46 Oct  2 12:53 doc -> /home/hoogland/svn-openfoam/trunk/openfoam/doc
drwxr-xr-x 3 hoogland hoogland 4.0K Oct  2 15:29 lib
lrwxrwxrwx 1 hoogland hoogland   46 Oct  2 15:29 src -> /home/hoogland/svn-openfoam/trunk/openfoam/src
lrwxrwxrwx 1 hoogland hoogland   52 Oct  2 12:54 tutorials -> /home/hoogland/svn-openfoam/trunk/openfoam/tutorials
drwxr-xr-x 5 hoogland hoogland 4.0K Oct  2 15:21 wmake
-rw-r--r-- 1 hoogland hoogland 258K Oct  2 16:08 wmake.log

The wmake directory is left intact, this is important. The wmake script itself traverses all directories in the svn tree

File: wmake/wmake

154 #------------------------------------------------------------------------------
155 # Recurse the application directories tree
156 #------------------------------------------------------------------------------
157 
158 if [ "$makeOption" = "all" ]
159 then
160     if [ -e Allwmake ]
161     then
162         ./Allwmake
163         exit $?
164     elif [ ! -d $MakeDir ]
165     then
166         $make -f $WM_DIR/MakefileApps FOAM_APPS="`find . -maxdepth 1 \( -type d -a ! -name "." -a ! -name "Make" \)  -printf "%f "`"
167         exit $?
168     fi
169 
170     # This is the end of the recursion down the application directories tree
171     # so remove the "all" option so that the call to make builds the application
172     makeOption=
173 fi

so to save it the effort of looking inside each of many .svn/ directories, change it to this,

File: wmake/wmake

164     elif [ ! -d $MakeDir ]
165     then
166         $make -f $WM_DIR/MakefileApps FOAM_APPS="`find . -maxdepth 1 \( -type d -a ! -name "." -a ! -name "Make" -a ! -name ".svn" \)  -printf "%f "`"
167         exit $?
168     fi

Delete everything in lib/, we will rebuild this in a moment. Note that it is also perfectly fine to make a versioned directory lib in the repository, with its subdirectory e.g. linux64DPOpt residing in the working copy but not in the repo by virtue of the ignore pattern "linux*". In this case you'd need to create a symlink as for the other working copy directories. Either way, compiled libraries will reside in libWe assume you have already followed the OpenCFD README to set up sourcing of the OpenFOAM configuration settings via:

source $HOME/OpenFOAM/OpenFOAM-1.4.1/.OpenFOAM-1.4.1/bashrc

in your <tt>$HOME/.bashrc (or shell equivalent). If your $WM_ARCH is not listed in wmake/rules/, then make a symlink to the closest equivalent (in this case pointing $WM_ARCH="linux64" to linux64Gcc):

hoogland@onceler ~/OpenFOAM/OpenFOAM-1.4.1/wmake/rules $ la
total 48K
drwxr-xr-x 12 hoogland hoogland 4.0K Oct  2 13:58 .
drwxr-xr-x  5 hoogland hoogland 4.0K Oct  2 15:21 ..
drwxr-xr-x  2 hoogland hoogland 4.0K Oct  2 13:48 General
lrwxrwxrwx  1 hoogland hoogland   10 Oct  2 13:58 linux64 -> linux64Gcc
drwxr-xr-x  2 hoogland hoogland 4.0K Oct  2 13:48 linux64Gcc
drwxr-xr-x  2 hoogland hoogland 4.0K Oct  2 13:48 linuxGcc
drwxr-xr-x  2 hoogland hoogland 4.0K Oct  2 13:48 linuxI32
drwxr-xr-x  2 hoogland hoogland 4.0K Oct  2 13:48 linuxI64
drwxr-xr-x  2 hoogland hoogland 4.0K Oct  2 13:48 linuxIA64Gcc
drwxr-xr-x  2 hoogland hoogland 4.0K Oct  2 13:48 linuxIA64I64
drwxr-xr-x  2 hoogland hoogland 4.0K Oct  2 13:48 sgi64Gcc
drwxr-xr-x  2 hoogland hoogland 4.0K Oct  2 13:48 sgiN32Gcc
drwxr-xr-x  2 hoogland hoogland 4.0K Oct  2 13:48 solarisGcc

Compile the entire package using:

hoogland@onceler ~/OpenFOAM/OpenFOAM-1.4.1/ $ ./Allwmake | tee wmake.log 2>&1

A bunch of new directories and files will appear in your $REPO, but most should be ignored by svn in accordance with the ignore-patterns.txt file. To see the ignored files and directories:

$REPO > svn status --no-ignore | grep '^I'

1.4 Using native versus drop in packages

Using the drop-in packages provided by OpenCFD such as gcc and openmpi is helpful if you do not have root access or if there are lots of system users who might get cranky with a change, but otherwise it can make sense to reduce duplication by installing/upgrading the necessary packages system-wide. OpenFOAM compiles with most recent versions of gcc but sometimes different systems put key files from these packages in different locations, in a way OpenFOAM cannot always anticipate. For example, when installing openmpi on a Gentoo 64 bit system using the native package manager:

# emerge openmpi

the mpi.h is placed in /usr/include and libmpi.so is placed in /usr/lib64, just where OpenFOAM expects them. When installing openmpi system-wide on a FC5 64 bit system using the native package manager, however,

# yum install openmpi-devel

puts the mpi.h header file in /usr/include/openmpi/ and libmpi.so in /usr/lib64/openmpi. This borks compilation of the src/Pstream/mpi library, and a fix is simply a matter of creating appropriate symlinks:

# ln -s /include/openmpi/mpi.h /include/mpi.h
# ln -s /usr/lib/libmpi.so -> /usr/lib64/openmpi/libmpi.so

Unfortunately, in this particular case, there is also a difference between the mpi.h files anticipated by OpenFOAM for version 1.1.2 (the Gentoo package in this case), but not version 1.0.1 (the FC5 package here). For the latter:

File: /usr/include/mpi.h (v1.0.1 FC5)

...
  74 /* Whether we want MPI cxx support or not */
  75 #define OMPI_WANT_CXX_BINDINGS 1
...
1757 /*
1758  * Conditional MPI 2 C++ bindings support.  Include if:
1759  *   - We want C++ bindings support
1760  *   - We are not building OMPI itself
1761  *   - We are using a C++ compiler
1762  */
1763 
1764 #if defined(OMPI_WANT_CXX_BINDINGS) && OMPI_WANT_CXX_BINDINGS && !OMPI_BUILDING
1765 #if defined(__cplusplus) || defined(c_plusplus)
1766 #include "mpi/cxx/mpicxx.h"
1767 #endif
1768 #endif

while for the former:

File: /usr/include/mpi.h (v1.1.2 Gentoo)

...
  75 /* Whether we want MPI cxx support or not */
  76 /* #define OMPI_WANT_CXX_BINDINGS 1 */
...
1750 /*
1751  * Conditional MPI 2 C++ bindings support.  Include if:
1752  *   - The user does not explicitly request us to skip it (when a C++ compiler
1753  *       is used to compile C code).
1754  *   - We want C++ bindings support
1755  *   - We are not building OMPI itself
1756  *   - We are using a C++ compiler
1757  */
1758 #if !defined(OMPI_SKIP_MPICXX) && OMPI_WANT_CXX_BINDINGS && !OMPI_BUILDING
1759 #if defined(__cplusplus) || defined(c_plusplus)
1760 #include "ompi/mpi/cxx/mpicxx.h"
1761 #endif
1762 #endif

This version uses the flag macro OMPI_SKIP_MPICXX, anticipated by OpenFOAM in:

File: $WM_PROJECT_DIR/wmake/rules/linux64/mplibOPENMPI

PFLAGS     = -DOMPI_SKIP_MPICXX
PINC       = -I$(OPENMPI_ARCH_PATH)/include
PLIBS      = -L$(OPENMPI_ARCH_PATH)/lib -lmpi

To get the FC5 v1.0.1 header to compile properly, line 75 must be commented out and a flag added to the wmake rules for openmpi:

File: $WM_PROJECT_DIR/wmake/rules/linux64/mplibOPENMPI

PFLAGS     = -DOMPI_SKIP_MPICXX -DOMPI_WANT_CXX_BINDINGS=0
PINC       = -I$(OPENMPI_ARCH_PATH)/include
PLIBS      = -L$(OPENMPI_ARCH_PATH)/lib -lmpi

If you aren't prepared for this kind of fiddling, using the OpenCFD drop-ins might be best for you :)

1.5 Keeping Cases in repository

It is easy and wise to keep your case in a repository. In this case, a separate repo is used, $REPO2. Again, the main things to do are:

Ensure you only svn add add the minimum files/directories necessary for each case,
Update your ignore-patterns.txt file as necessary to ignore files/directories you may not have initially anticipated.

In $REPO2/trunk/openfoam/runs you can keep ignore-patterns.txt and an update-patterns.sh script, run periodically. My current ignore patterns are:

File: ignore-patterns.txt

[0-9]*
VTK*

This is a work in progress, stay tuned. Hoogs 11:12, 8 Oct 2007 (CEST)

2 Community

An OpenFOAM-extend project was announced on 27 September 2007.