How to Compile LAMMPS on HPC Clusters using Mighty Makefiles!!

6 min readMay 31, 2023

Memes apart, compiling LAMMPS is non-trivial if you don’t know what is going on during the whole compilation process or if you want to compile it on your school’s HPC. Compiling LAMMPS with an accelerator package is even trickier and needs detailed attention. Here, I will try to show the process of compiling LAMMPS on your HPC (high-performance computing) cluster that uses a Unix-based system.

LAMMPS is a massively parallel molecular simulation package written in C++. LAMMPS has multiple packages for different types of simulations. Compiling LAMMPS on Windows is just as easy as it can get. However, compiling it on a Unix-based system needs more attention and understanding of what’s happening in the background. Let’s start the process.

STEP 1:

Change the directory to where you want to install LAMMPS.

cd /work/abcd/lammps_hpc_2023

STEP 2:

Clone the LAMMPS repo from its GitHub page.

git clone -b stable https://github.com/lammps/lammps.git LAMMPS_dir

Here, “LAMMPS_dir” is your directory name (change as you wish)

However, if you want to compile an earlier version, then use the following command (example: compiling 4 Nov 2016 version).

git clone -b stable_4Nov2016 https://github.com/lammps/lammps.git LAMMPS_dir

STEP 3:

Move to the ‘src’ directory:

cd LAMMPS_dir/src/

STEP 4:

Purge any old files (if any)

make purge

STEP 5:

Clean old files (obj files)

make clean-all

STEP 6:

Check your system's available C/C++ compiler, wrapper, and linker. For example, for serial runs, use GNU compiler: ‘g++’ or ‘gcc’; for MPI runs, use ‘mpicxx’ (OpenMPI C++ wrapper compiler) or ‘mpicc’ (Open MPI C wrapper compiler). mpicxx compiles and links MPI programs written in C++. Since LAMMPS source files are written in C++, we need mpicxx or gcc. mpicxx is a wrapper for the underlying C++ compiler and it links relevant libraries to its different layers. So, if you need MPI runs, you better start with a wrapper that has access to your C++ compiler. Using gcc for compiling LAMMPS will not allow parallel runs.

which gcc
which mpicxx
whereis mpicxx

ASIDE 1:

A compiler wrapper does not actually compile or link to the required libraries, rather it modifies the command-line arguments and adds optional flags (arguments passed to the actual compiler) to the actual linkers/compiler. This is why the term ,‘wrapper’, is used. It wraps around the compiler and modifies it on the go.

If you want to simulate a 2D graphene sheet’s characteristics under simple shear loading, then you might need to run it in parallel as a serial run can only delay the results. Let’s say you break the whole process into 32 tasks and you are using 2 nodes; so each node will handle 16 tasks and if each node has a maximum of 32 cores, then each task in each node will have access to 2 cores — 2 cores/task/node = 64 cores being used in 32 tasks in 2 nodes in total. So, you need a lot of communication between the nodes. Parallel runs allow this communication and make calculations faster. If you, for some reason, decide to run only 1 task for the entire simulation, you will be running it in serial. If you look at the log file of LAMMPS, you will see if you’re running in serial or parallel.

Serial:

80.5% CPU use with 1 MPI tasks x no OpenMP threads

Parallel (MPI):

Performance: 0.962 ns/day, 24.938 hours/ns, 111.387 timesteps/s
100.0% CPU use with 32 MPI tasks x no OpenMP threads

Let’s get back to compiling LAMMPS on your HPC. If you write mpi and press tab twice, a lot of options will pop up and you can see all the available options for your MPI compiler wrapper.

mpi ##press tab twice and see options below
mpicc          mpiexec.hydra  mpifc          mpiicc         mpirun
mpicxx         mpif77         mpigcc         mpiicpc        mpitune
mpiexec        mpif90         mpigxx         mpiifort       mpitune_fast

## Do the same for gcc:
gcc ##press tab twice and see options below
gcc           gcc-ar        gcc-nm        gcc-ranlib
gcc-9         gcc-ar-9      gcc-nm-9      gcc-ranlib-9

If you want to know all the directories of that module, then write:

module show gcc

This will give you all the locations of that module: GCCHOME, PATH, CPATH, LIBRARY_PATH, LD_LIBRARY_PATH, and so on. You might (!!) need this information during compiling LAMMPS (for compiling LAMMPS with an accelerator — GPU package).

STEP 7:

Look for the ‘Makefile.mpi’ file in the /src/MAKE/ directory and open it in the vi editor.

cd src/MAKE/
vi Makefile.mpi

Make sure your CC is set to the compiler you want to use (in my case, it is mpicxx, the default GNU MPI C++ compiler wrapper), but I also used INTEL MPI compiler wrappers as well before. Let me show you how they look —

Default GNU C++ Compiler:

Intel C++ Compiler:

CC = C/C++ compiler
CCFLAGS = flags passed to CC (can be overridden from the command line)
Make sure your HPC uses C++ or a standard ISO C++ version to ensure proper compilation of LAMMPS. In my case, it is C++11 (it might be C++17 in your case).
SHFLAGS = shell flags/functions for better handling
DEPFLAGS = dependency flags

LINK and CC should be same

The rest of the makefile is self-explanatory and that requires its own article since makefiles can be pretty confusing (I still struggle with it).

Although my g++ version is 9.3, it can utilize C++11 features by enabling the -std=c++11 flags. g++ version 9.3 can use features from the C++11 standard. If your g++ version does not support C++11 standards, then change it to some other standard (C++03) or update your g++ version to something. In the first case, you need to do the following:

CCFLAGS = -g -O3 -std=c++03

This means you have been using an old g++ version!!!

Or, in the second case, where you only need to update your g++ version, you don’t need to change the Makefile.mpi’s CCFLAGS anymore. Just make sure the g++ version is properly working.

g++ 4.8.1 onwards support C++11 standard features. Refer to this website if you want to find compatibility between your g++ and c++XX standard:

C++ Standards Support in GCC

std= command-line option.For information about the status of C++ defect reports, please see this page. For information…

gcc.gnu.org

STEP 8 (Finally):

If you need to install any packages then do it using make yes-PACKAGENAME. First, go back to the ‘src/’ directory and use the following:

cd ..                     ## gets you to src directory
make yes-COMPRESS         ## compile COMPRESS package
make yes-MANYBODY         ## compile MANYBODY package
make yes-KSPACE           ## compile KSPACE package (required for PHONON)
make yes-PHONON           ## compile PHONON package

STEP 9:

If you’re done with the optional package installation, then check them before compiling LAMMPS (this command will show the package status):

make ps

STEP 10:

Now that you’ve set the proper compiler and linker settings, just go and make it using:

make mpi

ASIDE 2:

If you need to change JPG, FFT, or MPI libraries, just change their “INC”, “PATH”, and “LIB” args in the ‘Makefile.mpi’ file.

INC = include header files
PATH = path to the library functions
LIB = name of the library functions

I will address this in a separate article in the near future, hopefully. I think that’s all for today. Let me know if you stumble upon any issues. I am still learning the nuances and trying to make notes for everyone so that we all can learn without suffering. If you know a better and easier way to compile LAMMPS on a specific HPC, let me know.

Also, for completeness, LAMMPS already has machine-specific makefiles, which is a pretty cool thing, right? You don’t need to understand the makefiles, just use the one that is applicable to your HPC system, right? Right? xD