【2.0】amber的安装

December 05, 2019 molecular_dynamics 阅读量：次

amber18在centos7的安装
amber20在centos7的安装

一、Amber18 (centos7)

1.1 下载amber

Amber18.tar.bz2
AmberTools18.tar.bz2

终端输入：

cd /data/software/amber

tar -xjf Amber18.tar.bz2
tar -xjf AmberTools18.tar.bz2

两个压缩包会解压到一个名为amber18的文件夹里

1.2 设置AMBERHOME主目录环境变量：　

vim ~/.bashrc

export AMBERHOME=/mnt/nfs/data/software/amber/amber18

然后使其生效

source ~/.bashrc

1.3 安装依赖的软件及软件库：

sudo yum install gcc gcc-gfortran gcc-c++ flex tcsh zlib-devel bzip2-devel libXt-devel libXext-devel libXdmcp-devel tkinter openmpi openmpi-devel perl perl-ExtUtils-MakeMaker

yum -y install tcsh make  gcc gcc-gfortran gcc-c++ which flex bison patch bc libXt-devel libXext-devel perl util-linux wget bzip2 bzip2-devel zlib-devel

yum -y install openmpi openmpi-devel environment-module

1.4 切换到amber目录，执行编译脚本：

cd $AMBERHOME
$ ./configure --help　　　　　　　　　　　　　　　　

#执行此命令可以看到不同的编译模式，不同的参数设置，我以前也什么都不懂，看完后理解就好很多

$ ./configure gnu　　　　　　　　　　　　　　　　　　# 使用gcc编译器编译（gnu包含了gcc套件）
此命令会产生出amber.sh和amber.csh文件，这两个文件可以用于添加amber所需的环境变量。

[显卡驱动及cuda9.0安装：设置环境变量，尤其是要设置CUDA_HOME(先要安装cuda)：

#CUDA （amber18不能支持cuda10）

export CUDA_HOME=/usr/local/cuda-9.0;
export PATH=$CUDA_HOME/bin:$PATH;
export LD_LIBRARY_PATH=$CUDA_HOME/lib:$LD_LIBRARY_PATH;

报错：

Testing the g++ compiler with boost:
     g++ -fPIC  testp.cpp -lboost_thread -lboost_system -o testp

Warning: Could not link to boost. Ensure boost-thread, boost-system and boost headers are installed.
Warning: Failed command:
     g++ -fPIC  -lboost_thread -lboost_system -o testp testp.cpp
Warning: Check boost.compile.err for error details.
Warning: Memembed functionality in packmol_memgen not available.

暂时不管上面的报错。也没查出问题

1.5 环境变量的修改

configure命令会产生出amber.sh和amber.csh文件，这两个文件可以用于添加amber所需的环境变量。

If you use a Bourne shell (e.g., bash, sh, zsh, etc.), source the
/mnt/nfs/data/software/amber/amber18/amber.sh file in your shell. Consider adding the line
  test -f /mnt/nfs/data/software/amber/amber18/amber.sh && source /mnt/nfs/data/software/amber/amber18/amber.sh
to your startup file (e.g., ~/.bashrc)

If you use a C shell (e.g., csh, tcsh), source the
/mnt/nfs/data/software/amber/amber18/amber.csh file in your shell. Consider adding the line
  test -f /mnt/nfs/data/software/amber/amber18/amber.csh && source /mnt/nfs/data/software/amber/amber18/amber.csh
to your startup file (e.g., ~/.cshrc)

或者自己手动添加到：

vim ~/.bashrc 中：

export AMBER_PREFIX="/mnt/nfs/data/software/amber/amber18"
export AMBERHOME=/mnt/nfs/data/software/amber/amber18
export PATH="${AMBER_PREFIX}/bin:${PATH}"
# Add location of Amber Python modules to default Python search path
if [ -z "$PYTHONPATH" ]; then
    export PYTHONPATH="${AMBER_PREFIX}/lib/python2.7/site-packages"
else
    export PYTHONPATH="${AMBER_PREFIX}/lib/python2.7/site-packages:${PYTHONPATH}"
fi
if [ -z "${LD_LIBRARY_PATH}" ]; then
   export LD_LIBRARY_PATH="${AMBER_PREFIX}/lib"
else
   export LD_LIBRARY_PATH="${AMBER_PREFIX}/lib:${LD_LIBRARY_PATH}"
fi

顺便把cuda也放进去：

export CUDA_HOME=/usr/local/cuda-9.0
export PATH=$CUDA_HOME/bin:$PATH
export LD_LIBRARY_PATH=$CUDA_HOME/lib:$LD_LIBRARY_PATH

source ~/.bashrc 使环境变量生效。

1.6 编译安装与测试：

make install　　# 编译安装
make test　　# 编译测试

测试，也有部分报错：

make[2]: Leaving directory `/mnt/nfs/data/software/amber/amber18/test'
176 file comparisons passed
2 file comparisons failed
0 tests experienced errors
Test log file saved as /mnt/nfs/data/software/amber/amber18/logs/test_amber_serial/2019-10-30_19-08-33.log
Test diffs file saved as /mnt/nfs/data/software/amber/amber18/logs/test_amber_serial/2019-10-30_19-08-33.diff
make[1]: Leaving directory `/mnt/nfs/data/software/amber/amber18/test'

Summary of AmberTools serial tests:

2245 file comparisons passed
15 file comparisons failed
0 tests experienced errors
Test log file saved as /mnt/nfs/data/software/amber/amber18/logs/test_at_serial/2019-10-30_17-58-51.log
Test diffs file saved as /mnt/nfs/data/software/amber/amber18/logs/test_at_serial/2019-10-30_17-58-51.diff

1.7 多线程

1.7.1 安装openmpi

如果本身系统已经装了openmpi合适版本，which mpirun命令返回openmpi的目录即可,否则：

下载openmpi安装包：

网址：http://www.open-mpi.org/software/ompi/v1.8/

版本要求在 $AMBERHOME/AmberTools/src/configure_openmpi 中有说明， mpidirs=/bin/ls -d openmpi-1.[5-9]*/ 2>/dev/null | tail -n 1

解压openmpi解压到 $AMBERHOME/AmberTools/src：　

wget -c https://download.open-mpi.org/release/open-mpi/v1.8/openmpi-1.8.8.tar.gz
tar zxvf openmpi-1.8.8.tar.gz

1.7.2 在src目录编译

cd $AMBERHOME/AmberTools/src
./configure_openmpi gnu 　　　#编译，此命令会产生出amber.sh和amber.csh文件，这两个文件可以用于添加amber所需的环境变量。

1.7.3 环境变量的修改

将$AMBERHOME/AmberTools/src/amber.sh 环境变量添加到~/.bashrc中

vim ~/.bashrc ：

export MPI_HOME=$AMBERHOME/AmberTools
export PATH=$PATH:$MPI_HOME/bin
export LD_LIBRARY_PATH=$MPI_HOME/lib:$LD_LIBRARY_PATH
export DO_PARALLEL="mpirun -np 4"　　　　

# Note the value below may depend on your MPI implementation，因为某些测试中需要用到的线程较多，所以最好设置4个或8个

1.7.7.4 配置openmpi后，回到$AMBERHOME编译amber：

cd $AMBERHOME
./configure -mpi gnu

1.7.7.5 Amber安装测试：

make install
make test

测试结果在：

176 file comparisons passed
6 file comparisons failed
15 tests experienced an error
Test log file saved as /mnt/nfs/data/software/amber/amber18/logs/test_amber_parallel/2019-10-22_13-44-58.log
Test diffs file saved as /mnt/nfs/data/software/amber/amber18/logs/test_amber_parallel/2019-10-22_13-44-58.diff
make[1]: Leaving directory `/mnt/nfs/data/software/amber/amber18/test'

Summary of AmberTools parallel tests:

1002 file comparisons passed
162 file comparisons failed
81 tests experienced errors
Test log file saved as /mnt/nfs/data/software/amber/amber18/logs/test_at_parallel/2019-10-22_13-21-14.log
Test diffs file saved as /mnt/nfs/data/software/amber/amber18/logs/test_at_parallel/2019-10-22_13-21-14.diff

报错的内容：

CPPTRAJ: CRD Replica Trajectory Run with remdout
[g02:24670] 3 more processes have sent help message help-mpi-btl-base.txt / btl:no-nics
[g02:24670] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
diffing all.dat.save with all.dat

暂时先不管报错

1.9 支持GPU

1.9.1 显卡驱动及cuda9.0安装：

见我cuda安装的博客

1.9.2 切换到amber目录，编译：

cd $AMBERHOME
make clean
./configure -cuda gnu     #不可省略
./configure -cuda -mpi gnu  #紧接着上一步

make install

如果只有./configure -cuda -mpi gnu;make install，看到的是cuda是

[sam@g02 bin]$ ll |grep cuda
-rwxr-xr-x 1 sam molinfo 5.6M Oct 31 10:03 cpptraj.MPI.cuda
-rwxr-xr-x 1 sam molinfo  91M Oct 31 10:22 pmemd.cuda_DPFP.MPI
lrwxrwxrwx 1 sam molinfo   19 Oct 31 10:12 pmemd.cuda.MPI -> pmemd.cuda_SPFP.MPI
-rwxr-xr-x 1 sam molinfo  73M Oct 31 10:12 pmemd.cuda_SPFP.MPI

接着./configure -cuda gnu;make install，就能看到了

[sam@g02 bin]$ ll |grep cuda
-rwxr-xr-x 1 sam molinfo 5.5M Oct 31 13:43 cpptraj.cuda
-rwxr-xr-x 1 sam molinfo 5.6M Oct 31 10:03 cpptraj.MPI.cuda
-rwxr-xr-x 1 sam molinfo 8.6M Oct 31 13:37 pbsa.cuda
lrwxrwxrwx 1 sam molinfo   15 Oct 31 13:53 pmemd.cuda -> pmemd.cuda_SPFP
-rwxr-xr-x 1 sam molinfo  91M Oct 31 14:04 pmemd.cuda_DPFP
-rwxr-xr-x 1 sam molinfo  91M Oct 31 10:22 pmemd.cuda_DPFP.MPI
lrwxrwxrwx 1 sam molinfo   19 Oct 31 10:12 pmemd.cuda.MPI -> pmemd.cuda_SPFP.MPI
-rwxr-xr-x 1 sam molinfo  73M Oct 31 13:53 pmemd.cuda_SPFP
-rwxr-xr-x 1 sam molinfo  73M Oct 31 10:12 pmemd.cuda_SPFP.MPI
[sam@g02 bin]$ pwd
/mnt/nfs/data/software/amber/amber18/bin

Warning: No parallel NetCDF specified (--with-pnetcdf <dir>).
       Parallel NetCDF trajectory output will not be available
       in cpptraj.MPI.

1.9.3 测试

make test.cuda_serial

测试结果：

232 file comparisons passed
       1 file comparison failed
       1 test experienced an error
Test log file saved as /mnt/nfs/data/software/amber/amber18/logs/test_amber_cuda/2019-10-31_14-16-16.log
Test diffs file saved as /mnt/nfs/data/software/amber/amber18/logs/test_amber_cuda/2019-10-31_14-16-16.diff
make[1]: Leaving directory `/mnt/nfs/data/software/amber/amber18/test'

测试：

export DO_PARALLEL='mpirun -np 2
make test.cuda_parallel

测试结果：

make[3]: [test.pmemd.cuda.gb] Error 1 (ignored)
cd gamd/rna_gb && ./Run.gb.gamd2  DPFP /mnt/nfs/data/software/amber/amber18/include/netcdf.mod
cudaDeviceEnablePeerAccess all CUDA-capable devices are busy or unavailable
cudaDeviceEnablePeerAccess all CUDA-capable devices are busy or unavailable

debug的问题

http://cseweb.ucsd.edu/classes/wi15/cse262-a/static/cuda-5.5-doc/html/cuda-runtime-api/group__CUDART__TYPES.html

原因：

This indicates that all CUDA devices are busy or unavailable at the current time. Devices are often busy/unavailable due to use of cudaComputeModeExclusive, cudaComputeModeProhibited or when long running CUDA kernels have filled up the GPU and are blocking new work from starting. They can also be unavailable due to memory constraints on a device that already has active CUDA work being performed.


[sam@g02 amber18]$ nvidia-smi -a |grep Compute
    Compute Mode                    : Exclusive_Process
    Compute Mode                    : Exclusive_Process
    Compute Mode                    : Exclusive_Process
    Compute Mode                    : Exclusive_Process
    Compute Mode                    : Exclusive_Process
    Compute Mode                    : Exclusive_Process
    Compute Mode                    : Exclusive_Process
    Compute Mode                    : Exclusive_Process


cudaComputeModeDefault = 0
Default compute mode (Multiple threads can use cudaSetDevice() with this device)
cudaComputeModeExclusive = 1
Compute-exclusive-thread mode (Only one thread in one process will be able to use cudaSetDevice() with this device)
cudaComputeModeProhibited = 2
Compute-prohibited mode (No threads can use cudaSetDevice() with this device)
cudaComputeModeExclusiveProcess = 3
Compute-exclusive-process mode (Many threads in one process will be able to use cudaSetDevice() with this device)

解决办法：

nvidia-smi -c 0  # 测试的时候改成这个

nvidia-smi -c 3  # 测试结束以后，改成这个


make test.cuda_parallel

128 file comparisons passed
0 file comparisons failed
3 tests experienced errors
Test log file saved as /mnt/nfs/data/software/amber/amber18/logs/test_amber_cuda_parallel/2019-10-31_16-30-43.log
No test diffs to save!
make[1]: Leaving directory `/mnt/nfs/data/software/amber/amber18/test'

1.4 其他问题

K80上可以使用： https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5843186/

参考集群命令：

export AMBERHOME=/data/software/amber18/ export MOE=/data/software/moe/moe2019/ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/data/software/g03-ti-lib

1.5 升级的话，可以这样来：

./update_amber --upgrade

Upgrade of AmberTools18 to AmberTools19 is complete. Please re-run configure, applying any available updates to Amber18 and/or AmberTools19, and then recompile everything.

cd $AMBERHOME
make clean
./configure -cuda gnu
./configure -cuda -mpi gnu


Checking NetCDF...
        Using bundled NetCDF library.
        Using existing NetCDF in '/mnt/nfs/data/software/amber/amber18'


Warning: No parallel NetCDF specified (--with-pnetcdf <dir>).
         Parallel NetCDF trajectory output will not be available
         in cpptraj.MPI.

Fatal Error: File 'netcdf.mod' opened at (1) is not a GFORTRAN module file

这个错误需要回到$AMBERHOME/include文件夹内，把所有的netcdf.*文件全部删除，然后再重复make clean;configure等

make install
make test

二、 amber20在centos7的安装

amber20和以前版本安装有很大差别，且自动生成安装目录amber20,安装文件采用cmake_run文件安装，每部安装要设置其中的参数

安装依赖的软件及软件库：

yum -y install tcsh make gcc gcc-gfortran gcc-c++ which flex bison patch bc libXt-devel libXext-devel perl perl-ExtUtils-MakeMaker util-linux wget bzip2 bzip2-devel zlib-devel tar

下载以及解压缩：

cd /data/software/amber
tar -xjf Amber20.tar.bz2
tar -xjf AmberTools20.tar.bz2

2.1 串行安装

cd amber20_src/build
# optional: edit the run_cmake script to make any needed changes;
# most users should not need to do this
./run_cmake
# Next, build and install the code:
make install

会自动安装到 /data/software/amber/amber20

配置环境变量：

vim /data/modulefiles/amber/amber20

#%Module 1.0
#
#  OpenMPI module for use with 'environment-modules' package:
#
#

setenv  AMBERHOME                 /data/software/amber/amber20
setenv  CUDA_HOME                 /usr/local/cuda-10.1

prepend-path  PATH   /data/software/amber/amber20/bin
prepend-path  LD_LIBRARY_PATH  /data/software/amber/amber20/lib
prepend-path  LD_LIBRARY_PATH     /data/software/g03-ti-lib
prepend-path  PERL5LIB     /data/software/amber/amber20/lib/perl
prepend-path  PYTHONPATH   /data/software/amber/amber20/lib/python3.8/site-packages

测试

module load amber/amber20
cd $AMBERHOME
make test.seria

2.2 并行安装

compile the parallel (MPI)

cd /data/software/amber/amber20_src/AmberTools/src
wget -c http://www.mpich.org/static/downloads/3.3.2/mpich-3.3.2.tar.gz
tar xzvfm mpich-3.3.2.tar.gz
./configure_mpich gnu


cd /data/software/amber/amber20_src/build
# edit the run_cmake script to set -DMPI=TRUE
./run_cmake
make install

source /home/user/amber2020/amber20/amber.sh

To run tests: Note the value below may depend on your MPI implementation

export DO_PARALLEL="mpirun -np 28"
cd $AMBERHOME
source amber.sh
make test.parallel
# Note, some tests, like the replica exchange tests, require more
# than 2 threads, so we suggest that you test with either 4 or 8
# threads as well
export DO_PARALLEL="mpirun -np 4"
make test.parallel

configure_mpich gnu这一步报错：

aclocal-1.15' is missing on your system.

解决办法: 安装automake 1.15：

直接编译（失败）

wget -c https://ftp.gnu.org/gnu/automake/automake-1.15.tar.gz
tar xzvfm automake-1.15.tar.gz
cd automake-1.15
./configure  --prefix=/usr/local/automake-1.15
make

接着报错报错（没有解决）：

Can't locate /data/src/automake-1.15/bin/aclocal in @INC (@INC contains: /

通过rpm来安装：

wget -c http://springdale.princeton.edu/data/springdale/7/x86_64/os/Computational/automake115-1.15-9.sdl7.noarch.rpm
rpm -Uvh automake115-1.15-9.sdl7.noarch.rpm
 yum install automake115

vim /etc/profile   #g01
export PATH=/usr/local/automake/1.15/bin/:$PATH

2.3 单GPU安装

export CUDA_HOME=/usr/local/cuda-10.1
cd build

# edit the run_cmake script to set -DCUDA=TRUE (you should also be using the GNU compilers here)
./run_cmake
make install

cd $AMBERHOME
export CUDA_VISIBLE_DEVICES=0
make test.cuda.serial
(choose the GPU id you wish to test)

2.4 多GPU安装

Building pmemd.cuda.MPI

cd build
# edit the run_cmake script to have -DMPI=TRUE -DCUDA=TRUE
./run_cmake
make install

cd $AMBERHOME
export DO_PARALLEL="mpirun -np 28" # for bash/sh
make test.cuda.parallel

四、讨论

精度的选择：

single precision arithmetic 更快？ double precision arithmetic （CPU用的这个，但是GPU效果不好）

termed hybrid precision (SPDP)：单精度和双精度的混合

SPFP (Default) Uses a combination of single precision for calculation and fixed (integer) precision for accumulation

DPFP - Uses double precision (and double precision equivalent fixed precision) for the entire calculation

参考资料

药企，独角兽，苏州。团队长期招人，感兴趣的都可以发邮件聊聊：tiehan@sina.cn

个人公众号，比较懒，很少更新，可以在上面提问题，如果回复不及时，可发邮件给我： tiehan@sina.cn