【2.0】amber的安装

(centos7)

一、安装

1.1 下载amber

http://ambermd.org 上下载

  • Amber18.tar.bz2
  • AmberTools18.tar.bz2

终端输入:

cd /data/software/amber

tar -xjf Amber18.tar.bz2
tar -xjf AmberTools18.tar.bz2

两个压缩包会解压到一个名为amber18的文件夹里

1.2 设置AMBERHOME主目录环境变量: 

vim ~/.bashrc

export AMBERHOME=/mnt/nfs/data/software/amber/amber18   

然后使其生效

source ~/.bashrc

1.3 安装依赖的软件及软件库:

sudo yum install gcc gcc-gfortran gcc-c++ flex tcsh zlib-devel bzip2-devel libXt-devel libXext-devel libXdmcp-devel tkinter openmpi openmpi-devel perl perl-ExtUtils-MakeMaker

yum -y install tcsh make  gcc gcc-gfortran gcc-c++ which flex bison patch bc libXt-devel libXext-devel perl util-linux wget bzip2 bzip2-devel zlib-devel

yum -y install openmpi openmpi-devel environment-module

1.4 切换到amber目录,执行编译脚本:

cd $AMBERHOME
$ ./configure --help                

#执行此命令可以看到不同的编译模式,不同的参数设置,我以前也什么都不懂,看完后理解就好很多

$ ./configure gnu                  # 使用gcc编译器编译(gnu包含了gcc套件)
此命令会产生出amber.sh和amber.csh文件,这两个文件可以用于添加amber所需的环境变量。

[显卡驱动及cuda9.0安装:设置环境变量,尤其是要设置CUDA_HOME(先要安装cuda):

#CUDA (amber18不能支持cuda10)

export CUDA_HOME=/usr/local/cuda-9.0;
export PATH=$CUDA_HOME/bin:$PATH;
export LD_LIBRARY_PATH=$CUDA_HOME/lib:$LD_LIBRARY_PATH;

报错:

Testing the g++ compiler with boost:
     g++ -fPIC  testp.cpp -lboost_thread -lboost_system -o testp

Warning: Could not link to boost. Ensure boost-thread, boost-system and boost headers are installed.
Warning: Failed command:
     g++ -fPIC  -lboost_thread -lboost_system -o testp testp.cpp
Warning: Check boost.compile.err for error details.
Warning: Memembed functionality in packmol_memgen not available.

暂时不管上面的报错。也没查出问题

1.5 环境变量的修改

configure命令会产生出amber.sh和amber.csh文件,这两个文件可以用于添加amber所需的环境变量。

If you use a Bourne shell (e.g., bash, sh, zsh, etc.), source the
/mnt/nfs/data/software/amber/amber18/amber.sh file in your shell. Consider adding the line
  test -f /mnt/nfs/data/software/amber/amber18/amber.sh && source /mnt/nfs/data/software/amber/amber18/amber.sh
to your startup file (e.g., ~/.bashrc)

If you use a C shell (e.g., csh, tcsh), source the
/mnt/nfs/data/software/amber/amber18/amber.csh file in your shell. Consider adding the line
  test -f /mnt/nfs/data/software/amber/amber18/amber.csh && source /mnt/nfs/data/software/amber/amber18/amber.csh
to your startup file (e.g., ~/.cshrc)

或者自己手动添加到:

vim ~/.bashrc 中:

export AMBER_PREFIX="/mnt/nfs/data/software/amber/amber18"
export AMBERHOME=/mnt/nfs/data/software/amber/amber18
export PATH="${AMBER_PREFIX}/bin:${PATH}"
# Add location of Amber Python modules to default Python search path
if [ -z "$PYTHONPATH" ]; then
    export PYTHONPATH="${AMBER_PREFIX}/lib/python2.7/site-packages"
else
    export PYTHONPATH="${AMBER_PREFIX}/lib/python2.7/site-packages:${PYTHONPATH}"
fi
if [ -z "${LD_LIBRARY_PATH}" ]; then
   export LD_LIBRARY_PATH="${AMBER_PREFIX}/lib"
else
   export LD_LIBRARY_PATH="${AMBER_PREFIX}/lib:${LD_LIBRARY_PATH}"
fi

顺便把cuda也放进去:

export CUDA_HOME=/usr/local/cuda-9.0
export PATH=$CUDA_HOME/bin:$PATH
export LD_LIBRARY_PATH=$CUDA_HOME/lib:$LD_LIBRARY_PATH

source ~/.bashrc 使环境变量生效。

1.6 编译安装与测试:

make install  # 编译安装
make test  # 编译测试

测试,也有部分报错:

make[2]: Leaving directory `/mnt/nfs/data/software/amber/amber18/test'
176 file comparisons passed
2 file comparisons failed
0 tests experienced errors
Test log file saved as /mnt/nfs/data/software/amber/amber18/logs/test_amber_serial/2019-10-30_19-08-33.log
Test diffs file saved as /mnt/nfs/data/software/amber/amber18/logs/test_amber_serial/2019-10-30_19-08-33.diff
make[1]: Leaving directory `/mnt/nfs/data/software/amber/amber18/test'

Summary of AmberTools serial tests:

2245 file comparisons passed
15 file comparisons failed
0 tests experienced errors
Test log file saved as /mnt/nfs/data/software/amber/amber18/logs/test_at_serial/2019-10-30_17-58-51.log
Test diffs file saved as /mnt/nfs/data/software/amber/amber18/logs/test_at_serial/2019-10-30_17-58-51.diff

1.7 多线程

1.7.1 安装openmpi

如果本身系统已经装了openmpi合适版本,which mpirun命令返回openmpi的目录即可,否则:

下载openmpi安装包:

网址:http://www.open-mpi.org/software/ompi/v1.8/

版本要求在 $AMBERHOME/AmberTools/src/configure_openmpi 中有说明, mpidirs=/bin/ls -d openmpi-1.[5-9]*/ 2>/dev/null | tail -n 1

解压openmpi解压到 $AMBERHOME/AmberTools/src: 

wget -c https://download.open-mpi.org/release/open-mpi/v1.8/openmpi-1.8.8.tar.gz
tar zxvf openmpi-1.8.8.tar.gz

1.7.2 在src目录编译

cd $AMBERHOME/AmberTools/src
./configure_openmpi gnu    #编译,此命令会产生出amber.sh和amber.csh文件,这两个文件可以用于添加amber所需的环境变量。

1.7.3 环境变量的修改

将$AMBERHOME/AmberTools/src/amber.sh 环境变量添加到~/.bashrc中

vim ~/.bashrc :

export MPI_HOME=$AMBERHOME/AmberTools
export PATH=$PATH:$MPI_HOME/bin
export LD_LIBRARY_PATH=$MPI_HOME/lib:$LD_LIBRARY_PATH
export DO_PARALLEL="mpirun -np 4"    

# Note the value below may depend on your MPI implementation,因为某些测试中需要用到的线程较多,所以最好设置4个或8个

1.7.7.4 配置openmpi后,回到$AMBERHOME编译amber:

cd $AMBERHOME
./configure -mpi gnu

1.7.7.5 Amber安装测试:

make install
make test

测试结果在:

176 file comparisons passed
6 file comparisons failed
15 tests experienced an error
Test log file saved as /mnt/nfs/data/software/amber/amber18/logs/test_amber_parallel/2019-10-22_13-44-58.log
Test diffs file saved as /mnt/nfs/data/software/amber/amber18/logs/test_amber_parallel/2019-10-22_13-44-58.diff
make[1]: Leaving directory `/mnt/nfs/data/software/amber/amber18/test'

Summary of AmberTools parallel tests:

1002 file comparisons passed
162 file comparisons failed
81 tests experienced errors
Test log file saved as /mnt/nfs/data/software/amber/amber18/logs/test_at_parallel/2019-10-22_13-21-14.log
Test diffs file saved as /mnt/nfs/data/software/amber/amber18/logs/test_at_parallel/2019-10-22_13-21-14.diff

报错的内容:

CPPTRAJ: CRD Replica Trajectory Run with remdout
[g02:24670] 3 more processes have sent help message help-mpi-btl-base.txt / btl:no-nics
[g02:24670] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
diffing all.dat.save with all.dat

暂时先不管报错

1.9 支持GPU

1.9.1 显卡驱动及cuda9.0安装:

见我cuda安装的博客

1.9.2 切换到amber目录,编译:

cd $AMBERHOME
make clean
./configure -cuda gnu     #不可省略
./configure -cuda -mpi gnu  #紧接着上一步

make install

如果只有./configure -cuda -mpi gnu;make install,看到的是cuda是

[sam@g02 bin]$ ll |grep cuda
-rwxr-xr-x 1 sam molinfo 5.6M Oct 31 10:03 cpptraj.MPI.cuda
-rwxr-xr-x 1 sam molinfo  91M Oct 31 10:22 pmemd.cuda_DPFP.MPI
lrwxrwxrwx 1 sam molinfo   19 Oct 31 10:12 pmemd.cuda.MPI -> pmemd.cuda_SPFP.MPI
-rwxr-xr-x 1 sam molinfo  73M Oct 31 10:12 pmemd.cuda_SPFP.MPI

接着./configure -cuda gnu;make install,就能看到了

[sam@g02 bin]$ ll |grep cuda
-rwxr-xr-x 1 sam molinfo 5.5M Oct 31 13:43 cpptraj.cuda
-rwxr-xr-x 1 sam molinfo 5.6M Oct 31 10:03 cpptraj.MPI.cuda
-rwxr-xr-x 1 sam molinfo 8.6M Oct 31 13:37 pbsa.cuda
lrwxrwxrwx 1 sam molinfo   15 Oct 31 13:53 pmemd.cuda -> pmemd.cuda_SPFP
-rwxr-xr-x 1 sam molinfo  91M Oct 31 14:04 pmemd.cuda_DPFP
-rwxr-xr-x 1 sam molinfo  91M Oct 31 10:22 pmemd.cuda_DPFP.MPI
lrwxrwxrwx 1 sam molinfo   19 Oct 31 10:12 pmemd.cuda.MPI -> pmemd.cuda_SPFP.MPI
-rwxr-xr-x 1 sam molinfo  73M Oct 31 13:53 pmemd.cuda_SPFP
-rwxr-xr-x 1 sam molinfo  73M Oct 31 10:12 pmemd.cuda_SPFP.MPI
[sam@g02 bin]$ pwd
/mnt/nfs/data/software/amber/amber18/bin

Warning: No parallel NetCDF specified (--with-pnetcdf <dir>).
       Parallel NetCDF trajectory output will not be available
       in cpptraj.MPI.

1.9.3 测试

make test.cuda_serial

测试结果:

232 file comparisons passed
       1 file comparison failed
       1 test experienced an error
Test log file saved as /mnt/nfs/data/software/amber/amber18/logs/test_amber_cuda/2019-10-31_14-16-16.log
Test diffs file saved as /mnt/nfs/data/software/amber/amber18/logs/test_amber_cuda/2019-10-31_14-16-16.diff
make[1]: Leaving directory `/mnt/nfs/data/software/amber/amber18/test'

测试:

export DO_PARALLEL='mpirun -np 2
make test.cuda_parallel

测试结果:

make[3]: [test.pmemd.cuda.gb] Error 1 (ignored)
cd gamd/rna_gb && ./Run.gb.gamd2  DPFP /mnt/nfs/data/software/amber/amber18/include/netcdf.mod
cudaDeviceEnablePeerAccess all CUDA-capable devices are busy or unavailable
cudaDeviceEnablePeerAccess all CUDA-capable devices are busy or unavailable

debug的问题

http://cseweb.ucsd.edu/classes/wi15/cse262-a/static/cuda-5.5-doc/html/cuda-runtime-api/group__CUDART__TYPES.html

原因:

This indicates that all CUDA devices are busy or unavailable at the current time. Devices are often busy/unavailable due to use of cudaComputeModeExclusive, cudaComputeModeProhibited or when long running CUDA kernels have filled up the GPU and are blocking new work from starting. They can also be unavailable due to memory constraints on a device that already has active CUDA work being performed.


[sam@g02 amber18]$ nvidia-smi -a |grep Compute
    Compute Mode                    : Exclusive_Process
    Compute Mode                    : Exclusive_Process
    Compute Mode                    : Exclusive_Process
    Compute Mode                    : Exclusive_Process
    Compute Mode                    : Exclusive_Process
    Compute Mode                    : Exclusive_Process
    Compute Mode                    : Exclusive_Process
    Compute Mode                    : Exclusive_Process


cudaComputeModeDefault = 0
Default compute mode (Multiple threads can use cudaSetDevice() with this device)
cudaComputeModeExclusive = 1
Compute-exclusive-thread mode (Only one thread in one process will be able to use cudaSetDevice() with this device)
cudaComputeModeProhibited = 2
Compute-prohibited mode (No threads can use cudaSetDevice() with this device)
cudaComputeModeExclusiveProcess = 3
Compute-exclusive-process mode (Many threads in one process will be able to use cudaSetDevice() with this device)

解决办法:

nvidia-smi -c 0  # 测试的时候改成这个

nvidia-smi -c 3  # 测试结束以后,改成这个


make test.cuda_parallel

128 file comparisons passed
0 file comparisons failed
3 tests experienced errors
Test log file saved as /mnt/nfs/data/software/amber/amber18/logs/test_amber_cuda_parallel/2019-10-31_16-30-43.log
No test diffs to save!
make[1]: Leaving directory `/mnt/nfs/data/software/amber/amber18/test'

精度的选择:

single precision arithmetic 更快? double precision arithmetic (CPU用的这个,但是GPU效果不好)

termed hybrid precision (SPDP): 单精度和双精度的混合

SPFP (Default) Uses a combination of single precision for calculation and fixed (integer) precision for accumulation

DPFP - Uses double precision (and double precision equivalent fixed precision) for the entire calculation

二、讨论

2.1

K80上可以使用 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5843186/

2.2 参考集群命令:

export AMBERHOME=/data/software/amber18/ export MOE=/data/software/moe/moe2019/ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/data/software/g03-ti-lib

2.3 升级的话,可以这样来:

./update_amber --upgrade 

Upgrade of AmberTools18 to AmberTools19 is complete. Please re-run configure, applying any available updates to Amber18 and/or AmberTools19, and then recompile everything.

cd $AMBERHOME
make clean
./configure -cuda gnu
./configure -cuda -mpi gnu


Checking NetCDF...
        Using bundled NetCDF library.
        Using existing NetCDF in '/mnt/nfs/data/software/amber/amber18'


Warning: No parallel NetCDF specified (--with-pnetcdf <dir>).
         Parallel NetCDF trajectory output will not be available
         in cpptraj.MPI.

Fatal Error: File 'netcdf.mod' opened at (1) is not a GFORTRAN module file

这个错误需要回到$AMBERHOME/include文件夹内,把所有的netcdf.*文件全部删除,然后再重复make clean;configure等

make install
make test

参考资料

个人公众号,比较懒,很少更新,可以在上面提问题,如果回复不及时,可发邮件给我: tiehan@sina.cn

Sam avatar
About Sam
专注生物信息 专注转化医学