This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Using ARMPL for my math libraries in CP2K using a Mac machine

Hi I'm attempting to use ARMPL as the FFTW library and the BLAS library in CP2K for my Mac M1 Max Macbook Pro, but I'm having issues during the build. I keep getting the following error:

 ...
  "___kmpc_reduce_nowait", referenced from:
      __ZN5armplL13asum_parallelIfEENS_14remove_complexIT_E4typeExPKS2_xPFS4_xS6_xEi.omp_outlined in libarmpl_lp64_mp.a[204](41979031_asum_apple_m1_flang-new_mp.o)
      __ZN5armplL13asum_parallelIfLb0EEENS_14remove_complexIT_E4typeEiPS2_ii.omp_outlined in libarmpl_lp64_mp.a[204](41979031_asum_apple_m1_flang-new_mp.o)
      __ZN5armplL13asum_parallelIfLb0EEENS_14remove_complexIT_E4typeEiPS2_ii.omp_outlined.1 in libarmpl_lp64_mp.a[204](41979031_asum_apple_m1_flang-new_mp.o)
      __ZN5armplL13asum_parallelIdEENS_14remove_complexIT_E4typeExPKS2_xPFS4_xS6_xEi.omp_outlined in libarmpl_lp64_mp.a[204](41979031_asum_apple_m1_flang-new_mp.o)
      __ZN5armplL13asum_parallelIdLb0EEENS_14remove_complexIT_E4typeEiPS2_ii.omp_outlined in libarmpl_lp64_mp.a[204](41979031_asum_apple_m1_flang-new_mp.o)
      __ZN5armplL13asum_parallelIdLb0EEENS_14remove_complexIT_E4typeEiPS2_ii.omp_outlined.2 in libarmpl_lp64_mp.a[204](41979031_asum_apple_m1_flang-new_mp.o)
      __ZN5armplL13asum_parallelINSt3__17complexIfEEEENS_14remove_complexIT_E4typeExPKS5_xPFS7_xS9_xEi.omp_outlined in libarmpl_lp64_mp.a[204](41979031_asum_apple_m1_flang-new_mp.o)
      ...
ld: symbol(s) not found for architecture arm64
collect2: error: ld returned 1 exit status

I do not know what I am doing wrong. I have copied the ssmp file I am using if anyone has any insights into what errors I am making. Any help would be greatly appreciated.  Thanks.

#!/bin/bash
#
# CP2K Darwin arch file for a serial arm64 binary
# (https://www.cp2k.org/howto:compile_on_macos)
#
# Tested with: GNU 13.2.0, FFTW 3.3.10, LIBINT 2.6.0, LIBVORI 220621,
#              LIBXC 6.2.2, OpenBLAS 0.3.26, SPGLIB 2.3.1,
#              LIBGRPP 20231215
# on an Apple M1 (macOS 14.2.1 Sonoma)
#
# Usage: Source this arch file and then run make as instructed.
#        Ensure the links in /usr/local/bin to the lastest gcc version.
#
# Last update: 19.03.2024
#
# \
   if [[ "${0}" == "${BASH_SOURCE}" ]]; then \
      echo "ERROR: Script ${0##*/} must be sourced"; \
      echo "Usage: source ${0##*/}"; \
      exit 1; \
   fi; \
   this_file=${BASH_SOURCE##*/}; \
   cd tools/toolchain; \
   [[ -z "${target_cpu}" ]] && target_cpu="native"; \
   if $(command -v brew >/dev/null 2>&1); then \
      brew install cmake; \
      brew install coreutils; \
      brew install libxc; \
      brew install pkg-config; \
      brew install wget; \
   else \
      echo "ERROR: Homebrew installation not found"; \
      echo '  Run: /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"'; \
      cd ../..; \
      return 1; \
   fi; \
   ./install_cp2k_toolchain.sh -j${maxtasks} --mpi-mode=no --no-arch-files --target-cpu=${target_cpu} \
      --with-cmake=$(brew --prefix cmake) --with-fftw=no --with-gcc=system \
      --with-libxc=$(brew --prefix libxc) --with-libxsmm=no --with-openblas=no; \
   source ./install/setup; \
   cd ../..; \
   echo; \
   echo "Check the output above for error messages and consistency!"; \
   echo; \
   echo "If everything is OK, you can build a CP2K production binary with"; \
   echo "   make -j ARCH=${this_file%.*} VERSION=${this_file##*.}"; \
   echo; \
   echo "Alternatively, you can add further checks, e.g. for regression testing, with"; \
   echo "   make -j ARCH=${this_file%.*} VERSION=${this_file##*.} DO_CHECKS=yes"; \
   echo; \
   echo "Run always the following command before using the CP2K binary"; \
   echo "   source ${PWD}/tools/toolchain/install/setup"; \
   echo; \
   return

# Set options
DO_CHECKS      := no
TARGET_CPU     := native
USE_LIBGRPP    := 20231215
USE_LIBINT     := 2.6.0
USE_LIBVORI    := 220621
USE_LIBXC      := 6.2.2
USE_SPGLIB     := 2.3.1

LMAX           := 5
MAX_CONTR      := 4

CC             := gcc
CXX            := g++
FC             := gfortran
LD             := gfortran
AR             := ar -r -s

CFLAGS         := -O2 -fopenmp -fopenmp-simd -ftree-vectorize -funroll-loops -g -mtune=$(TARGET_CPU)

DFLAGS         += -D__MAX_CONTR=$(strip $(MAX_CONTR))
DFLAGS         += -D__NO_STATM_ACCESS

INSTALL_PATH   := $(PWD)/tools/toolchain/install

# Settings for regression testing
ifeq ($(DO_CHECKS), yes)
   DFLAGS         += -D__CHECK_DIAG
   FCFLAGS_DEBUG  := -fcheck=bounds,do,recursion,pointer
   FCFLAGS_DEBUG  += -fcheck=all,no-array-temps
#  FCFLAGS_DEBUG  += -ffpe-trap=invalid,overflow,zero
   FCFLAGS_DEBUG  += -fimplicit-none
   FCFLAGS_DEBUG  += -finit-derived
   FCFLAGS_DEBUG  += -finit-real=snan
   FCFLAGS_DEBUG  += -finit-integer=-42
   FCFLAGS_DEBUG  += -finline-matmul-limit=0
   WFLAGS         := -Werror=aliasing
   WFLAGS         += -Werror=ampersand
   WFLAGS         += -Werror=c-binding-type
   WFLAGS         += -Werror=conversion
   WFLAGS         += -Werror=intrinsic-shadow
   WFLAGS         += -Werror=intrinsics-std
   WFLAGS         += -Werror=line-truncation
   WFLAGS         += -Wrealloc-lhs
   WFLAGS         += -Werror=tabs
   WFLAGS         += -Werror=target-lifetime
   WFLAGS         += -Werror=underflow
   WFLAGS         += -Werror=unused-but-set-variable
   WFLAGS         += -Werror=unused-dummy-argument
   WFLAGS         += -Werror=unused-variable
endif

ifneq ($(USE_LIBVORI),)
   USE_LIBVORI    := $(strip $(USE_LIBVORI))
   LIBVORI_LIB    := $(INSTALL_PATH)/libvori-$(USE_LIBVORI)/lib
   DFLAGS         += -D__LIBVORI
   LIBS           += $(LIBVORI_LIB)/libvori.a
endif

LIBXC_HOME     := $(shell brew --prefix libxc)
CFLAGS         += -I$(LIBXC_HOME)/include
DFLAGS         += -D__LIBXC
LIBS           += -Wl,-rpath,$(LIBXC_HOME)/lib -L$(LIBXC_HOME)/lib -lxcf03 -lxc

ifneq ($(USE_LIBGRPP),)
   USE_LIBGRPP    := $(strip $(USE_LIBGRPP))
   LIBGRPP_INC    := $(INSTALL_PATH)/libgrpp-main-$(USE_LIBGRPP)/include
   LIBGRPP_LIB    := $(INSTALL_PATH)/libgrpp-main-$(USE_LIBGRPP)/lib
   CFLAGS         += -I$(LIBGRPP_INC)
   DFLAGS         += -D__LIBGRPP
   LIBS           += $(LIBGRPP_LIB)/liblibgrpp.a
endif

ifneq ($(USE_LIBINT),)
   USE_LIBINT     := $(strip $(USE_LIBINT))
   LMAX           := $(strip $(LMAX))
   LIBINT_INC     := $(INSTALL_PATH)/libint-v$(USE_LIBINT)-cp2k-lmax-$(LMAX)/include
   LIBINT_LIB     := $(INSTALL_PATH)/libint-v$(USE_LIBINT)-cp2k-lmax-$(LMAX)/lib
   CFLAGS         += -I$(LIBINT_INC)
   DFLAGS         += -D__LIBINT
   LIBS           += $(LIBINT_LIB)/libint2.a
   LIBS           += $(LIBINT_LIB)/libint2.a
endif

ifneq ($(USE_SPGLIB),)
   USE_SPGLIB     := $(strip $(USE_SPGLIB))
   SPGLIB_INC     := $(INSTALL_PATH)/spglib-$(USE_SPGLIB)/include
   SPGLIB_LIB     := $(INSTALL_PATH)/spglib-$(USE_SPGLIB)/lib
   CFLAGS         += -I$(SPGLIB_INC)
   DFLAGS         += -D__SPGLIB
   LIBS           += $(SPGLIB_LIB)/libsymspg.a
endif

FFTW_HOME      := /opt/arm/armpl_23.10_flang-new_clang_17
CFLAGS         += -I$(FFTW_HOME)/include_lp64_mp
DFLAGS         += -D__FFTW3
LIBS           += $(FFTW_HOME)/lib/libarmpl_lp64_mp.a
LIBS           += $(FFTW_HOME)/lib/libarmpl_lp64.a

OPENBLAS_HOME  := /opt/arm/armpl_23.10_flang-new_clang_17
CFLAGS         += -I$(OPENBLAS_HOME)/include_lp64_mp
LIBS           += $(OPENBLAS_HOME)/lib/libarmpl_lp64.a

CFLAGS         += $(DFLAGS)

FCFLAGS        := $(CFLAGS) $(FCFLAGS_DEBUG) $(WFLAGS)
FCFLAGS        += -fallow-argument-mismatch
FCFLAGS        += -fbacktrace
FCFLAGS        += -ffree-form
FCFLAGS        += -ffree-line-length-none
FCFLAGS        += -fno-omit-frame-pointer
FCFLAGS        += -std=f2008

LDFLAGS        += $(FCFLAGS)

LIBS           += -ldl -lstdc++

# End

Parents
  • Hi.

    Using gcc/gfortran will not be compatible with the current version of ArmPL on macOS.  We have built against the libc and libomp instances that come with clang.  This is why you're getting the missing symbols from both your CP2K and GROMACS builds. As such I recommend you recompile using the LLVM Fortran compiler.  Instruction on how to get this are on our Getting Started with ArmPL - macOS page.

    As far as your build recipe for CP2K goes, the reason that FFTW has both libfftw3.a and libfftw3_omp.a is that they contain a non-overlapping set of functions; for example "fftw_init_threads" only appears in 'libfftw3_omp.a' whereas "fftw_plan_dft" appears only in 'libfftw3.a'.  In Arm Performance Libraries we include all functionality in the same library.  As such we recommend you do not link to two different versions of ArmPL as whether you get threading or not will be determined by the link order.

    Hope this helps.

    Chris

Reply
  • Hi.

    Using gcc/gfortran will not be compatible with the current version of ArmPL on macOS.  We have built against the libc and libomp instances that come with clang.  This is why you're getting the missing symbols from both your CP2K and GROMACS builds. As such I recommend you recompile using the LLVM Fortran compiler.  Instruction on how to get this are on our Getting Started with ArmPL - macOS page.

    As far as your build recipe for CP2K goes, the reason that FFTW has both libfftw3.a and libfftw3_omp.a is that they contain a non-overlapping set of functions; for example "fftw_init_threads" only appears in 'libfftw3_omp.a' whereas "fftw_plan_dft" appears only in 'libfftw3.a'.  In Arm Performance Libraries we include all functionality in the same library.  As such we recommend you do not link to two different versions of ArmPL as whether you get threading or not will be determined by the link order.

    Hope this helps.

    Chris

Children