k-Wave

1. SamPichardo
   

   Member
   Joined: Jan '13
   Posts: 3

   Hi,

   First of all, thanks for the tool, it has been quite useful to teach ultrasound course material.

   Some users have been asking for how to compile in MacOs (there are some old threads). With the advent of support to external GPUs via USB-C 3.1 (including NVIDIA after a minor hack https://egpu.io/forums/mac-setup/wip-nvidia-egpu-support-for-high-sierra/paged/1/), the use of NVIDIA processors is becoming more accessible for Mac users. Below you can see an ensemble of diff outputs of all the changes I had to do. All of them very minor, just involving some macro definitions and add some extra libraries. I can compile and run this under MacOS Sierra (10.13.3), using latest stable XCode (9.2) and CUDA 9.1 (it also works with CUDA 9.0, but you'd need to use an older version of XCode). For zlib, szip and HDF5 libraries, I used brew. Changes in Makefile are just for "laziness" :).

   I hope this can help to other people who are interested in using k-wave GPU binaries in Mac. It should be also easy to include the changes in the distribution code.

   Cheers

   Sam

   *************************************
   

   diff -ENwbur orig-k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Hdf5/Hdf5File.h k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Hdf5/Hdf5File.h
   --- orig-k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Hdf5/Hdf5File.h	2017-09-04 17:12:12.000000000 -0600
   +++ k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Hdf5/Hdf5File.h	2018-02-09 16:34:52.000000000 -0700
   @@ -459,6 +459,11 @@
    #ifndef HDF5_FILE_H
    #define HDF5_FILE_H
   
   +#ifdef __APPLE__
   +#include <string>
   +#include <sstream>
   +#include <iostream>
   +#endif
   
    #include <hdf5.h>
    #include <hdf5_hl.h>
   diff -ENwbur orig-k-wave-toolbox-version-1.2-cpp-gpu-linux-source/KSpaceSolver/KSpaceFirstOrder3DSolver.cpp k-wave-toolbox-version-1.2-cpp-gpu-linux-source/KSpaceSolver/KSpaceFirstOrder3DSolver.cpp
   --- orig-k-wave-toolbox-version-1.2-cpp-gpu-linux-source/KSpaceSolver/KSpaceFirstOrder3DSolver.cpp	2017-09-04 17:12:12.000000000 -0600
   +++ k-wave-toolbox-version-1.2-cpp-gpu-linux-source/KSpaceSolver/KSpaceFirstOrder3DSolver.cpp	2018-02-09 16:32:48.000000000 -0700
   @@ -31,8 +31,8 @@
     * If not, see [http://www.gnu.org/licenses/](http://www.gnu.org/licenses/).
     */
   
   -// Linux build
   -#ifdef __linux__
   +// Linux and MacOS build
   +#if defined(__linux__) || defined (__APPLE__)
      #include <sys/resource.h>
      #include <cmath>
    #endif
   diff -ENwbur orig-k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Logger/ErrorMessages.h k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Logger/ErrorMessages.h
   --- orig-k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Logger/ErrorMessages.h	2017-09-04 17:12:12.000000000 -0600
   +++ k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Logger/ErrorMessages.h	2018-02-09 16:32:48.000000000 -0700
   @@ -34,8 +34,7 @@
   
    #ifndef ERROR_MESSAGES_H
    #define ERROR_MESSAGES_H
   -
   -#ifdef __linux__
   +#if defined(__linux__) || defined (__APPLE__)
      #include <Logger/ErrorMessagesLinux.h>
    #endif
   
   diff -ENwbur orig-k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Logger/OutputMessages.h k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Logger/OutputMessages.h
   --- orig-k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Logger/OutputMessages.h	2017-09-04 17:12:12.000000000 -0600
   +++ k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Logger/OutputMessages.h	2018-02-09 16:32:48.000000000 -0700
   @@ -32,7 +32,7 @@
    #ifndef OUTPUT_MESSAGES_H
    #define OUTPUT_MESSAGES_H
   
   -#ifdef __linux__
   +#if defined(__linux__) || defined (__APPLE__)
      #include <Logger/OutputMessagesLinux.h>
    #endif
   
   @@ -42,4 +42,3 @@
    #endif
   
    #endif /* OUTPUT_MESSAGES_H */
   -
   diff -ENwbur orig-k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Makefile k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Makefile
   --- orig-k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Makefile	2017-09-04 22:31:34.000000000 -0600
   +++ k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Makefile	2018-02-09 16:41:52.000000000 -0700
   @@ -79,10 +79,10 @@
    LINKING = SEMI
   
    #Set up paths to necessary libraries:
   -HDF5_DIR=${EBROOTHDF5}
   -CUDA_DIR=${CUDA_HOME}
   -ZLIB_DIR=${EBROOTZLIB}
   -SZIP_DIR=${EBROOTSZIP}
   +HDF5_DIR=/usr/local
   +CUDA_DIR=/Developer/NVIDIA/CUDA-9.1
   +ZLIB_DIR=/usr/local/Cellar/zlib/1.2.11
   +SZIP_DIR=/usr/local
   
    #Get GIT hash (only if you build form Gitlab repository)
    #KWAVE_GIT_HASH=$(shell git rev-parse HEAD)
   @@ -96,14 +96,12 @@
   
    # Set compiler flags and header files directories
   -CXXFLAGS = -Xcompiler="-Wall -O3 -fopenmp -ffast-math -fassociative-math" \
   +CXXFLAGS = -Xcompiler="-Wall -O3  -ffast-math -fassociative-math" \
               -O3 -std=c++11  -I$(HDF5_DIR)/include  -I.  --restrict \
    	   -D__KWAVE_GIT_HASH__=\"$(KWAVE_GIT_HASH)\"   \
       	   --device-c
   
   -CUDA_ARCH = --generate-code arch=compute_20,code=sm_20	\
   -	    --generate-code arch=compute_20,code=sm_21	\
   -	    --generate-code arch=compute_30,code=sm_30	\
   +CUDA_ARCH =  --generate-code arch=compute_30,code=sm_30	\
    	    --generate-code arch=compute_32,code=sm_32	\
    	    --generate-code arch=compute_35,code=sm_35	\
    	    --generate-code arch=compute_37,code=sm_37	\
   @@ -115,16 +113,15 @@
   
   -LDFLAGS  =  -Xcompiler="-fopenmp" \
   -	    -Xlinker="-rpath,$(HDF5_DIR)/lib:$(CUDA_DIR)/lib64:." -std=c++11  \
   -	    -L$(HDF5_DIR)/lib  -L$(CUDA_DIR)/lib64
   +LDFLAGS  =  -Xlinker="-rpath,$(HDF5_DIR)/lib:$(CUDA_DIR)/lib:." -std=c++11  \
   +	    -L$(HDF5_DIR)/lib  -L$(CUDA_DIR)/lib
   
    ifeq ($(LINKING),STATIC)
    	LIBS  = $(HDF5_DIR)/lib/libhdf5_hl.a 	     \
                	$(HDF5_DIR)/lib/libhdf5.a	     \
   -	        $(CUDA_DIR)/lib64/libcufft_static.a  \
   -	        $(CUDA_DIR)/lib64/libculibos.a       \
   -        	$(CUDA_DIR)/lib64/libcudart_static.a \
   +	        $(CUDA_DIR)/lib/libcufft_static.a  \
   +	        $(CUDA_DIR)/lib/libculibos.a       \
   +        	$(CUDA_DIR)/lib/libcudart_static.a \
    	        $(ZLIB_DIR)/lib/libz.a		     \
    	        $(SZIP_DIR)/lib/libsz.a 	     \
    		-ldl
   diff -ENwbur orig-k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Parameters/CommandLineParameters.cpp k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Parameters/CommandLineParameters.cpp
   --- orig-k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Parameters/CommandLineParameters.cpp	2017-09-04 17:12:12.000000000 -0600
   +++ k-wave-toolbox-version-1.2-cpp-gpu-linux-source/Parameters/CommandLineParameters.cpp	2018-02-09 16:32:48.000000000 -0700
   @@ -30,7 +30,7 @@
     */
   
    //Linux build
   -#ifdef __linux__
   +#if defined(__linux__) || defined (__APPLE__)
      #include <getopt.h>
    #endif

   Posted 6 years ago #

2. Bradley Treeby
   

   Developer
   Joined: Mar '10
   Posts: 1,643

   Great, thanks Sam, that's very helpful!

   Brad.

   Posted 6 years ago #

3. lennartverhagen
   

   Member
   Joined: May '20
   Posts: 4

   Hi Sam, thanks for sharing this here! Your diff report above has helped me to successfully compile the CPU OPM on macOS.

   Brad, Jiri, is there a way I can contribute my instructions to the community? Do you prefer a post here? Or should I post my adapted source code and readme somewhere?

   Posted 4 years ago #

4. lennartverhagen
   

   Member
   Joined: May '20
   Posts: 4

   Quick update: I've put my changes on GitHub: https://github.com/lennartverhagen/k-Wave-CPUGPU-macOS. The changes only hold for kspaceFirstOrder-OMP. I didn't get acousticFieldPropagator-OMP to compile and seeing how I'm not currently using this, I'll leave that untouched for now.

   Posted 4 years ago #

5. Bradley Treeby
   

   Developer
   Joined: Mar '10
   Posts: 1,643

   Thanks Lennart. Providing an official macOS release has been on our list for a while, it's just never quite made it to the top!

   Posted 4 years ago #

6. Jiri Jaros
   

   Developer
   Joined: Feb '12
   Posts: 115

   Hi Lennart,
   Actually, the main reason is me not having access to any Mac, so I can't test it properly.
   I'll take a look at the repository and try to incorporate necessary changes.
   Jiri

   Posted 4 years ago #

7. lennartverhagen
   

   Member
   Joined: May '20
   Posts: 4

   Hi both, I have to be honest with you, I can see why it's not on top of your list :) As you can see from the top post, you need to be quite dedicated with an external Nvidia GPU and some hacks to benefit from the CUDA acceleration. I'm using a macOS for dev work and teaching. We're running the sims on our linux HPC cluster. I don't have an eGPU for my Mac, so I also wouldn't be able to test the CUDA binaries...

   Posted 4 years ago #

8. nellens1
   

   Member
   Joined: Apr '16
   Posts: 8

   Has anyone managed to get the binaries working with Apple's new M2 processor architecture? I've downloaded lennartverhagen's kindly provided github repo and followed the instructions, but I've crashed into some issues with Apple's clang compiler and I haven't found a fix elsewhere.

   clang: error: the clang compiler does not support '-march=native'

   I've tried various alternatives to march=native and followed this thread: https://discourse.llvm.org/t/why-does-march-native-not-work-on-apple-m1/2733/7 but I think I'm exceeding my C skills. Has anyone else mastered this?

   Posted 1 year ago #

9. Jiri Jaros
   

   Developer
   Joined: Feb '12
   Posts: 115

   The CUDA code will never work on M2 processors, it is not CUDA compatible.

   The OpenMP code will work after changing memory allocation from __mm_malloc, which is an x86 instruction, to aligned_alloc. The -march parameter is only optional and can be removed. However, the code was only tested with Intel and GCC compilers.

   Posted 1 year ago #

RSS feed for this topic

Reply

You must log in to post.