k-Wave

1. kwoolfe2
   

   Member
   Joined: Nov '15
   Posts: 1

   I am trying to run the 3-D C++ k-wave code on a HPC. The HPC has hundreds of nodes, and each node only has 32 processors. I have found that OpenMP can't parallelize the code over nodes, but it can parallelize over the 32 processors in a single node. Is there any way to parallelize the code over multiple nodes? Is it even worth it to try, or would the gains made by parallelizing over, say, 100 processors be negated by increased overhead time?

   Posted 8 years ago #

2. Jiri Jaros
   

   Developer
   Joined: Feb '12
   Posts: 115

   Hi Kwoolfe2,
   We do have such a code but it is highly experimental and before being released some additional work has to be done. The functionality is here, however, the deployment is not trivial. Typically, I'd need access to your cluster to compile the software stack, test all dependencies, find best configurations, prepare job scripts and the give the bunde to you.

   Considering you have a node with 32 cores, you'd need at least 4 nodes (128 cores) to get any resonable speed-up. On the other hand, the code is able to scale up to thousands of cores.

   Please send an email to jarosjir@fit.vutbr.cz and we can discuss details.

   Jiri

   Posted 8 years ago #

3. samantp
   

   Member
   Joined: Jun '16
   Posts: 17

   Hi Jiri,
   Do you have any packages that would allow me to compile and install the code onto a Linux based cluster? I'd need to be able to deploy it myself due to access restrictions on my university's HPC.

   Thank you

   Posted 7 years ago #

4. Bradley Treeby
   

   Developer
   Joined: Mar '10
   Posts: 1,643

   Hi samantp,

   We don't have a problem with other people using the MPI code, however, the reason we haven't released it widely is that it is difficult to support. It's sometimes non-trivial to compile depending on the machine, available modules, compilers, etc. There are also various compatibility problems and bugs with different combinations of libraries that k-Wave relies on.

   Do you have experience in compiling libraries and codes from source on HPC clusters?

   Brad.

   Posted 7 years ago #

5. ttiennot
   

   Member
   Joined: Jul '17
   Posts: 5

   Hi,

   I used to work with the GPU code but due to memory issues on large simulations I have to use the CPU code now.

   I am working on a cluster with more than 2000 cores but I need to use openMPI to work on several nodes. Would you agree to send me the code using Open MPI ?

   Best regards,

   Thomas

   Posted 7 years ago #

6. Bradley Treeby
   

   Developer
   Joined: Mar '10
   Posts: 1,643

   Hi Thomas,

   Are you experienced with compiling libraries and source codes on linux-based HPC clusters, or do you have local IT support that can do this for you?

   Brad.

   Posted 7 years ago #

7. ttiennot
   

   Member
   Joined: Jul '17
   Posts: 5

   Hi Brad,

   I have some experience and I also have a local IT support that can do this for me.

   Thomas

   Posted 7 years ago #

8. Jiri Jaros
   

   Developer
   Joined: Feb '12
   Posts: 115

   Hi Thomas,
   please send me an email address to jarosjir@fit.vutbr.cz and I'll prepare a code snapshot for you.

   Cheers,
   Jiri

   Posted 7 years ago #

RSS feed for this topic

Reply

You must log in to post.