Barracuda Virtual Reactor is a particle in cell specific commercial CFD produced by CPFD (computation particle fluid dynamics). The currently installed versions of barracuda are:
- 17.2.0
- 17.3.1
Note: This document is written using Barracuda 17. If other versions are used, please replace the “17” with the correct major version number.
Checking license usage
To check the current usage of the licenses, enter “barrstat” in the terminal
> barrstat
This will output the current number of available licenses, and will look something like:
License availability for all products from all ISVs
ISV: cpfd
server host: maint02 (port 27015)
barracuda-15 v2019.12 available: 10
barracuda-15-setup v2019.12 available: 1
barracuda-15-chem v2019.12 available: 10
barracuda-15-setup-chem v2019.12 available: 1
barracuda-gpu v2019.12 available: 10
The important line to look at is:
barracuda-15 v2019.12 available: 10
indicating the current number of available license, in this particular instance, there are 10 licenses available.
Launching the application on a GPU node
To run barracuda on a GPU node with the GUI, first create a job script with:
> nedit run_barracuda
This will open the text editor nedit. Paste the following script into nedit and change the “#” symbols in the export DISPLAY=”login##:#.0 line to match your specific node. The default time limit for a job is 48hrs. If you want to run for a week, add “#SBATCH –qos=long” to the script however you can only have 10 jobs using “long”.
#!/bin/bash -l
##Accounting
#SBATCH --partition=gpu
#SBATCH --qos=normal
##Submission
#SBATCH --nodes=1
#SBATCH --job-name="barracuda"
##Load Modules
module purge
module load vgl barracuda
# export the display to your login node
export DISPLAY="login##:#.0"
##Run the program
vglrun barracuda.17
Save the script and exit nedit. Next, submit the script to the queue with:
> sbatch run_barracuda
Launching the application on the login node
Please do not run simulations on the head node. You need to submit the simulation to the queue.
To run Barracuda, first load the latest module with
> module load barracuda
The application can now be started with
> barracuda.17
Specific versions can be loaded by specifying the version number when loading the module:
> module load barracuda/17.2.0
Submitting batch jobs
Since Barracuda is a serial code (can only use one CPU), the simulations need to be submitted to the shared partition. To do this, first create a job script in the directory of your simulation with:
> nedit run_barracuda
This will open the text editor nedit. Paste the following script into nedit and change the “project_file.prj” to your specific project file name. The default time limit for a job is 48hrs. If you want to run for a week, add “#SBATCH –qos=long” to the script however you can only have 10 jobs using “long”.
#!/bin/bash -l
#SBATCH --job-name="barracuda"
#SBATCH --tasks=1
#SBATCH --partition=shared
#SBATCH --qos=long
module load barracuda
cpfd.x.17 project_file.prj
Save the file and exit nedit. You can now submit the simulation to the queue using “sbatch”:
> sbatch run_barracuda
Restarting batch jobs
Restarting simulations is similar to running a new simulation except we need to create a special project file that points to a restart file. To do this, create the file with nedit:
> nedit restart.prj
On the first line, enter the restart file you would like to start from. Typically this is “IC_” which gets overwritten at a periodic frequency and should be the latest restart file
IC_
Next we need to create another job script:
> nedit restart_barracuda
Which is similar to the previous script with two changes:
- Change the project name to “restart.prj”
- add “-aw” to the run command to append existing transient data files
#!/bin/bash -l
#SBATCH --job-name="barracuda"
#SBATCH --tasks=1
#SBATCH --partition=shared
#SBATCH --qos=long
module load barracuda
cpfd.x.17 -aw restart.prj
Save the file and exit. The job can be submitted with
> sbatch restart_barracuda
We provide an example Barracuda submission script, run_barracuda, in the /nfs/apps/Submission directory. You should not need to edit the submission, by default it will launch on the GPU systems. Barracuda requires a graphical session.
You can use the barrstat command from the login systems to monitor the Barracuda license availability.
You need to export your DISPLAY, where N is your login system, and XX is your display number, then use sbatch to request an allocation for Barracuda.
login:~> export DISPLAY="loginN:XX"
login:~> sbatch run_barracuda