Generic Page

Bash Command Line and Scripting Tutorial

1. Download Dataset

This tutorial will help you learn how to use Bash command line and scripting to work with the ds000157 dataset available at OpenNeuro. The dataset is in BIDS (Brain Imaging Data Structure) format, used for neuroimaging data. For more information on Open Science and Data Management, visit our OpenNeuro and BIDS introduction pages.

Find the "ds000157" dataset on OpenNeuro. This dataset includes NIfTI files for each subject’s functional and anatomical MRI scans, organized in BIDS format.

image

You need to download the entire OpenNeuro dataset, either directly with your browser or using DataLad. Note: this may take a few minutes.

Option 1: Browser

image

If you download the data directly with your browser, ensure that the files are unzipped correctly in a folder named ds000157.

Option 2: DataLad

image

If you do not already have DataLad installed, you will need to install it using your terminal. Then, download the dataset using DataLad and navigate to the dataset directory. You will then download the dataset contents recursively.

# Install datalad
$ pip install datalad

# Download the dataset
$ datalad install https://github.com/OpenNeuroDatasets/ds000157.git
$ cd ds000157
datalad get -r .

For more detailed instructions on using DataLad to download OpenNeuro datasets, refer to the DataLad Handbook.

Once the dataset is downloaded, ensure all the files are in a folder named ds000157. For this tutorial, we'll assume you have extracted it into ~/Downloads/ds000157.

This OpenNeuro dataset was collected by the OpenfMRI project. For the study, 30 female subjects performed a passive viewing task with blcoks of food andnonfood images. Cite: Smeets PA, Kroese FM, Evers C, de Ridder DT. Behav Brain Res. 2013 Jul 1;248:41-5. doi: 10.1016/j.bbr.2013.03.041. Epub 2013 Apr 8. http://www.ncbi.nlm.nih.gov/pubmed/23578759

2. Navigating Directories

To start, open terminal and navigate to the Downloads directory

$ cd ~/Downloads

Let's take a look inside:

$ ls -tr

In this code, ls lists the files and directories within Downloads, while -t sorts by modification time, newest first and -r OR --reverse prints in reverse order while sorting. Therefore, the output of ls -ltr is a list with the most recently modified files and directories at the bottom!

Now let's see whats inside the 

# Navigate to the dataset directory
cd ~/datasets/ds000157

# List files and directories
ls

# If you need more details
ls -lh

# To navigate to a specific directory
cd sub-01/func

3. Viewing Files and Directories

You might need to view the contents of files or check the structure.

# Display the first few lines of a file
head sub-01_task-rest_bold.nii.gz

# Display the last few lines of a file
tail sub-01_task-rest_bold.nii.gz

# Check the structure of a directory
tree

4. File Management

Copy, move, or remove files as needed.

# Copy a file
cp sub-01_task-rest_bold.nii.gz ~/my_data_backup/

# Move a file
mv sub-01_task-rest_bold.nii.gz ~/my_data_backup/

# Remove a file
rm ~/my_data_backup/sub-01_task-rest_bold.nii.gz

# Remove a directory and its contents
rm -r ~/my_data_backup/

5. Searching for Files

Find files based on their names or patterns.

# Find all .nii.gz files in the directory
find . -name "*.nii.gz"

# Search for a specific file
find . -name "sub-01_task-rest_bold.nii.gz"

6. Viewing and Editing Files

For BIDS datasets, you might need to inspect metadata or configuration files.

# View the content of a text file
cat dataset_description.json

# Use a pager to view large files
less dataset_description.json

# Edit a file using nano text editor
nano dataset_description.json

7. Scripting with Bash

Automate tasks using Bash scripts. Create a file named process_data.sh and make it executable.

# Create a new Bash script file
nano process_data.sh

# Add the following script to the file
#!/bin/bash

# Navigate to the dataset directory
cd ~/datasets/ds000157

# Find and list all .nii.gz files
echo "Listing all .nii.gz files:"
find . -name "*.nii.gz"

# Backup all .nii.gz files to a backup directory
mkdir -p ~/my_data_backup
find . -name "*.nii.gz" -exec cp {} ~/my_data_backup/ \;

# Print a completion message
echo "Backup completed."

# Save and exit nano (Ctrl+X, then Y, then Enter)

# Make the script executable
chmod +x process_data.sh

# Run the script
./process_data.sh

8. Using Commands in Scripts

Sometimes you need to include commands in a script for more complex operations.

# Example of a more complex script
#!/bin/bash

# Example of a more complex script
echo "Starting data processing..."

# Check disk space
df -h

# Count files in a directory
file_count=$(ls | wc -l)
echo "Number of files in the directory: $file_count"

# Compress all .nii.gz files into a tarball
tar -cvzf fMRI_data_backup.tar.gz *.nii.gz

echo "Data processing completed."

9. Advanced Commands

You might need more advanced processing depending on your needs.

# Find files modified in the last 7 days
find . -type f -mtime -7

# Execute a command on each file found
find . -name "*.nii.gz" -exec gzip -d {} \;

Conclusion

This tutorial covers basic Bash commands and scripting for managing and processing the BIDS-format fMRI dataset. As you become more familiar with Bash, you can create more complex scripts to automate and analyze your data.

If you need more specific examples or have questions about particular tasks, feel free to ask!