Introduction to Unix

• who: displays a list of the users who are currently using this Unix computer.
• whoami: displays your username (i.e. they person currently logged in).
• date: displays the current date and time.
• cal: displays a calendar on the terminal. It can be configured to display more than just the current month.
• hostname: displays the name of the computer we are logged in to.
• /home/group/common/training/Intro_to_Unix/hi: displays the text "Hello World"

$ ls
exp01  file01  muscle.fq  task01  task02  task03

When running the ls command with no options it will list files in your current working directory. The place where you start when you first login is your HOME directory.

Answer: 6 (exp01, file01, muscle.fq, task01, task02 and task03)

Answer 1: --all (or -a) flag

Now you should see several files in your home directory whose names all begin with a dot. All these files are created automatically for your user account. They are mostly configuration options for various programs including the shell. It is safe to ignore them for the moment.

$ ls --all
.   .bash_history  .bash_profile  exp01   .kshrc    muscle.fq  task01  task03
..  .bash_logout   .bashrc        file01  .lesshst  .ssh       task02  .viminfo

There are two trick files here; namely . and .. which are not real files but instead, shortcuts. . is a shortcut for the current directory and .. a shortcut for the directory above the current one.

Answer 2: 14 files (don't count . and ..)

You can find out the full path name of the current working directory with the pwd command. Your home directory will look something like this:

$ pwd
/home/trainingXX

Answer: /home/trainingXX

where XX is replaced by some 2 digit sequence.

Alternate method: You can also find out the name of your home directory by printing the value of the $HOME shell variable:

echo $HOME

Answer: it changed the output to place 1 file/directory per line. It also added some extra information about each.

$ ls -l
total 192
drwxr-x--- 2 training01 training 2048 Jul 26 15:50 exp01
-rw-r----- 1 training01 training   97 Jul 26 15:50 file01
-rw-r----- 1 training01 training 2461 Jul 26 15:50 muscle.fq
-rw-r----- 1 training01 training  300 Jul 26 15:50 task01
-rw-r----- 1 training01 training  317 Jul 26 15:50 task02
-rw-r----- 1 training01 training  303 Jul 26 15:50 task03

Details:

drwxr-x--- 2 training01 training 2048 Jun 14 11:28 exp01
\--------/ ^ \--------/ \------/ \--/ \----------/ \---/
permission |  username   group   size    date       name
       /---^---\
       linkcount

Where:

• permissions: 4 parts, file type, user perms, group perms and other perms
  • filetype: 1 character, d = directory and - regular file
  • user permissions: 3 characters, r = read, w = write, x = execute and - no permission
  • group permissions: same as user except for users within the owner group
  • other permissions: same as user except for users that are not in either user or group
• username: the user who owns this file/directory
• group: the group name who owns this file/directory
• size: the number of bytes this file/directory takes to store on disk
• date: the date and time when this file/directory was last edited
• name: name of the file
• linkcount: technical detail which represents the number of links this file has in the file system (safe to ignore)

Answer:

• exp01: Directory (given the 'd' as the first letter of its permissions)
• muscle.fq: Regular File (given the '-')

If you pass the -l flag to ls it will display a "long" listing of file information including file permissions.

There are various ways you could find out the permissions on your home directory.

Method 1: given we know the CWD is our home directory.

$ ls -l ..
...
drwxr-x--- 4 trainingXY training  512 Feb  9 14:18 trainingXY
...

The .. refers to the parent directory.

Method 2: using $HOME. This works no matter what our CWD is set to.

You could list the permissions of all files and directories in the parent directory of your home:

$ ls -l $HOME/..
...
drwxr-x--- 4 trainingXY training  512 Feb  9 14:18 trainingXY
...

In this case we use the shell variable to refer to our home directory.

Method 3: using ~ (tilde) shortcut

You may also refer to your home directory using the ~ (tilde) character:

$ ls -l ~/..
...
drwxr-x--- 4 trainingXY training  512 Feb  9 14:18 trainingXY
...

All 3 of the methods above mean the same thing.

You will see a list of files and directories in the parent directory of your home directory. One of them will be the name of your home directory, something like trainingXX. Where XX is replaced by a two digit string

Altername: using the -a flag and looking at the . (dot) special file.

$ ls -la
...
drwxr-x--- 4 trainingXY training  512 Feb  9 14:18 .
...

Answer: drwxr-x---

• You: read (see filenames), write (add, delete files), execute (change your CWD to this directory).
• Training users: read, execute
• Everyone else: No access

Discussion on Permissions:

The permission string is "drwxr-x---". The d means it is a directory. The rwx means that the owner of the directory (your user account) can read, write and execute the directory. Execute permissions on a directory means that you can cd into the directory. The r-x means that anyone in the same user group as training can read or execute the directory. The --- means that nobody else (other users on the system) can do anything with the directory.

Use the following command to view the man page for ls:

$ man ls

Answer: You should discover that the -h option prints file sizes in human readable format

-h, --human-readable
              with -l, print sizes in human readable format (e.g., 1K 234M 2G)

$ ls -lh
...
-rw-r----- 1 training01 training 2.5K Jun 14 11:28 muscle.fq

Answer: it changed the output so the filesize of muscle.fq is now 2.5K instead of 2461

The pwd command prints the value of your current working directory.

$ pwd
/home/training01

$ ls /
applications-merged  etc         media    opt      srv
bin                  home        misc     proc     sys
boot                 lib         mnt      root     tmp
data                 lib64       net      sbin     usr
dev                  lost+found  oldhome  selinux  var

Here we see that ls can take a filepath as its argument, which allows you to list the contents of directories other than your current working directory.

The cd command changes the value of your current working directory. To change to the root directory use the following command:

$ cd /

Answer: Yes, it now says the CWD is / instead of ~.

Some people imagine that changing the working directory is akin to moving your focus within the file system. So people often say "move to", "go to" or "charge directory to" when they want to change the working directory.

The root directory is special in Unix. It is the topmost directory in the whole file system.

Assuming you have changed to the root directory then this can be achieved with ls, or ls -a (for all files) or ls -la for a long listing of all files.

If you are not currently in the root directory then you can list its contents by passing it as an argument to ls:

$ ls
applications-merged  etc         media    opt      srv
bin                  home        misc     proc     sys
boot                 lib         mnt      root     tmp
data                 lib64       net      sbin     usr
dev                  lost+found  oldhome  selinux  var

Answer: Yes, we got the same output as exercise 3.2

Use the cd command to change your working directory to your home directory. There are a number of ways to refer to your home directory:

cd $HOME

is equivalent to:

cd ~

The simplest way to change your current working directory to your home directory is to run the cd command with no arguments:

Answer: the simplest for is cd with NO options.

cd

This is a special-case behaviour which is built into cd for convenience.

Answer: cd /home/group/common/training/Intro_to_Unix

You can do this with ls

$ ls
expectations.txt  hello.c  hi  jude.txt  moby.txt  sample_1.fastq  sleepy

Answer: 7 files (expectations.txt hello.c hi jude.txt moby.txt sample_1.fastq sleepy)

Use the file command to get extra information about the contents of a file:

Assuming your current working directory is /home/group/common/training/Intro_to_Unix

$ file sleepy
sleepy: Bourne-Again shell script text executable

Otherwise specify the full path of sleepy:

$ file /home/group/common/training/Intro_to_Unix/sleepy
sleepy: Bourne-Again shell script text executable

Answer: Bourne-Again shell script text executable

The "Bourne-Again shell" is more commonly known as BASH, "text" means you can read it (rather than a program) and "executable" means it can be run like other programs in the system. The file command is telling us that sleepy is (probably) a shell script written in the language of BASH.

The file command uses various heuristics to guess the "type" of a file. If you want to know how it works then read the Unix manual page like so:

man file

Use the file command again. If you are in the same directory as hi then:

$ file hi
ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 
2.6.9, not stripped

Answer: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.9, not stripped

This rather complicated output is roughly saying that the file called hi contains a binary executable program (raw instructions that the computer can execute directly).

You can find the permissions of sleepy using the ls command with the -l flag. If you are in the same directory as sleepy then:

$ ls -l sleepy
-rw-r--r-- 1 arobinson common 183 Feb  9  2015 sleepy

Answer: The Answer is dependent on the computer you are connected too however will follow something like above. We can see that this particular instance of sleepy is owned by the user arobinson, and is part of the common user group. It is 183 bytes in size, and was last modified on the 9th of February. The file is readable to everyone, and write-able only to arobinson. The digit '1' between the file permission string and the owner indicates that there is one link to the file. The Unix file system allows files to be referred to by multiple "links". When you create a file it is referred to by one link, but you may add others later. For future reference: links are created with the ln command.

You should know how to do this with the cd command:

cd

Make sure you are in your home directory first. If not cd to your home directory.

Use the mkdir command to make new directories:

$ mkdir test

Use the ls command to check that the new directory was created.

$ ls
exp01  file01  muscle.fq  task01  task02  task03  test

Use the cp command to copy files.

There are a number of ways you could do this depending on how you specify the source and destination paths to cp. You only need to perform one of these ways, but we show multiple ones for your reference.

Answer 1: From your home directory:

$ cp /home/group/common/training/Intro_to_Unix/* test

Answer 2: Change to the test directory and then copy (assuming you started in your home directory):

$ cd test
$ cp /home/group/common/training/Intro_to_Unix/* .

In the example above the '.' (dot) character refers to the current working directory. It should be the test subdirectory of your home directory.

Answer 3: Change to the /home/group/common/training/Intro_to_Unix directory and then copy:

cd /home/group/common/training/Intro_to_Unix/
cp * ~/test

Remember that ~ is a shortcut reference to your home directory.

Use ls -l to check the size of files.

You could do this in many ways depending on the value of your working directory. We just show one possible way for each file:

$ ls -l /home/group/common/training/Intro_to_Unix/expectations.txt

$ ls -l ~/test/expectations.txt

From the output of the above commands you should be able to see the size of each file and check that they are the same.

Answer: They should each be 1033773 bytes

Alternate: Sometimes it is useful to get file sizes reported in more "human friendly" units than bytes. If this is true then try the -h option for ls:

$ ls -lh /home/group/common/training/Intro_to_Unix/expectations.txt
-rw-r--r-- 1 arobinson common 1010K Mar 26  2012 /home/group/common/training/Intro_to_Unix/expectations.txt

In this case the size is reported in kilobytes as 1010K. Larger files are reported in megabytes, gigabytes etcetera.

Use the diff command to compare the contents of two files.

$ diff /home/group/common/training/Intro_to_Unix/expectations.txt expectations.txt

If the two files are identical the diff command will NOT produce any output)

Answer: Yes, they are the same since no output was given.

Use the wc (for "word count") to count the number of characters, lines and words in a file:

$ wc expectations.txt
  20415  187465 1033773 expectations.txt

Answer: There are 20415 lines, 187465 words and 1033773 characters in expectations.txt.

To get just the line, word or character count:

$ wc -l expectations.txt
20415 expectations.txt
$ wc -w expectations.txt
187465 expectations.txt
$ wc -c expectations.txt
1033773 expectations.txt

Take some time to play around with the nano text editor.

Nano is a very simple text editor which is easy to use but limited in features. More powerful editors exist such as vim and emacs, however they take a substantial amount of time to learn.

Use diff to check that the two files are different after you have made the change to the copy of expectations.txt in your ~/test directory.

diff ~/test/expectations.txt \
/home/group/common/training/Intro_to_Unix/expectations.txt

You could also use ls to check that the files have different sizes.

Answer: No, diff now shows 1 line different so they are independent files

Use the mv command to rename the file:

$ mv expectations.txt foo.txt
$ ls
foo.txt  hello.c  hi  jude.txt  moby.txt  sample_1.fastq  sleepy

Use the mv command to rename the file:

$ mv foo.txt expectations.txt
$ ls
expectations.txt  hello.c  hi  jude.txt  moby.txt  sample_1.fastq  sleepy

Use ls to check that the file is in fact renamed.

Use the rm command to remove files (carefully):

$ rm expectations.txt
$ ls
hello.c  hi  jude.txt  moby.txt  sample_1.fastq  sleepy

You could use the rm command to remove each file individually, and then use the rmdir command to remove the directory. Note that rmdir will only remove directories that are empty (i.e. do not contain files or subdirectories).

A faster way is to pass the -r (for recursive) flag to rm to remove all the files and the directory in one go:

Logical Answer:

cd ~
rm test/*
rmdir test

Easier Answer:

cd ~
rm -r test

Repeat exercises 4.1 and 4.2.

$ cd ~
$ mkdir test
$ cp /home/group/common/training/Intro_to_Unix/* test

$ cd ~/test
$ cat hello.c
#include <stdio.h>
int main(void) {
    printf ("Hello World\n");
    return 0;
}

hello.c contains the source code of a C program. The compiled executable version of this code is in the file called hi, which you can run like so:

$ ./hi
Hello World

$ head -n 20 sample_1.fastq
@IRIS:7:1:17:394#0/1
GTCAGGACAAGAAAGACAANTCCAATTNACATTATG
+IRIS:7:1:17:394#0/1
aaabaa`]baaaaa_aab]D^^`b`aYDW]abaa`^
@IRIS:7:1:17:800#0/1
GGAAACACTACTTAGGCTTATAAGATCNGGTTGCGG
+IRIS:7:1:17:800#0/1
ababbaaabaaaaa`]`ba`]`aaaaYD\\_a``XT
@IRIS:7:1:17:1757#0/1
TTTTCTCGACGATTTCCACTCCTGGTCNACGAATCC
+IRIS:7:1:17:1757#0/1
aaaaaa``aaa`aaaa_^a```]][Z[DY^XYV^_Y
@IRIS:7:1:17:1479#0/1
CATATTGTAGGGTGGATCTCGAAAGATATGAAAGAT
+IRIS:7:1:17:1479#0/1
abaaaaa`a```^aaaaa`_]aaa`aaa__a_X]``
@IRIS:7:1:17:150#0/1
TGATGTACTATGCATATGAACTTGTATGCAAAGTGG
+IRIS:7:1:17:150#0/1
abaabaa`aaaaaaa^ba_]]aaa^aaaaa_^][aa

tail -n 8 sample_1.fastq
@IRIS:7:32:731:717#0/1
TAATAATTGGAGCCAAATCATGAATCAAAGGACATA
+IRIS:7:32:731:717#0/1
ababbababbab]abbaa`babaaabbb`bbbabbb
@IRIS:7:32:731:1228#0/1
CTGATGCCGAGGCACGCCGTTAGGCGCGTGCTGCAG
+IRIS:7:32:731:1228#0/1
`aaaaa``aaa`a``a`^a`a`a_[a_a`a`aa`__

$ grep Ahab moby.txt
"Want to see what whaling is, eh? Have ye clapped eye on Captain Ahab?"
"Who is Captain Ahab, sir?"
"Aye, aye, I thought so. Captain Ahab is the Captain of this ship."
... AND MUCH MUCH MORE ...

If you want to know how many lines are in the output of the above command you can "pipe" it into the wc -l command:

$ grep Ahab moby.txt | wc -l
491

which shows that there are 491 lines in moby.txt that contain the word Ahab.

Use the -i flag to grep to make it perform case insensitive search:

$ grep -i the expectations.txt
The Project Gutenberg EBook of Great Expectations, by Charles Dickens
This eBook is for the use of anyone anywhere at no cost and with
re-use it under the terms of the Project Gutenberg License included
[Project Gutenberg Editor's Note: There is also another version of
... AND MUCH MUCH MORE ...

Again, "pipe" the output to wc -l to count the number of lines:

$ grep -i the expectations.txt  | wc -l
8165

Check the file size of sample_1.fastq before compressing it:

# check filesize
$ ls -l sample_1.fastq
-rw-r--r-- 1 training01 training 90849644 Jun 14 20:03 sample_1.fastq

# compress it (takes a few seconds)
$ gzip sample_1.fastq

# check filesize (Note: its name changed)
$ ls -l sample_1.fastq.gz
-rw-r--r-- 1 training01 training 26997595 Jun 14 20:03 sample_1.fastq.gz

# decompress it
$ gunzip sample_1.fastq.gz

$ ls -l sample_1.fastq
-rw-r--r-- 1 training01 training 90849644 Jun 14 20:03 sample_1.fastq

You will see that when it was compressed it is 26997595 bytes in size, making it about 0.3 times the size of the original file.

Note: in the above section the lines starting with # are comments so don't need to be copied but if you do then they won't do anything.

We can answer this question by counting the number of lines in the file and dividing by 4:

$ wc -l sample_1.fastq
3000000

Answer: There are 3000000 lines in the file representing 750000 reads.

If you want to do simple arithmetic at the command line then you can use the "basic calculator" called bc:

$ echo "3000000 / 4" | bc
750000

bc is suitable for small calculations, but it becomes cumbersome for more complex examples. If you want to do more sophisticated calculations then we recommend to use a more general purpose programming language (such as Python etcetera).

Use grep to find all the lines that contain GATTACA and "pipe" the output to wc -l to count them:

$ grep GATTACA sample_1.fastq | wc -l
1119

Answer: 1119

If you are unsure about the possibility of upper and lower case characters then consider using the -i (ignore case option for grep).

You can use the -n flag to grep to make it prefix each line with a line number:

Answer 1:

$ grep -n GATTACA sample_1.fastq
5078:AGGAAGATTACAACTCCAAGACACCAAACAAATTCC
7170:AACTACAAAGGTCAGGATTACAAGCTCTTGCCCTTC
8238:ATAGTTTTTTCGATTACATGGATTATATCTGTTTGC
... AND MUCH MUCH MORE ...

Answer 2: Or you can use the nl command to number each line of sample_1.fastq and then search for GATTACA in the numbered lines:

$ nl sample_1.fastq | grep GATTACA
  5078  AGGAAGATTACAACTCCAAGACACCAAACAAATTCC
  7170  AACTACAAAGGTCAGGATTACAAGCTCTTGCCCTTC
  8238  ATAGTTTTTTCGATTACATGGATTATATCTGTTTGC
... AND MUCH MUCH MORE ...

Just the line numbers:

If you just want to see the line numbers then you can "pipe" the output of the above command into cut -f 1:

$ nl sample_1.fastq | grep GATTACA | cut -f 1
  5078
  7170
  8238
... AND MUCH MUCH MORE ...

cut will remove certain columns from the input; in this case it will remove all except column 1 (a.k.a. field 1, hence the -f 1 option)

$ grep -n GATTACA sample_1.fastq | cut -d: -f 1
5078
7170
8238
... AND MUCH MUCH MORE ...

$ nl sample_1.fastq
     1  @IRIS:7:1:17:394#0/1
     2  GTCAGGACAAGAAAGACAANTCCAATTNACATTATG
     3  +IRIS:7:1:17:394#0/1
     4  aaabaa`]baaaaa_aab]D^^`b`aYDW]abaa`^
     5  @IRIS:7:1:17:800#0/1
     6  GGAAACACTACTTAGGCTTATAAGATCNGGTTGCGG
     7  +IRIS:7:1:17:800#0/1
     8  ababbaaabaaaaa`]`ba`]`aaaaYD\\_a``XT
... AND MUCH MUCH MORE ...

There are a lot of lines in that file so this command might take a while to print all its output. If you get tired of looking at the output you can kill the command with control-c (hold the control key down and simultaneously press the "c" character).

$ nl sample_1.fastq > sample_1.fastq.nl

The greater-than sign ">" is the file redirection operator. It causes the standard output of the command on the left-hand-side to be written to the file on the right-hand-side.

You should notice that the above command is much faster than printing the output to the screen. This is because writing to disk can be performed much more quickly than rendering the output on a terminal.

To check that the first 20 lines of the file look reasonable you can use the head command like so:

$ head -n 20 sample_1.fastq.nl
     1  @IRIS:7:1:17:394#0/1
     2  GTCAGGACAAGAAAGACAANTCCAATTNACATTATG
     3  +IRIS:7:1:17:394#0/1
     4  aaabaa`]baaaaa_aab]D^^`b`aYDW]abaa`^
     5  @IRIS:7:1:17:800#0/1
     6  GGAAACACTACTTAGGCTTATAAGATCNGGTTGCGG
     7  +IRIS:7:1:17:800#0/1
     8  ababbaaabaaaaa`]`ba`]`aaaaYD\\_a``XT
...

The less command allows you to interactively view a file. The arrow keys move the page up and down. You can search using the '/' followed by the search term. You can quit by pressing "q". Note that the less command is used by default to display man pages.

$ less sample_1.fastq.nl

The '-' (dash) character has a special meaning when used in place of a file; it means use the standard input instead of a real file. Note: while it is fairly common in most Unix programs, not all will support it.

The paste command is useful for merging multiple files together line-by-line, such that the Nth line from each file is joined together into one line in the output, separated by default with a tab character. In the above example we give paste 4 copies of the contents of sample_1.fastq, which causes it to join consecutive groups of 4 lines from the file into one line of output.

Answer: No, in the second instance we get 4 copies of each line.

Why: In the first command paste will use the input file (standard input) 4 times since the cat command will only give one copy of the file to paste, where as, in the second command paste will open the file 4 times. Note: this is quite confusing and is not necessory to remember; its just an interesting side point.

We can count the number of lines in sample_1.tsv using wc:

$ wc -l sample_1.tsv

The output should be 750000 as expected (1/4 of the number of lines in sample_1.fastq).

To view the first 20 lines of sample_1.tsv use the head command:

$ head -20 sample_1.tsv

The file sample_1.tsv is in column format. The cut command can be used to select certain columns from the file. The DNA sequences appear in column 2, we select that column using the -f 2 flag (the f stands for "field").

cut -f 2 sample_1.tsv > sample_1.dna.txt

Check that the output file looks reasonable using head or less.

$ sort sample_1.dna.txt > sample_1.dna.sorted.txt

Running head on the output file reveals that there are duplicate DNA sequences in the input FASTQ file.

$ uniq sample_1.dna.sorted.txt > sample_1.dna.uniq.txt

Compare the outputs of:

$ wc -l sample_1.dna.sorted.txt
750000
$ wc -l sample_1.dna.uniq.txt
614490

View the contents of sample_1.dna.uniq.txt to check that the duplicate DNA sequences have been removed.

Use the -d flag to uniq to print out only the duplicated lines from the file:

$ uniq -d sample_1.dna.sorted.txt > sample_1.dna.dup.txt

Finally we can 'pipe' all the pieces together into a sophisticated pipeline which starts with a FASTQ file and ends with a list of duplicated DNA sequences:

Answer:

$ cat sample_1.fastq | paste - - - - | cut -f 2 | sort | uniq -d | wc -l
56079

The output file should have 56079 lines.

Put the answer to 5.13 into a file called sample_1_dups.sh (or whatever you want). Use nano to create the file.

Answer: the contents of the file will look like this:

#!/bin/bash

cat sample_1.fastq | paste - - - - | cut -f 2 | sort | uniq -d | wc -l

Give everyone execute permissions on the file with chmod:

$ chmod +x sample_1_dups.sh 

You can run the script like so:

$ ./sample_1_dups.sh

If all goes well the script should behave in exactly the same way as the answer to 5.13.

Copy the shell script from 5.14 into a new file:

$ cp sample_1_dups.sh fastq_dups.sh

Edit the new shell script file and change it to use the command line parameters:

#!/bin/bash

cat $1 | paste - - - - | cut -f 2 | sort | uniq -d | wc -l

You can run the new script like so:

$ ./fastq_dups.sh sample_1.fastq

In the above example the script takes sample_1.fastq as input and prints the number of duplicated sequences as output.

A better Answer:

Ideally we would write our shell script to be more robust. At the moment it just assumes there will be at least one command line argument. However, it would be better to check and produce an error message if insufficient arguments were given:

#!/bin/bash
if [ $# -eq 1 ]; then
    cat $1 | paste - - - - | cut -f 2 | sort | uniq -d | wc -l
else
    echo "Usage: $0 <fastq_filename>"
    exit 1
fi

The 'if ...; then' line means: do the following line(s) ONLY if the ... (called condition) bit is true.

The 'else' line means: otherwise do the following line(s) instead. Note: it is optional.

The 'fi' line means: this marks the end of the current if or else section.

The '[ $# -eq 1 ]' part is the condition:

• $#: is a special shell variable that indicates how many command line arguments were given.
• -eq: checks if the numbers on either side of it are equal.
• 1: is a number one

So in words our script is saying "if user provided 1 filename, then count the duplicates, otherwise print an error".

We can add a loop to our script to accept multiple input FASTQ files:

#!/bin/bash
for file in $@; do
    dups=$(cat $file | paste - - - - | cut -f 2 | sort | uniq -d | wc -l)
    echo "$file $dups"
done

There's a lot going on in this script.

The $@ is a sequence of all command line arguments.

The 'for ...; do' (a.k.a. for loop) iterates over that sequence one argument at a time, assigning the current argument in the sequence to the variable called file.

The $(...) allow us to capture the output of another command (in-place of the ...). In this case we capture the output of the pipeline and save it to the variable called dups.

If you had multiple FASTQ files available you could run the script like so:

./fastq_dups.sh sample_1.fastq sample_2.fastq sample_3.fastq

And it would produce output like:

sample_1.fastq 56079
sample_2.fastq XXXXX
sample_3.fastq YYYYY