This section lists some of the most commonly used Linux tools and other concepts. There is much more to learn, and the more of these little building blocks are at your disposal the more productive you will be.
The df command (disk free) shows the available space on all devices:
% df
Filesystem | 1K-blocks | Used | Available | Use% | Mounted on |
/dev/sda1 | 61169696 | 56601044 | 1461328 | 98% | / |
udev | 1986456 | 4 | 1986452 | 1% | /dev |
tmpfs | 798580 | 864 | 797716 | 1% | /run |
none | 5120 | 0 | 5120 | 0% | /run/lock |
none | 1996444 | 24 | 1996420 | 1% | /run/shm |
/dev/sdb1 | 4874192 | 1246720 | 3627472 | 26% | /media/mystick |
The first column shows the device files used by the system to access the hardware. The last column shows the corresponding mount points. The first partition on the fixed hard disk /dev/sda1 is mounted as / meaning the root of the file system.
Removable devices such as USB sticks are mounted to directories automatically. Here a removable drive is mounted via device file /dev/sdb1 as /media/mystick. This directory is accessible as long as the stick is plugged in.
Remember to unmount it (usually via the file manager) before unplugging the device, otherwise data written to the stick might be lost.
Note that in some distributions the mount directory is /media/username/mystick
As always, the hard disk is almost completely full. Sooner or later this will cause problems. There should be some percentage reserved for root which is not contained in the Use% above. However, if for some reason the Linux system runs out of space completely, strange situations can occur, such as users not being able to log in. Check with the following command:
% sudo tune2fs -l /dev/sda1
...
Last mounted on: /
...
Filesystem state: clean
...
Block count: ..
Reserved block count: ..
The filesystem is clean (if not, we are in trouble! boot from rescue/install disk/usb stick and fsck). Reserved block count divided by Block count should be about 5%, the default reserved for root.
Sometimes it is not entirely clear which types of filesystems are actually in use on the various mount points. The following command lists the filesystem for each partition, including encrypted ones:
sudo blkid
The top command provides an overview of system activity and various data on the machine, such as
Press 1 to get a listing for each core. Exit the listing by pressing q.
Most processes are in the sleep mode (S) most of the time, consuming very little resources. R means running, D means dead, and sometimes Z for zombie can be seen.
The grep command searches text files for lines containing the given string and prints them out:
grep h2 *.html
This will print all headings of level 2 in all HTML files in the current directory (if markup and text is on the same line).
grep printf *.java | wc -l
This command counts the number of usages of the printf statement in all Java programs.
grep -l printf *.java | sort -u
This command prints the file names rather than the matching line, and the sort command with the unique option drops doubles. The result is a list of the files that contain at least one printf statement.
du -s tmp/* | sort -n
Disk usage for all files and directories in tmp/ piped to sort numerically. Note that disk usage and file size are usually not the same.
Directory hierarchies can develop into huge structures that are difficult to search interactively. The find command searches all directories starting at the given point(s) for files matching some condition:
find . -name '*.zip' -size +1000k
Here we look for ZIP files bigger than 1 MB.
find . -name '*.java' -exec grep -l printf {} \; | sort -u
In this case we perform the grep from the previous section on all Java files in the current directory, or any subdirectory. Note the {} syntax to access the currently found file name, and the \; at the end.
find . -name '*.zip' -size +1000k -exec ls -s {} \; | sort -n
Again, search for big ZIP files, list them with block usage, and sort the result numerically, i.e. by file size.
Looks at file content and tries to detect type and other info; -i for mime type.
% file -i *.html
shell.html: text/html; charset=us-ascii
tools.html: text/html; charset=us-ascii
There are many backup solutions, all with their own quirks; mostly they use their own formats, and restores are often more unreliable than expected. This has caused a lot of grief for countless users.
The rsync command provides a very simple and elegant solution. It creates a full copy of the given directory tree(s) on the backup medium, allowing direct access to files without a restore operation. It is also very fast, since it only copies those files that have changed since the last backup. Consider the following shell script:
#!/bin/sh
# removable media
dest=/media/usbdisk/bak/
# rsync options
OPTS="-av"
# check media is plugged in
if [ ! -x $dest ]; then
echo "Backup media not mounted"
exit 1
fi
# local host
cd
rsync $OPTS vw publ projects "$dest/pc"
# remote host
rsync $OPTS balrog:www/le "$dest/balrog/www"
Our backup medium is a USB stick or external hard disk with a partition labelled usbstick. This partition contains a directory bak at the top level. It may contain other data, which remains unaffected.
Here are the more commonly used editors; they often come installed with the system.
The basic development framework of Unix is the makefile; newer versions include helpful features such as automated configuring, but the initial concept is still valid. The makefile specifies the dependencies among source files and binaries, and the actions to be taken when source files have changed. The make command consults the makefile and acts accordingly. Here is a sample makefile:
JAVAC=/usr/lib/jvm/java-6-sun-1.6.0.03/bin/javac
JAVA=/usr/lib/jvm/java-6-sun-1.6.0.03/bin/java
Hello.class: Hello.java
$(JAVAC) Hello.java
$(JAVA) Hello
hello: hello.c
cc -o hello hello.c
./hello
In this case the Java compiler and runtime are defined, and dependencies and compile actions for a simple project are stated; similarly for a sample project in the C language.
Additional applications that are not provided via the package manager can be compiled and installed. This involves downloading the source code, often from a git repository. Look for files called README and INSTALL for instructions on how to compile.
You probably need to install development libraries or even the compiler itself first, as they are usually not included in desktop distributions. Use your package manager to do that.
Typically a configure script will be included with the software to check dependencies and set options so that the make command will succeed with compilation. In some cases the configure script itself is generated by autogen.sh. If all goes well, the following sequence of commands will compile and install the application:
./configure
make
sudo make install
However, typically some additional libraries or other pieces of code must be downloaded, compiled, and installed first. Of course, the process can continue recursively.
The package manager is the preferred way to install software. Only compile and install from source if
Compiling and installing end-user applications that are not depended on by other components is usually not problematic. Otherwise, be very careful:
Good old simple passwords like Netw0rk or g0ldf1sh can be cracked almost instantly nowadays. Graphics cards can be used for massive parallel processing and password cracking; any password of up to 7 random characters (upper case, lower case, and digits) can be cracked in about 18 minutes with a humble Radeon 5770 [1]. With 9 characters the same card would slave for about 48 days. However, parallel and distributed processing with graphics cards and other (possibly hijacked) devices achieve much higher speeds.
A strong password should be
Obviously such passwords are rather hard to remember. Therefore, passphrases are often used instead of passwords.
A common approach for creating secure passphrases is to chose several random words from a list of several thousand. A sufficiently long passphrase is both harder to crack and easier to remember than random passwords; e.g.
Method | Combinations |
---|---|
five word passphrase from a list of 5000 words | 5000^5 = 3,125,000,000,000,000,000 |
10 character password, upper and lower case plus digits | 62^10 = 839,299,365,868,340,224 |
Note that the well-known passphrase "correct horse battery staple" is considered too short nowadays, five or even six words are recommended. A passphrase generator can be found e.g. at fourmilab.ch.
A keyring holds a number of keys that are provided to applications automatically once the keyring is unlocked. The keyring 'default' will be created the first time you check the option 'Remember forever' when you enter a passphrase.
The keyring holds various keys, e.g. for ssh, email access, and encrypted USB devices. When unlocked the keyring provides those keys to all applications for the duration of the session. Therefore, once you have entered your keys into the keyring and saved them permanently (option Remember Forever) it reduces passphrase entry to one per session.
Even more conveniently, the keyring can be unlocked with the login password, if it is identical to the keyring passphrase, but this requires some setup depending on your distribution; e.g. on Linux Mint 17 the startup applications 'Certificate and Key Storage' and 'Secret Storage Service' must be added in Settings/Session and Startup.
USB sticks get lost, and notebooks get stolen. Do you want strangers and criminals to access your data? LUKS and the cryptsetup package provide a simple yet reliable method of encryption:
The following commands show the setup for encryption on a USB stick and a FAT 32 file system:
sudo bash
apt-get install cryptsetup
lsmod | grep crypt
$ df
Filesystem ... Mounted on
...
/dev/sdc1 ... /media/DISK_IMG
$ umount /dev/sdc1
$ cryptsetup luksFormat /dev/sdc1
cryptsetup luksOpen /dev/sdc1 mystick
mkfs.vfat -F 32 -n mystick /dev/mapper/mystick
Or, if you are only using the external drive on Linux computers, create a proper ext4 Linux file system
mkfs.ext4 -m 0 -L mystick /dev/mapper/mystick
For external disks over 1 TB save some space on inodes with
mkfs.ext4 -m 0 -L mydisk -T huge /dev/mapper/mydisk
The number of inodes determines the maximum number of files, and inodes need space, too; the usage type huge reduces the number of inodes by a factor of four, and on a 2 TB disk this results in about 23 GB more usable space, with still an ample maximum of 30 million files.
Done. Remove the stick and plug it in again. You should be asked for the passphrase. Check the option Remember Forever. That way you only have to enter the passphrase for your keyring once at the begin of the session, not for every device you plug in.
The secure shell provides access to remote hosts with a high level of security. It uses public key cryptography to provide secure communication over an insecure network.
In the most simple version the public/private keys are generated automatically to encrypt the communication:
ssh otherhost
If your userid on the remote host is different use the following form:
ssh userid@otherhost
The password will be queried interactively. It is more convenient and actually safer to use a manually generated public/private key pair, which can be generated with
ssh-keygen
Choose a strong passphrase. It is used to encrypt the private key. Do NOT leave the passphrase empty. The directory .ssh on the local host will now contain a file id_rsa.pub which holds your public key. The public key needs to be transferred to the remote host. The command
ssh-copy-id otherhost
should work, but if it fails just use text editors on both ends and copy and paste the public key into a file .ssh/authorized_keys on the remote host. Create the directory .ssh if necessary.
Now you login to the remote host via ssh. Current Linux distributions should be set up so that you are asked the passphrase the first time, then you can save it in your keyring (option 'Remember Forever').
All installed programs should have a manual page:
man chmod
man ls
man cp
Use the PageUp/PageDown keys to navigate, and the q key to exit.