Operating Systems Introduction to Linux Linux is an operating system that was initially created as a hobby by a young student, Linus Torvalds, at the University of Helsinki in Finland. Linus had an interest in Minix, a small UNIX system, and decided to develop a system that exceeded the Minix standards. He began his work in 1991 when he released version 0.02 and worked steadily until 1994 when version 1.0 of the Linux Kernel was released. The kernel, at the heart of all Linux systems, is developed and released under the GNU (GNU's Not Unix) General Public License (GNU (GNU's Not Unix) GPL (GNU General Public License)) and its source code is freely available to everyone. It is this kernel that forms the base around which a Linux operating system is developed. There are now literally hundreds of companies and organizations and an equal number of individuals that have released their own versions of operating systems based on the Linux kernel. Because of Linux's success on the x86 desktop, and the proliferation and porting to various processor architectures over the past two years, Linux has become very popular on embedded devices, especially consumer products, telecom routers and switches, Internet appliances, and industrial and automotive devices. Using Linux in embedded devices which require some intelligence is attractive for many reasons: it is a mature, robust operating system it supports a large number of devices, file systems, and networking protocols upgrades and new features are constantly being added, tested and refined by a large community of programmers and users it gives everyone from developers to end users complete visibility of the source code a large number of applications (such as GNU (GNU's Not Unix) software) exist which require little to no porting effort the low cost of Linux Some examples of Linux in embedded systems can be found at Linux Devices. What is Linux? Linux is a monolithic, multitasking, multiuser operating system (OS (Operating System)) which is a UNIX workalike. It is modularly designed with well understood interdependencies, which allows one to tailor Linux for a variety of purposes. In this course, Linux exists on the development workstation as a full blown OS (Operating System) with an incredibly large variety of features. However, it also exists on the target system as a very small, focused OS (Operating System) appropriate for an embedded system. An exciting attribute of Linux is that it can be run on a wide variety of microprocessors and microcontrollers. Once you are comfortable with Linux, you will find this to be a great advantage in the embedded systems arena. In particular, your development platform is a Linux box which can be used for many different embedded targets spanning a range of processor architectures, but all running Linux. It is not the purpose of this introductory chapter to explore Linux in either great depth or breadth. The assumption is that the course participants are quite computer literate, but may have little previous experience with this particular OS (Operating System). Hence our goal is to get up and running on a generic Linux box. To accomplish this the following sections will look at: logging on to a Linux system the file system a selection of commands editing text files Then we'll close out the chapter with some activities/exercises followed by salient references. Loggin On If you are a new Linux user and have your own machine, then you will be able to log in as root or as a user. Our assumption here is that you are logging in as a user. Working as root (also called super user or system administrator) level should be reserved for tasks that really require it e.g. (for example) many system administration tasks. When Linux boots up, it either boots into the X Windows System (providing a GUI (Graphical User Interface)) or into console mode (no GUI (Graphical User Interface)). In either case, you will be prompted to log on. The logon is done in two steps in response to two successive prompts requiring that you: enter your username enter your password So the system administrator must provide you a username and password ahead of time. Upon successful logon, the OS (Operating System) starts a shell which responds to your input. There are many possible shells with the bash shell being the default shell for most Linux distributions. However, an embedded system is likely to use a more compact and less powerful shell. The shell interprets commands you enter at the command line, but also is a powerful scripting language. In particular, the shell can interpret text files which are written using the appropriate shell's grammar. It is beyond the scope of this course to investigate shell scripting. At any rate, once successfully logged in, you can enter commands and, in the X Windows environment, point and click on various icons. To log off, type exit at the command line. If you have your own personal Linux machine and need to power down, this must be done carefully. In particular, you need to be logged on with root privileges (don't exit) and then enter at the command line shutdown -h now There are variations on this (see man shutdown). The File System Linux, like most operating systems, has a hierarchical file structure. Application developers would like sufficient standardization in that hierarchy (and in other aspects of Linux) that they need not tailor their application's installation and operation for each style of Linux distribution. There is an evolving Linux Standard Base and perhaps the most mature component is the file system hierarchy. However, the space and performance constraints of an embedded system will force an embedded Linux to stray from the standard. Nor is that particularly serious; an embedded system is a special purpose system, less likely to cause an application developer pain. In this section we'll look at the file system hierarchy standard and then at the hierarchy found in uClinux, as an example of an embedded system. Note that our embedded system development will typically be on the Linux workstation where the standard is hopefully followed. At the same time an understanding of the uClinux hierarchy will give us insight into the embedded Linux world. The File System Hierarchy Standard The File System Hierarchy Standard, which at this writing is at Version 2.3 can be found here. We'll give an overview of what we need for this course, but you are encouraged to look at the standard for the underlying rationale. The root of the file hierarchy is symbolized simply by /. There is some confusion possible because one of the top level subdirectories under / is called root, but the latter is essentially a home directory for the root user (system administrator). The immediate subdirectories of / are described in the following table. The File System Hierarchy Standard discusses each of these directories in some detail, not only the contents, but accepted conventions for their usage. Of course, the subdirectories contain files and further subdirectories and so on. Note that a directory is actually one of several kinds of files from the perspective of the OS (Operating System), but it is convenient to make the distinction between directories and other files here. All common distributions allow one additional directory at the same level as those of Table 1, the proc directory. It is not part of the standard, but deserves mention. It is often called a pseudo directory. It contains information about the executing kernel. The files hold data on such things as the resource usage of current processes, what CPU is present, interrupt allocation, I/O port allocations, pci bus components, devices, modules currently installed, and so on. Much of the information is generated dynamically when it is accessed. Directory Name , Purpose/Contents bin, essential command binaries which may be used by the system administrator and by ordinary users, required for system boot, boot kernel image and configuration files used by the boot loader dev, device files etc, host-specific configuration files home, user home directories lib, essential shared libraries and kernel modules mnt , mount point for temporarily mounting file systems such as those on a CDROM or floppy disk opt , add-on application software packages root, the root user's home directory sbin, system binaries, essential for system administration, but not for system boot tmp, location of temporary files usr, secondary hierarchy, intended as shareable, read- only data. Here you'll find games, include files used by C programs, the linux source code hierarchy, and so on var, variable data such as spool directories and log files. The file system hierarchy in uClinux The directories at the top level of the unmodified uClinux hierarchy are tabulated next. As would be expected, these 'standard' directories are much sparser than a workstation resident Linux. Keep in mind that we do our development on our workstation rather than on the target. For example, if we write a C program to incorporate into our STAMP/uClinux target, it is written and compiled on the workstation - so there is no need for the C program include files to be in the uClinux hierarchy. Pathnames If your username is, say, jsmythe, then when you log on you will be placed in jsmythe/ as your current working directory. Where is this located with respect to the root of the file hierarchy? The top level under root contains the subdirectories of Table 1, which includes the home subdirectory. user home directories including jsmythe/ (the trailing slash indicates a directory, but may be omitted at the end of a pathname). Designating the root of the file system by a leading, solitary slash (/), we find the jsmythe log on directory by navigating like so / → home → jsmythe. This is compacted as a pathname /home/jsmythe If you used an editor to create the file vita.txt in this home directory, it would be found at /home/jsmythe/vita.txt / is called the parent directory of home,home/is the parent directory ofjsmythe, andjsmythe/is the parent directory ofvita.txt. A pathname that starts at the root of the file hierarchy starts with / and is called an absolute pathname. Clearly if you are 42 levels deep in the hierarchy, absolute pathnames become unwieldy. Hence, Linux also provides relative pathnames. Let's look at some examples of relative pathnames. If you have logged on to your home directory at /home/jsmythe, where you have created a file vita.txt, then its absolute pathname is, as before, /home/jsmythe/vita.txt but working from within the home directory /home/jsmythe/, you can refer to this file as vita.txt or as ./vita.txt The dot (.) indicates that the path starts at the current working directory; so, in our example, it effectively replaces /home/jsmythe. Hence, when working within /home/jsmythe, all three of the following are equivalent: /home/jsmythe/vita.txt ./vita.txt vita.txt. Similarly the double dot (..) represents the parent directory of the current working directory. File Permissions File permissions are part of the security system. They help protect the OS (Operating System) from users, whether malicious or clumsy protect each user from other users Permissions cover reading a file writing a file executing a file Further, a file has three user categories the file's owner the file's group all other users File permissions are written as three concatenated triads. The first triad refers to the file owner, and the second to the file group, and the third to all other user's. Here are some example permissions and what they mean: rwxrwxrwx - file owner, file group, and all others have read/write/execute permission. rw-rw—- - file owner and file group have read/write access, but all others have no access. rwxr-xr-x - file owner has read/write/execute access, while group and all others have read/execute access. Setuid and Getgid Programs The three concatenated permission triads (for owner, group, and others) are preceded by another field whose purpose it is to give the user of the program special permissions. In particular, the user's effective uid (user ID) or effective gid (group ID) can be set to the file owner's uid or gid. As a somewhat dated example consider a user changing his/her password in the days before shadow passwords became prevalent. The passwords were kept in /etc/passwd with permissions rw-r--r-and root as owner. So the average user does not have write access to change a personal password. However, the /usr/bin/passwd command has permissions r-s--x--x where the s means that the user's effective uid is set to that of root and will be allowed to update /etc/passwd. The following commands can be used to set the user's effective id and effective gid to that of the file's owner chmod u+s program … we now say the program is a setuid program chmod g+s program … we now say the program is a setgid program We'll see the chmod command in the next section. Commands Here we'll look at some useful commands which perform such tasks as file system manipulation and navigation, getting information, and modifying file permissions. Many of the commands care about permissions and the command will fail if you don't have the proper permissions. Linux, although relatively new, profits from the historical evolution of UNIX in that it comes close to achieving the following goals: there is a command or combination of commands to perform any task you can dream up the securely configured system prevents you from harming the system or other users Part of the philosophy of Linux (again, inherited from UNIX) is that commands tend to be very simple, but can be combined very compactly to do complex tasks. Note: Linux is case sensitive so Cat, CAT, and cat would be distinct - our typical commands will be lower case. In this chapter, we'll consider some useful commands and then look at what might be called plumbing - standard input, standard output, standard error, redirection, and pipes. These plumbing operators are not commands, but rather operators provided by the shell. They are used in concert with the commands, allowing the commands to be used in combination. Some useful commands From the many commands provided by Linux, we'll consider those tabulated next. We'll discuss these only briefly, and in most cases they have many further options which can be explored via the man pages. Command , Purpose ls, lists directory contents cd, changes from the current working directory to another pwd, show pathname of present working directory mkdir, make a directory rmdir, remove a directory (directory must be empty) rm, removes a file, but has a recursive option that can be used to remove non empty directories as well cp, copies a file mv, moves its argument (a file name) to a new name more, lists its argument a page at a time cat, concatenates its arguments and sends the result to standard out grep, print lines matching a pattern (regular expression) tar, archives its argument (with compression if specified) man, displays the manual pages of its argument su, switch to another user chmod, changes file/directory permissions chown, changes file/directory owner chgrp, changes file/directory group. The ls command If you want to list the contents of your current working directory, you can simply type ls and if you want lots of information (e.g. (for example), permissions, sizes), use the long form ls -l As you would expect, you can list contents of other directories (as allowed by the permissions) e.g. (for example) ls -l /usr/src Other options allow sorting on such information as date etc. The cd command This allows you to change your working directory to another, if permissions allow. Here are some examples: cd / change to the root directory of the file system cd /usr/src change to /usr/src cd .. change to the parent of your current directory, i.e. (in other words) one level up cd ../.. change two levels up cd - change back to the directory you were in before this one The pwd command This is the 'present working directory' command. It displays the absolute pathname of where in the directory hierarchy you currently are. The mkdir and rmdir command These can make and remove (empty) directories. Assuming our current working directory is /home/jsmythe, consider these sequential examples: mkdir emails - creates the directory /home/jsmythe/emails mkdir emails/outgoing - creates directory /home/jsmythe/emails/outgoing rmdir emails/outgoing - removes directory /home/jsmythe/emails/outgoing. The rm command This command allows you to delete files. For example, if you are in directory /home/jsmythe and want to delete a file at that level named vita.txt, you could enter rm vita.txt This is a very powerful command in that you can delete a whole chunk of a hierarchy. For example, if /home/jsmythe/project1/ was the top of a significant hierarchy of subdirectories and files, you can delete it via rm -rf /home/jsmythe/project1 The cp and mv command If you want a copy of a program, the format is cp existing new for example, cp vita.txt vita1.txt Now there are two files, the original (vita.txt) and a new copy (vita1.txt). On the other hand, if you want to rename a file, the syntax is mv existing new for example mv vita.txt resume.txt In this case, the file vita.txt has been replaced by the identical file, except that it has a new name, resume.txt. The mv command can also be used to rename any node in the file hierarchy e.g. (for example) mv project/ project1 The more command This command can display a file at the console, one screen full at a time. For example, more vita.txt Once invoked, more provides some helpful key strokes: space key - display next screen q key - quit A similar, but more powerful command isless. The cat command The cat command concatenates its arguments and displays the entire result at the console, so a multi-screen result will quickly scroll off the end. For example, cat file1 file2 The grep command The grep command can search a file or set of files for a pattern. For example, let's say we're searching for mention of sys_write in any of the C files in a directory. This could be done via grep 'sys_write' *.c Then grep would print all lines in all files mentioning the sought for pattern, sys_write. There is also a recursive grep (rgrep) available which can search recursively through a directory hierarchy. While we're on the subject of searching, there is a wonderful tool for searching through the source hierarchy which can be used over the web or downloaded. See http://lxr.linux.no The tar command We'll just give two examples of this convenient and powerful command. Let's say you have a project at work which you also pursue at home and are faced with moving the project back and forth as you work on it in both places. Let's say the project hierarchy starts at the node /home/jsmythe/project/ and consists of many subdirectories and files. To compress and archive the hierarchy into a single file called proj.tgz, use the command tar cvfz proj.tgz project/ where we have assumed that the command is issued from directory /home/jsmythe. To subsequently expand and unarchive the file on your other machine from directory /home/jsmythe, use tar xvfz proj.tgz and it will produce the /home/jsmythe/project hierarchy if it does not currently exist and append to it if it does. Linux provides a powerful tool called cvs for help in managing projects which have multiple developers. The man command If a manual page on some entity, name, exists, then man name will give the information. For example, if you want to learn what options are supported for the ls command, enter man ls or to learn about the vi editor, enter man vi. You can even learn more about man by entering man man. The su command The switch user command (sometimes misnamed the super user command) allows you to switch to a different user. If no argument is supplied, you will switch to super user if you successfully respond to the subsequent prompt for the super user password. If an argument (a valid username) is provided then you will switch to that user if you successfully respond to the subsequent prompt for that user's password. You return to your previous identity with the exit command. As you can see, it is essentially a nested log on scenario. In this document there are several command line instructions given that must be run as root. To switch to the root user, without having to logout from a user session, the switch user command may be used as follows: bash$ su - You will then be prompted to enter the password for root access, if this is successful a root session will be started. You can exit back to the user session by typing the command exit. If you wish to run graphical programs which use the XWindows interface then use the sux - command instead. This transfers your XWindows credentials. Alternately, to execute a single command as root use the sudo command as follows: bash$ sudo If this is your first time running sudo in this session you will be prompted for a password. After successfully entering the password your command will be run as root. The chmod, chown, and chgrp command These allow you to change permissions (chmod), ownership (chown), and group (chgrp) of files to which you have appropriate access. There are typically used for system administration. For example, as root I could change a file's owner and group to fred_smith (assuming there is such a user and group) via chown fred_smith.fred_smith filename Similarly, if an owner wants to change a file's permissions to rw-rw-r--, the appropriate command would be chmod 664 filename For those uncomfortable with octal permissions (like 664 representing rw-rw-r--), there is the option of using letter based syntax. Investigate these commands further via their man pages. Standard Input, Output, Error Commands and other programs often expect input to come from the keyboard, which we call standard input. Similarly they often send output to the console, which we call standard output. Error information is also typically sent to the console, but the destination is then called standard error. We'll see in the next section that the distinction between standard output and standard error is not as trivial as it first seems. A command such as more vita.txt sends its output to standard output, the console. Redirection and Pipes The simple concepts of standard input, output, and error become significantly more powerful when used with redirection and pipes. Some examples are in order. Redirection Recall that the command cat file1 file2 will concatenate file1 and file2, sending the output to standard out, the console. However, standard output can be redirected to a file other than the console e.g. (for example) cat file1 file2 > file3 where the right arrow (>) is the redirection operator for standard output. In this case file3 will hold the concatenated result of file1 and file2. In the case just shown file3 is created and any existing file by that name would have been destroyed. If file3 already existed and you wanted to append the new information, you would enter cat file1 file2 >> file3 where the double right arrow is called the nondestructive redirection operator. Similarly, the left arrow (<) is the redirection operator for standard input. There is also a syntax (somewhat shell dependent) for redirecting standard error. For example, in the bash shell we might perform a compilation with this syntax: gcc file5.c -o file5.c 2> file5.err where the last piece 2> file5.err redirects standard error (but not standard output) to the file called file5.err. This is quite useful when your compilation triggers many errors which scroll down the screen. Typically, the early errors are significant, but may scroll off the screen. However, they will be available in a redirected error file such as the example's file5.err. To capture all the output of a command into a log file use the following syntax: gcc file5.c -o file5.c > file5.log 2>&1 This directs the output of the compile to file5.log and also sends the error output to the same output file. These redirection operators appear very handy, but when combined with the pipe concept the power is much enhanced. The Pipe The pipe connects commands so the output of one becomes input for the second. Let's look at its use without redirection, first. Consider using ls -l on a very large directory. It would scroll off the screen. However you can pipe it to more, which you'll recall, displays one screen at a time. The syntax is ls -l | more where the vertical bar (|) is the pipe operator. You can pipe several commands together and combine them with redirection. Here is a slightly more involved example, cat file1 file2 | sort > file3 which does these tasks in order concatenates file1 and file2 sends the result to the sort command, which sorts the lines in alphabetical order stores the alphabetized, concatenated result as a new file called file3 The above set of tasks might be given to beginning programming students who would then write some high level language program to get the result. Here we see how the UNIX philosophy of simple commands used in combination can do complex tasks. If you require more information on the Linux operating system a good introductory guide is available here. Activities/Excercises The exercises assume you have read through the entire chapter. These excercises should be completed on the host or development PC. Logging on Log on your workstation. Try the command ls -l and then ls -al What does the additional a switch do? [see man ls] Try the command sh -version What does this tell you? Standard Directories Using the command ls -F, check out the contents of each of the standard directories from Table 1 of this chapter. For example, try ls -F /dev and so on. What does the F switch do? Standard Directories Look at the contents of the /proc directory. Examine these files in /proc with cat: cpuinfo meminfo interrupts ioports pci If any of the cat displays are more than a screen full, pipe the display through more e.g. (for example) cat /proc/pci | more More about ls Try ls with the f and t options: ls -f or ls -lf ls -t or ls -lt What do these options do? mkdir and tree Using mkdir, make a directory hierarchy in your home directory e.g. (for example) mkdir projects mkdir projects/proj0 mkdir projects/proj1 and so on, adding breadth and depth. Then try the tree command, if it's present on your system, e.g. (for example) tree projects tar Archive and compress your directory hierarchy from the prior problem i.e. (in other words) tar cvfz proj.tgz projects/ Now try to remove the directory e.g. (for example) rmdir projects/ Why didn't that work? Now try rm -rf projects/ Was it removed this time? Now restore it via tar xvfz proj.tgz In the two instances where we used tar, the switches used were c, v, f, z, and x. What does each mean? Use vi Create two different multi line files with vi, file1.txt and file2.txt. We'll use them subsequently. The cp and mv commands Try the cp command: cp file1.txt file3.txt cp file2.txt file4.txt Use ls -l to verify that the new files were made. Then try mv file3.txt file5.txt mv file4.txt file6.txt Now what does ls -l tell you? Using chmod, chown, chgrp See the man pages for these commands. Make some files and try them out. Using pipes Try something like cat file1.txt file2.txt | sort > file12.txt. Describe what this command has done. Now try this ls -l | wc -w What does this do? Does the result make sense? [see man wc]