All posts by Michael Gao

Navigating Disk Storage

Before we program, we need to first get us familiar with how and where computer stores programs, so we know what program is available and more importantly, where to find and show off the program that we are going to write.

We know computer has disk somewhere inside, that’s where all data storage happens. We store different sorts of data in the name of ‘file’, the document we write is a file, the picture we draw is a file, the game we download there might have more than one file, and the Operating system itself has a ton of different files. Of course, the program we are going to write is also a file.

A file is a collection of ‘bytes”, where ‘byte’ is the minimal storage unit. So the more ‘bytes’ available in the disk, the more storage space we have. When we say we have a disk of 100GB, we have a disk that has 100 Giga Bytes. One ‘Giga Byte’ has 1,000,000,000 bytes, or rather 1,073,741,824 bytes in computer term. We’ll explain this once we learn the difference between decimal and binary.

We have a big disk that can store lots of files, in order for us to tell which file is which, we need to give each file a unique name. That seems easy, we can simply name each file by a number, say file1, file2, file9999 etc. Right?

The only problem of that approach is we will have lots of magic files that we don’t know what they are for. Hmmm, what is file3789 for again?

Instead, we, or rather Operating System, manage files like a tree. There’s one and only one ‘root’, then there are ‘branches’ above root, there can be more ‘sub-branches’ above any branch, eventually there is ‘leaf’. We call branch as ‘directory’, thus ‘sub-branch’ is ‘sub-directory’, and the ‘leaf’ is our ‘file’.

We can then give each directory/sub-directory and file a unique name, to help us remember which is which. This way, anywhere in the tree, we can climb to find any ‘leaf’, the ‘file’ we want. Any file will be stored at given directory, which can be a sub-directory of other directory. We call this chain of directories that leads to a given file the ‘path’, since this is the ‘path’ we climb in order to find this file.

For example,

/user/frank/games/pvz2/2015/score

So we first climb to ‘user’ branch, then ‘frank’ sub-branch, then ‘games’ sub-branch and so on, eventually we get to the ‘score’ file, this is probably where Frank keeps his pvz2 game score. :-)

This sounds all familiar, yes, this is what the Windows ‘file explorer’ does. When clicking to expand a ‘directory’, we actually climb onto it. When collapsing it, we climb back to its parent ‘directory’. File explorer exists in Linux as well, but we instead will introduce the navigating methods when we are in a command terminal, since that’s also where we learn to run the program we write etc.

It is actually quite simple to navigate directory tree, by knowing only the following two commands and one special case,

1. Command #1: cd

‘cd’ command is used to Change directory. ‘cd <directory-name>’ will navigate from current directory to new directory specified by <directory-name>.

2. Command #2 : ls

‘ls’ command is used to list contents of current directory.

3. Parent directory: ..

‘..’ is a special name for parent directory. For example, ‘cd ..’  will leave current directory and go to its parent directory.

That is all you need to know to navigate directory tree, now open a terminal (by holding CTRL + ALT, and letter ‘t’), and try them out …

‘ls’ command also takes options. Option is additional typing to further change the ‘ls’ command behaviour. For example, ‘ls -a’, will show all contents of current directory, this will show all hidden files as well, if any is available.

It is important to know that most commands come with ‘user manual’ installed. Any time, ‘man <command>’, will print the manual for given <command>. For example, ‘man ls’ will print the detailed user manual for ‘ls’ command. To learn more on any command, simple ‘man <command-name>’. This is most of the time more convenient than asking our Google friend.

Now, try ‘man man’, yes, you guessed it, this will print the user manual for ‘man’ command itself.

With ‘cd’ and ‘ls’, we now are able to navigate the disk, next we’ll try to actually create a file and make sure we can find it as well.

Operating System Setup

We will learn programming on Linux operating system, so we need to first set up one.

Operating system is the first software our Computer gets right after it leaves the production line. Well, the OS might even be installed before the Computer is fully assembled. So if we don’t like or do not want to use the OS that comes with the computer, we need to replace it, right?

So most Computers come with Windows operating system on it, let’s wipe off that and install Linux instead.

But wait, you don’t want to lose all the data there on Windows, more importantly, you don’t want to lose the games you installed there. Programming is fun, but certainly you’ll want to come to those games sometime. :-)

We’ll do it in a non-destructive way. We’ll keep the Windows, and install Linux within Windows!

This is called virtualization, imagine there will be a another *soft* computer within the Windows, we simply install Linux to this *soft* computer.

We call this Computer *soft*,  because it is a computer made of Software.

Ok, let’s create this *soft* computer first by installing a software called VirtualBox, visit the downloads link, download and install VirtualBox for Windows host.

Once that is installed, you can now create as many *soft* computers as you want, and install any Operating System you like on them, including Windows. So you’ll have computer inside of computer, isn’t that cool?

Open VirtualBox, click ‘New’, then follow the on-screen instructions there to create a Computer (or machine), give it 40GB disk, 1GB memory, and 1 CPU.

Now you have a brand new ‘Computer’, let’s install Linux on it.

We’ll be using Ubuntu Linux, visit desktop downloads to get the installation file. Then install it onto the brand new *soft* Computer we just created. Search “Ubuntu virtualbox install”, you’ll find lots of instructions and tutorials. For example, after initial installation, you may also want to install VirtualBox Extension Pack so that the new *soft* Computer display can take up full screen.

Click the newly installed Ubuntu Linux, you now have a Linux operating system running, within Windows!

Next, let’s get familiar with our new Linux friend, as he’ll be the one that helps us programming …

Hello Operating System

Operating System, is the software that makes Computer the robot. Without OS, Computer would be just a piece of paper weight with different metal or plastic parts here and there. OS also provides support to extend the capability of the robot itself, by allowing programmer to further program on it!

OS does many jobs, such as taking our keyboard and mouse input, display information on screen, receive and send data from/to network etc. OS also manages users of Computer, to determine who can use computer when. Ok, I take the ‘when’ part back, Daddy is doing that part, for now.

Out of the many things OS can do, the one thing that interests us programmer is its capability to run the program we write. More interestingly it can run more than one program at the same time.

To run our program, or any program, it is common that we simply pick the program and click on it. To go a little deeper on this, Program is basically a collection of instructions CPU understands. This program needs to be stored somewhere on Computer. Correct, you guessed it, the program is stored in disk. After clicking, in order for CPU to quickly access its next instruction from this program, the program is loaded to memory from disk,  sometime as a whole, sometime only part of it.  After that, CPU runs the instructions one after the other, and all the magic starts.

Clicking is one way to run our program, OS provides us that capability through the graphical interface, you browse to what you want to run, and click. Easy?

That is the graphical way, there’s another way to tell OS to run our program, we call it ‘command line’, because this way we can give OS commands on a line-by-line basis. OS provides us command line access through a program called terminal. So you can open a terminal, type commands, or find our program and run from there.

Wait, it is so easy to click and go, why bother with those extra steps to use command line?

That can be a personal preference, but what if you are told that through command line you can search and manage your files more quickly and easily, and, you can remotely run your program, for example, start your program on your home computer, from your laptop or ipad while you are climbing the Great Wall? And, through command line interface, you can do a lot more than just clicking around.

Command line is a powerful tool for programmer, let’s keep that in mind.

We’ve talked about what OS does and what interface it provides to us in order to run program. But what is this magic piece of OS software called?

There are different flavours of them.

Three among them are well known for Computer world. Windows, OS X, and Linux. There are other Operating Systems for devices, such as Android, iOS etc, but we’ll focus on Computer side.

Linux? never heard of it.

Now you do, and that’s the OS we will be using to learn programming.

Why? read the following text from Wikipedia,

Examples of popular modern operating systems include Android, BlackBerry 10, BSD, Chrome OS, iOS, Linux, OS X, QNX, Microsoft Windows, Windows Phone, and z/OS. The first eight of these examples share roots in UNIX.

Yes, we want to start our learning with something that has a more popular root.

And, because Daddy says so. :-)

Let’s start by setting up a Computer with Linux OS …

Hello Computer

From Wikipedia,

A computer is a general-purpose device that can be programmed to carry out a set of arithmetic or logical operations automatically. Since a sequence of operations can be readily changed, the computer can solve more than one kind of problem.

That seems quite complicated, if I have to know all that to use computer to do my math, I probably will just count on my fingers, or maybe a calculator, if not an abacus.

Let’s describe computer in a more user understandable way. Computer is a robot, it will do exactly what we ask it to do, assuming we speak the language Computer understands of course. Computer ‘listens’ to us through keyboard/mouse, computer ‘tells’ us what it has done through screen, which is often called monitor. Computer ‘friends’ with other electrical devices through its ports: USB, serial etc.

Computer also likes to be social, they ‘talk’ to each other through network, and often exchanges lots of information with each other without us even knowing.

And of course Computer needs power, which is available either from a wall plug or the battery that comes with it.

That is the outside of it, Computer has magic parts inside as well.

First, Computer has storage to store our files such as pictures, movies, and the program that we are going to write. This storage is usually called disk or drive. The important thing for storage is that we can trust them that our files will not go away even if computer lost power.

Then there is a part called memory, so Computer can temporarily hold stuff there without going to actual storage place. You have a full shelf of story books at home,  you’ll still want to put a few of your favourites in your backpack. It is much easier and faster to check out book from your backpack than taking a trip back to home to get it. The key to memory is access speed, but anything Computer puts in memory gets lost when computer lost power. We’ll talk more about memory when we get to actual programming part.

The heart of Computer is called CPU, the Central Processing Uint, this is the big boss, who manages disk and memory, and takes our programming input and commands the other parts of the Computer to do the job.

CPU itself knows about how to work with the other part of the Computer, but does not know how to get the actual job done. Our brain knows how to drive our legs to walk, but we need to learn to get instructions in order to plan a trip to the Great Wall China from Chicago US. CPU gets instructions from its super boss, you, the programmer.

There are many instructions CPU understand, to read or write disk, to read keyboard input and monitor mouse movement, to display text and graphics on monitor, to play sound etc. It is a big task to learn to understand all those instructions all at once, although eventually we’ll learn to program them directly by sending instructions to CPU to drive different part of Computer, we do not start that way. Instead, we start with a piece of software written by other programmers.

This piece of software knows to send the right instructions to CPU to do basic things. For example, if we press letter ‘T’ on keyboard, this software informs the display to also display ‘T’ at the right place. If we move the mouse, this software knows to move the cursor accordingly on screen display. When we are tired, this software will take our request and politely shut down the Computer itself.

Before we start our programming adventure, we have a few words to say about this important piece of software.

This software is called Computer Operating System, or OS …

Hello Programming

Programming is about asking computer to do what we want it to do. Now follow me, “hey Mr. computer, please say hello” …

Well, nothing happens, no matter how polite I am, the computer just sits there doing nothing, or rather, doing its own thing.

To let computer actually do something, we need to be formal, we need to put everything in writing. This writing is called programming.

The above is not entirely true though, computer also takes voice command, for example with ‘Ok Google’ or ‘SiRi’, you can ‘tell’ computer what to do. Although what computer can do in this case is very limited today, this certainly will change. Maybe some day in the future, the programming can be entirely achieved verbally. For now though, we will focus on programming in writing. And hopefully our wonderful programming technique in writing will greatly shorten the time to that future, both ‘Ok Google’ and ‘SiRi’ are programs written by engineers after all.

Back to writing, there is a few questions we need to answer,

  • what do we use to write?
  • what language shall we write in?
  • how do we let computer read our writing?

Answering all that, we’ll have something in writing that the computer can understand, and hopefully computer will actually do what it reads.

This is something in writing is called Program.

To answer all those questions, let’s first take a step back, and see who will actually read our Program and do the things we write down there.

Computer, of course.

That is correct, but there is more. Our program can be read and understood by many more, for example, phone, tablet are the two where we can download program to. There is even more, music player, video game console, TV,  car, many of the electrical things around us can read and run program.

Computer is a little bit special, it understands the Program we write, it is also the tool we use to write our program with. So we will start with Computer …

Hello World

Writing something meaningful has always been on my wishlist of the things I’ll do once I retire, although that something has never been materialized: the more I think about it, the less I actually do anything with it. Writing seems a daunting task to take on, the amount of preparation work required itself is big enough to keep me away from even starting it, with the fact being I don’t even know what to prepare. :-)

So this thinking remains a wish, till a few days ago Frank said he wanted to learn how to program. Maybe I can share what I’ve learned over the past years of programming at work?

For all those who write software, ‘Hello World’ is the official starting point, so here it goes …