vCloud Director ORG VM’s cannot contact the internet

I have been playing around a lot of late with vCloud director.  I tried it out in the 1.0 days and found it to be lacking.  I used it again in the 1.5 days and also found it to be a pain.  It looks like VMware has got it figured out with 5.1.  Don’t get me wrong it’s still a lot of components pasted together but when setup it works great.   I did run into a few issues with changes. made between versions.  In 1.5 your would deploy vApp’s to a routed ORG network and it just worked they could get to the internet.   This has changed in 5.1.  See this article for full information.  Here are the steps to get it working again.   This example assumes that you want all your ORG vApp’s to have the same ip address.

  1. Create the ORG Edge gateway (this should be done when you create the org if not do it now)
  2. Sub-allocated some external IP’s to the Edge gateway
  3. Create a SNAT rule on the edge gateways external interface (this is the part that got me… not the ORG network but the edge external) This should be a internal range like 192.168.10.0/24 to a single external IP like 8.8.8.4 (if I was a google DNS server)
  4. Add a firewall rule that allows all outgoing traffic from your internal (use the term internal) to external (term external) protocol any and action is allow.

That’s it and it works.  I will be writing a lot more on vCloud in the near future so keep tuned.

Boot process for Linux

This will document the x86 boot process from a linux perspective. This document will attempt to provide a technical overview if you are not comfortable with hexidecimal, octal or binary you might want to brush up on them first.

 

Order of boot

  1. The BIOS completes it’s check (memory, cpu, video)
  2. The BIOS execututes the master boot code in the MBR
  3. The master boot code then has two functions identify any active partitions and any extended partitions.
  4. If the master boot code identifies a extended partition it follows the link to the extended partition and so on until it finds no additional partitions.
  5. The master boot loader moves to the active partition and turns over booting to that partition.
  6. The boot loader enters stage 1
  7. The boot loader enters stage 1.5 and displays the menu
  8. The boot loader enters stage 2 and waits for user input or default selection timout
  9. The Kernel initilizes the hardware.
  10. The boot loader loads drivers and modules out of the initrd in /boot/initrd
  11. The boot loader turns over booting to the kernel
  12. /sbin/init executes the rest of the system.
  13. int starts the run level scripts

The boot process starts with a 512 byte piece of code called the master boot record. The MBR is stored on the first 512 bytes of a drive. The BIOS accesses this section and it contains code that points to the rest of the boot process. The master boot record contains the partition table, bootloader and a section called the magic number. The bootloader takes the first 446 bytes. The partition table takes the next 64 bytes. The magic number takes the last 2 bytes.

 

The magic number is used as a crc check for your mbr it should always contain 0xAA55. You can dump the mbr on your system using:
dd if=/dev/hda of=/mbr.dump bs=512 count=1
This will dump the first 512 bytes of your hda drive to the file /mbr.dump. You can also rewrite this mbr to the file system using:
dd if=/mbr.dump of=/dev/hda bs=512 count=1
You can use strings to view the current boot loader:strings /mbr.dump


linuxmoney:~ # strings /mbr.dump
ZRrK
D|f1
GRUB
Geom
Hard Disk
Read
 Error

You can view the partitions on the disk by using:file /mbr.dumpThis will produce a output listing partitions and start and stop sectors:x86 boot sector;
partition 1: ID=0x83, starthead 1, startsector 63, 417627 sectors;
partition 2: ID=0x82, starthead 0, startsector 417690, 2104515 sectors; partition 3: ID=0x83, starthead 0, startsector 2522205, 4209030 sectors;
partition 4: ID=0xf, active, starthead 0, startsector 6731235, 149565150 sectors, code offset 0x48
You can see that partition 4 is active the ID displays a the type of partition. You can find a list of partition ID codes Here. Since each sector has 512 bytes we can find the size of each partition: For example partition 1 is 417627 sectors. You can find the size using:echo $(((417627/2)/1024))You can compare this information to a df -k outputFilesystem 1K-blocks  Used      Available Use%  Mounted on
/dev/hda5 20641788    5464224   14128924  28%   /
/dev/hda6 52964408    4147160   46126764   9%   /home
/dev/hdc1 244076732   100537572 143539160 42%   /data

You can dump hex of the mbr using: od -Ad -tx1 /mbr.dump

You can also dump it using hexdump

Key
Color Description
RED Boot Loader
GREEN 1st Partition table
YELLOW 2nd Partition table
BROWN 3rd Partition table
PINK 4th Partition table
BLUE Magic Number

You can also do a hex dump using xxd /mbr.dump

 


linuxmoney:~ # xxd mbr.dump
0000000: eb48 90d0 66bc 007c 0000 8ec0 8ed8 89e6  .H..f..|........
0000010: 66bf 0006 0000 66b9 0001 0000 f3a5 ea23  f.....f........#
0000020: 0600 0080 fa80 7c05 80fa 877e 02b2 8088  ......|....~....
0000030: 1649 0766 bfbe 0700 0031 f666 b904 0302  .I.f.....1.f....
0000040: ff00 0020 0100 0000 0002 fa90 90f6 c280  ... ............
0000050: 7502 b280 ea59 7c00 0031 c08e d88e d0bc  u....Y|..1......
0000060: 0020 fba0 407c 3cff 7402 88c2 52be 817d  . ..@|<.t...R..}
0000070: e836 01f6 c280 7456 b441 bbaa 55cd 135a  .6....tV.A..U..Z
0000080: 5272 4b81 fb55 aa75 45a0 417c 84c0 783e  RrK..U.uE.A|..x>
0000090: 7505 83e1 0174 3766 8b4c 10be 057c c644  u....t7f.L...|.D
00000a0: ff01 668b 1e44 7cc7 0410 00c7 4402 0100  ..f..D|.....D...
00000b0: 6689 5c08 c744 0600 7066 31c0 8944 0466  f.\..D..pf1..D.f
00000c0: 8944 0cb4 42cd 1372 05bb 0070 eb7d b408  .D..B..r...p.}..
00000d0: cd13 730a f6c2 800f 84e8 00e9 8d00 be05  ..s.............
00000e0: 7cc6 44ff 0066 31c0 88f0 4066 8944 0431  |
 .D..f1...@f.D.1
00000f0: d288 cac1 e202 88e8 88f4 4089 4408 31c0  
 ..........@.D.1.
0000100: 88d0 c0e8 0266 8904 66a1 447c 6631 d266  .....f..f.D|f1.f
0000110: f734 8854 0a66 31d2 66f7 7404 8854 0b89  .4.T.f1.f.t..T..
0000120: 440c 3b44 087d 3c8a 540d c0e2 068a 4c0a  D.;D.}<.T.....L.
0000130: fec1 08d1 8a6c 0c5a 8a74 0bbb 0070 8ec3  .....l.Z.t...p..
0000140: 31db b801 02cd 1372 2a8c c38e 0648 7c60  1......r*....H|`
0000150: 1eb9 0001 8edb 31f6 31ff fcf3 a51f 61ff  ......1.1.....a.
0000160: 2642 7cbe 877d e840 00eb 0ebe 8c7d e838  &B|..}
 .@.....}.8
0000170: 00eb 06be 967d e830 00be 9b7d e82a 00eb  .....}.0...}.*..
0000180: fe47 5255 4220 0047 656f 6d00 4861 7264  .GRUB .Geom.Hard
0000190: 2044 6973 6b00 5265 6164 0020 4572 726f   Disk.Read. Erro
00001a0: 7200 bb01 00b4 0ecd 10ac 3c00 75f4 c300  r.........<.u...
00001b0: 0000 0000 0000 0000 5147 0a00 0000 0001  ........QG......
00001c0: 0100 83fe 3f19 3f00 0000 5b5f 0600 0000  ....?.?...[_....
00001d0: 011a 82fe 3f9c 9a5f 0600 c31c 2000 0000  ....?.._.... ...
00001e0: 019d 83fe 7fa2 5d7c 2600 8639 4000 8000  ......]|&
 ..9@...
00001f0: 41a3 0ffe ffff e3b5 6600 de2e ea08 55aa  A.......f.....U.

linuxmoney:~ # hexdump mbr.dump
0000000 48eb d090 bc66 7c00 0000 c08e d88e e689
0000010 bf66 0600 0000 b966 0100 0000 a5f3 23ea
0000020 0006 8000 80fa 057c fa80 7e87 b202 8880
0000030 4916 6607 bebf 0007 3100 66f6 04b9 0203
0000040 00ff 2000 0001 0000 0200 90fa f690 80c2
0000050 0275 80b2 59ea 007c 3100 8ec0 8ed8 bcd0
0000060 2000 a0fb 7c40 ff3c 0274 c288 be52 7d81
0000070 36e8 f601 80c2 5674 41b4 aabb cd55 5a13
0000080 7252 814b 55fb 75aa a045 7c41 c084 3e78
0000090 0575 e183 7401 6637 4c8b be10 7c05 44c6
00000a0 01ff 8b66 441e c77c 1004 c700 0244 0001
00000b0 8966 085c 44c7 0006 6670 c031 4489 6604
00000c0 4489 b40c cd42 7213 bb05 7000 7deb 08b4
00000d0 13cd 0a73 c2f6 0f80 e884 e900 008d 05be
00000e0 c67c ff44 6600 c031 f088 6640 4489 3104
00000f0 88d2 c1ca 02e2 e888 f488 8940 0844 c031
0000100 d088 e8c0 6602 0489 a166 7c44 3166 66d2
0000110 34f7 5488 660a d231 f766 0474 5488 890b
0000120 0c44 443b 7d08 8a3c 0d54 e2c0 8a06 0a4c
0000130 c1fe d108 6c8a 5a0c 748a bb0b 7000 c38e
0000140 db31 01b8 cd02 7213 8c2a 8ec3 4806 607c
0000150 b91e 0100 db8e f631 ff31 f3fc 1fa5 ff61
0000160 4226 be7c 7d87 40e8 eb00 be0e 7d8c 38e8
0000170 eb00 be06 7d96 30e8 be00 7d9b 2ae8 eb00
0000180 47fe 5552 2042 4700 6f65 006d 6148 6472
0000190 4420 7369 006b 6552 6461 2000 7245 6f72
00001a0 0072 01bb b400 cd0e ac10 003c f475 00c3
00001b0 0000 0000 0000 0000 4751 000a 0000 0100
00001c0 0001 fe83 193f 003f 0000 5f5b 0006 0000
00001d0 1a01 fe82 9c3f 5f9a 0006 1cc3 0020 0000
00001e0 9d01 fe83 a27f 7c5d 0026 3986 0040 0080
00001f0 a341 fe0f ffff b5e3 0066 2ede 08ea aa55
0000200

You can manually decode the partition table using the following
information. Remember to flip the bytes to get the correct order e.g
0080 becomes 80 00.

Offset Size Description
0x00 1 byte Active flag 0x80 active otherwise 0x00
0x01 3 bytes Cylinder-head-sector address of the first sector in the partition
0x04 1 byte Partition type
0x05 3 bytes Cylinder-head-sector address of the last sector in the partition
0x08 4 bytes Logical block address of the first sector in the partition
0x0C 4 bytes Length of Parition in sectors

+--- Active partition flag 80H for active partition
|
|      +--- Cylinder-head-sector address of the first sector in the partition
|      |
|      |    +--- Partition Type List here.
|      |    |
|      |    |     +--- Cylinder-head-sector address of the last sector.
|      |    |     |
|      |    |     |        +--- Logical block address of the first sector.
|      |    |     |        |
|      |    |     |        |	    +--- Size of Parition in sectors.
|      |    |     |        |        |
-- -------- -- -------- -------- --------
DL DH CL CH TB DH CL CH LBA      SIZE
00 01 01 00 83 fe 3f 19 3f000000 5b5f0600	1st Partition
00 00 01 1a 82 fe 3f 9c 9a5f0600 c31c2000	2nd Partition
00 00 01 9d 83 fe 7f a2 5d7c2600 86394000	3rd Partition
80 00 41 a3 0f fe ff ff e3b56600 de2eea08	4th Partition

Decoding CHS

The CHS is used to decode the location of the first of the
partition if that location exists within the first 1024 cylinders of
the hard drive. When the location goes beyond that location the CHS
value is normaly set to the max values of 1024,254,63 or FE FF FF.
Decoding the values can be a challenge without switching to the binary
value. They are stored in the order of head, sector, and cylinder, the
cylinder value requires more than 8 bits (1 byte) the sector value uses
less than 8 bits, so you have to convert the values to binary to decode them:


If the ending value for cylinder is 1023 or above then you have to figure out the ending location by adding the size to the starting location.

Remember that we can only have four partition tables per disk. This
is why extended paritions were created an extended partition uses a
link table to create unlimited partitions. The partition entries on the
table are top down. The first partition on the physical disk is the
last entry in the MBR partition table.



Extended Partitions 
      Extended Partitions are a way of getting around the four
partition limit on file systems. Extended partitions cannot be marked
as active or used as a boot device. The extended partition section in
the MBR can describe up to at least 23 (Old DOS) additional partitions
under linux the amount of partitions possible is much higher. Extended
partitions have the partition type of 05h or 0Fh depending on size of
the disk. Extended partition boot records duplicate the MBR. Normally
the first 446 bytes of the extended section is empty (LILO and GRUB
both use it for internal code) The partition table is then full of
partition entries followed by the aa55 code. In extended partitions the
total size (LBA) is the size of all extended partitions.

GRUB
Grub stands for GRand Unified Bootloader. It is the most common boot loader for linux today. The boot process with GRUB is as follows:

  1. Starts executing bootloader code (GRUB stage 1) (boot.img).
  2. Bootloader jumps to the sector number of next stage. The stage 1.5 located in the “DOS compat space” immediately after the MBR.
  3. Stage 1.5 loads the file system and make full drive size available for loading. (diskboot.img+kernel.img+pc.mod+ext2.mod)
  4. Stage 2 takes over and loads the boot menu. (normal.mod+_chain.mod)
  5. After your selection the operating system is loaded.
Grub files are located in /boot/grub here you can find the stage1 stage2 and the menu.1st or grub.conf files. The configuration is done in the menu.1st or grub.conf file.
linuxmoney:/ # ls -al /boot/grub/
total 228
drwxr-xr-x 2 root root   4096 Sep 27 18:12 .
drwxr-xr-x 3 root root   4096 Jun 30 17:43 ..
-rw------- 1 root root     30 Jun 30 17:43 device.map
-rw------- 1 root root     30 Jun 30 17:37 device.map.old
-rw-r--r-- 1 root root   7552 Nov 25  2006 e2fs_stage1_5
-rw-r--r-- 1 root root   7424 Nov 25  2006 fat_stage1_5
-rw-r--r-- 1 root root   6688 Nov 25  2006 ffs_stage1_5
-rw-r--r-- 1 root root   6688 Nov 25  2006 iso9660_stage1_5
-rw-r--r-- 1 root root   8160 Nov 25  2006 jfs_stage1_5
-rw------- 1 root root   1385 Jun 30 17:43 menu.lst
-rw------- 1 root root   1188 Jun 30 17:36 menu.lst.joe
-rw------- 1 root root   1385 Jun 30 17:37 menu.lst.old
-rw-r--r-- 1 root root   6848 Nov 25  2006 minix_stage1_5
-rw-r--r-- 1 root root   9216 Nov 25  2006 reiserfs_stage1_5
-rw-r--r-- 1 root root    512 Nov 25  2006 stage1
-rw-r--r-- 1 root root 104042 May 19 11:13 stage2
-rw-r--r-- 1 root root   7040 Nov 25  2006 ufs2_stage1_5
-rw-r--r-- 1 root root   6240 Nov 25  2006 vstafs_stage1_5
-rw-r--r-- 1 root root   8904 Nov 25  2006 xfs_stage1_5

To reinstall grub in your mbr type:grub-install /dev/hda      Configuration for grub is done inside grub.lst (normally in /boot/grub/grub.lst) this file has the following settings:# Comments inside grub.lst ae done with a hash mark (#)

 

# default defines the default choice to boot without user interaction
default 0
# Time out sets how long the boot menu will display before it loads default
timeout 30
# fallback provides a another choice in case default fails.
fallback 1
# hiddenmenu allows you to choose not to display the boot menu instead boot the default
# hiddenmenu
# OS definitions begin with a title title is what is displayed on the screen to the user
title openSUSE 10.2 – 2.6.18.8-0.3
# After the title description everything that follows is part of the same boot loader until the title tag appears again.
# Common entries in linux are root, kernel, and initrd
# root defines the root partition and tries to get the size of the partition hd0 partition 4
root (hd0,4)
# kernel attempts to load the kernel image off the root device
kernel /boot/vmlinuz-2.6.18.8-0.3-bigsmp root=/dev/hda5 vga=0x31a resume=/dev/hda2 splash=silent showopts
# initrd Load an initial ramdisk (allows you to modify the kernel without a recompile
initrd /boot/initrd-2.6.18.8-0.3-bigsmp
Command Line Options
While at the boot menu you can also pass grub command line variables like what runlevel to boot into or additional options. To choose the run level to boot the kernel into:

  • On the graphical menu highlight the kernel you wish to boot
  • Press the e button to edit the kernel selection
  • At the prompt type the number of the run level you wish to boot into (1 to 5) single or emergecy
  • Once returned to the grub menu press b too boot the kernel and runlevl selection
      You can read more about grub options at The GNU grub menu.
LILO 
      LILO (LInux LOader) is a generic boot loader for Linux. Lilo is an older boot loader it follows the same process as GRUB. Unfortunatly, it does not contain a command line interface like grud making MBR changes required each time you want to change boot parameters. Also changes to LILO can cause the system to fail to boot. It is for this reason alone that GRUB has become the standard boot loader of linux. Lilo keeps some files in /boot but it's configuration is done in /etc/lilo.conf. To reinstall lilo as the boot loader:/sbin/lilo
Command Line Options 
      While at the boot menu you can choose what runlevel you want to boot by pressing:
  • Ctrl-X to get boot:
  • Type linux runlevel

Kernel

Once the boot loader has reached second stage it reads it’s configuration ahd displays a menu of available kernels to boot. Once the user or boot loader determines what kernel to load stage two boots the kernel file off the /boot partition. Once the kernel is loaded the first step is to initialize the hardware. Then the kernel loading is reading the initrd image this file contains drivers required by the kernel to load scsi devices and ext3 file systems. Once initrd image is completely loaded the boot loader turns the booting process over to the kernel file. The kernel creates a read-only root device and mounts it. At this point the kernel is loaded but since no user space files are loaded you cannot interact with it. This is where /sbin/init takes over.

/sbin/init

init is what process the rest of the boot and provides the user environment. init becomes the parent or grandparent process for all processes on a system it has a pid of 1 always. It first runs the /etc/rc.d/rc.sysinit script that starts swap, system clock, check file systems and many other processes. It the runs /etc/inittab which sets up the run levels.

 

Runlevels

A runlevel is a collection of scripts used to start applications and services used by a system.  Linux supports multiple runlevels.  You can change between runlevels very quickly on a Linux system dismounting file systems as you go.  The configuration for the runlevels is done inside the /etc/inittab file.  You can find the default runlevel inside inittab:

id:5:initdefault:

The default run level on this system is 5 which is multiuser with graphical X windows interface.  inittab also possibly defines:

  • First script to be executed before runlevels /etc/init.d/boot
  • Defines the RC scripts to be executed with each run level
  • It also defines special keyboard commands
  • The getty-programs for each run level

    The /etc/init.d/boot defines the following settings:

  • Sets the terminal size and dimentions for the terminal
  • Starts the initial boot messages and coloring
  • Sets up /proc /sys /dev /sys/kernel/debug
  • Starts user defined scripts boot.local

The default runlevels for Linux are:

Runlevel  State 
 0  Shutdown
 1  Single User Mode
 2  Multiuser without network
 3  Multiuser text based
 4  Unused
 5  Multiuser with Graphical X
6  Reboot

You can quickly change the runlevels using:

init runlevel

Each runlevel executes the scripts contained inside /etc/init.d/rc_runlevel.  The scripts inside here are normaly symbolic links to scripts inside /etc/init.d/ these scripts should take at least two variables stop and start.  The links inside /etc/init.d/rc_runlevel are of two types kill (K) scripts and start (S) scripts.  The type is followed by a two digit number used to denote the order inside this runlevel for the script to be executed.  For example:

# ls -al
total 8
drwxr-xr-x  2 root root 4096 Sep  2 21:52 .
drwxr-xr-x 11 root root 4096 Nov 16 20:25 ..
lrwxrwxrwx  1 root root    9 Sep  2 21:52 K02single -> ../single
lrwxrwxrwx  1 root root   12 Sep  2 21:52 K13microcode -> ../microcode
lrwxrwxrwx  1 root root    9 Sep  2 21:52 K13splash -> ../splash
lrwxrwxrwx  1 root root    8 Sep  2 21:52 K21fbset -> ../fbset
lrwxrwxrwx  1 root root   15 Sep  2 21:52 K21irq_balancer -> ../irq_balancer
lrwxrwxrwx  1 root root    8 May 19 10:46 S01fbset -> ../fbset
lrwxrwxrwx  1 root root   15 May 19 10:45 S01irq_balancer -> ../irq_balancer
lrwxrwxrwx  1 root root    6 May 19 10:47 S09kbd -> ../kbd
lrwxrwxrwx  1 root root   12 May 19 10:51 S09microcode -> ../microcode
lrwxrwxrwx  1 root root    9 May 19 10:47 S09splash -> ../splash
lrwxrwxrwx  1 root root    9 May 19 10:47 S20single -> ../single
You can see that I have many files that start as part of runlevel 1 for example S09splash starts before S20single.  It is very easy to automatically add an item to a run level using chkconfig in linux.  For example if I wanted to see if a script in /etc/init.d is started at runtime use the following command:

# chkconfig -l apache2
apache2  0:off  1:off  2:off  3:on   4:off  5:on   6:off

chkconfig can also be used to turn on specific run levels using

#chkconfig service_name runlevel/runlevels 

For example:

#chkconfig apache2 235

Will start the apache2 script in /etc/init.d on runlevel 2, 3 and 5.  You can manually add the links using ln.  Also running chkconfig alone will display all scripts and they status at the current run level or chkconfig -l will display all runlevels.

 

 

 

How to add space to a ext3 lvm online

Logical Volumes are great you can add disk space without unmounting disk.  With ext3 you can expand that space without a downtime.  It’s easy.

 

Assume you add a new disk called /dev/sdg and you partition it into a single partition to keep the blocked lined up with your storage system. Now you have /dev/sdg1

Make it into pv’s

 

pvcreate /dev/sdg1

 

Now add it to your current logical volume called storage with the following command

 

vgextend storage /dev/sdg1

 

Use vgdisplay to show how many pv’s are used and now many are free… using the next command we need to add them together..

 

lvextend -l number_of_pvs /dev/storage/storage

 

Now you need to expand the ext3 which can take some time:

 

resize2fs /dev/storage/storage

 

And wait …

 

Mysql Backup’s

Backing up your mysql instance can be a pain.  The following two commands work great…

Your backup user only needs select and Lock table rights.

mysqldump -u username -p’password‘ -f –routines –add-drop-database table_name | bzip2 -c > /path/to/dump/location/filename.sql.bz2

And to get the stored procs

mysqldump -u username -p’password‘ -f -R –triggers –no-create-info –no-data –no-create-db –skip-opt database_name | bzip2 -c > /path/to/dump/location/filename.sql.bz2

Network Protocol: ARP

ARP – Network Layer

Address Resolution Protocol (ARP) is used in networking to resolves IP addresses to MAC address. It allows machines on a network to talk node to node using mac addresses resolved by IP addresses.

ARP Steps

  • Your system checks for a router or next hop in communication
  • If there is a router with an IP address it sends a ARP request to get the routers MAC address
  • The server responds with a packet with it’s IP address.
  • As your packet travels it changes it’s destination MAC address changes based on ARP’s

Network Protocol: Mac Address

Mac Addresses – Data Layer

Each physical network interface card (NIC) has a unique identifer assigned to that NIC.   This unique identifier is called a Mac Address.  A mac address contains a vendor ID and a serial number and is made up of 12 hexidecimal characters.  Mac addresses are part of the Data layer of the OSI model and used heavily in Ethernet node to node transmissions.  Each nic responds to two addresses; it’s own unique address and the broadcast address of ff:ff:ff:ff:ff:ff.   In linux you can display your MAC address by using the command ifconfig -a.  It will display something similar to this:

ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:0E:A6:7A:19:E1
inet addr:192.168.10.10  Bcast:192.168.10.255  Mask:255.255.255.0
inet6 addr: fe80::20e:a6ff:fe7a:19e1/64 Scope:Link
UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
RX packets:42708883 errors:0 dropped:0 overruns:0 frame:0
TX packets:167206053 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3679921735 (3509.4 Mb)  TX bytes:879524352 (838.7 Mb)
Interrupt:201 Base address:0xe000

The HWaddr is the current mac address.  Most modern NIC’s support MAC address spoofing this can be done in almost any operating system.  Since mac addresses can be spoofed quickly it is not a good method for secure authentication of machines.  A lot of wireless routers use mac addresses as a method of access control this alone should not be the method of access control.    In Linux ifconfig allows you to change your mac address.  For example:

ifconfig hw ether 00:01:02:72:45:C2

Would allow me to change my mac address to 00:01:02:72:45:C2.

Network Protocol: DNS

DNS – Application Layer

DNS stands for Domain name system, it is a database that matches names with physical addresses.  In the case of the Internet when you type www.jgriffiths.org your machine queries a DNS server who returns 74.208.29.184.  You computer can then contact my webserver and request my website.  IP routing requires a number address, developers found the numbers hard to rembemer and identify.  So the domain name system was created.  Names like google.com, jgriffiths.org or even yast-tastic.com are all easier to remember than number sequences.  Originally users of the internet would download a file containing all DNS tables.  As the internet grew it became impossible to download all the names and keep a up to date copy of the files.  The sheer traffic for sending these tables alone would grind the internet to a halt.  Instead it was realized that the average user would only need a few DNS entries and could dynamically get those addresses when needed.

Top Level Domains:

DNS is divided into top level domains (TLD’s) the orginal TLD’s were:

  • .com
  • .org
  • .net
  • .edu
  • .mil
  • .gov
  • and country codes .us .ca .uk etc..

Each type of TLD is stored in a different database and groups inside government and private companies control the TLD names.  A single entity controls the whole TLD.  This entity also is always the authoritative answer for all TLD’s of it’s type.

A lot more TLD’s have been added since then.   A domain name consists of one TLD and one second level domain name.  The leftmost domain entry is normally the hostname.  The hostname could also be called a subdomain and these names can go 127 levels deep.  When looking up a domain name first the TLD is examined then the second level domain, this normally will connect you with a server that has access to more information about any subdomains.

Where are domain names stored?

Each Domain name is stored inside a “zone file”.  Each domain has it’s own “zone file” on it’s own domain server.  People on the internet can host their own “zone files” or pay someone else to host them.  Across the world there are about 13 root level DNS servers.  Between these 13 root servers all DNS records are stored.   Root servers do not always contain entries for every host they contain links to the TLD DNS server who then have the second level domain information, this could lead to a system or a additional dns server.  One advantage of having 13 root servers all over the world is they become fault tolerant based up geographical disaster.

How does your local PC Resolve DNS questions?

  1. First your PC consults it’s own hosts file (this is a relic to the days when everyone would download the full DNS list) it also allows you to manually force a DNS entry.  (Stored in /etc/hosts on linux or c:\windows\system32\drivers\etc\hosts)
  2. If it cannot resolve the hostname via localhost then it contacts your local dns server (ipconfig /all in windows or /etc/resolv.conf in Windows)
  3. If your local DNS server does not have the answer then it requests the information of it’s ISP’s DNS server.
  4. If the ISP does not have the answer it may consult a higher ISP or move to a TLD domain server.
  5. The TLD server should have the answer if not it know what server is authoritative for the information, then it passes it back down the line.  Each server in the line recieves the answer and caches the results for a set time.  (Each Authoritative server can set it’s own caching time known as time to live.)

How to perform non-automated DNS lookups

nslookup – performs a basic DNS lookup and provides the ip address to domain name mapping.  This command works on both Linux and windows

#nslookup jgriffiths.org
Server:         192.168.10.1
Address:        192.168.10.1#53

Non-authoritative answer:
Name:   jgriffiths.org
Address: 74.208.29.184

dig – dig performs a more comphensive search of DNS and returns the full record from DNS:

# dig google.com

; <<>> DiG 9.3.2 <<>> google.com
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44506
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 4, ADDITIONAL: 4

;; QUESTION SECTION:
;google.com.                    IN      A

;; ANSWER SECTION:
google.com.             108     IN      A       64.233.187.99
google.com.             108     IN      A       64.233.167.99
google.com.             108     IN      A       72.14.207.99

;; AUTHORITY SECTION:
google.com.             31118   IN      NS      ns1.google.com.
google.com.             31118   IN      NS      ns3.google.com.
google.com.             31118   IN      NS      ns2.google.com.
google.com.             31118   IN      NS      ns4.google.com.

;; ADDITIONAL SECTION:
ns1.google.com.         38856   IN      A       216.239.32.10
ns2.google.com.         30976   IN      A       216.239.34.10
ns3.google.com.         33194   IN      A       216.239.36.10
ns4.google.com.         33194   IN      A       216.239.38.10

;; Query time: 31 msec
;; SERVER: 192.168.10.1#53(192.168.10.1)
;; WHEN: Wed Nov 21 22:46:34 2007
;; MSG SIZE  rcvd: 212

As you can see dig returns alternate IP address for that domain name and DNS servers for google.com.

DNS Caching and Time to Live

Within your DNS you have the ability to set a Time to Live value: this represents the maxium time you would like a server to cache or store the information you provided about DNS lookup’s.   There is not guarentee that the DNS server will honor the TTL’s.  It seems most world wide DNS servers take 2 to 3 days before they will renew their caches.

Zone Files

The information gathered by DNS is normally stored inside zone files on unix or linux.  These files are read by BIND (Berkeley Internet Name Domain) and then BIND services DNS information to the world.  The BIND tutorial will cover bind topics in more detail.

Network Protocols

Protocols are agreed upon standards.  Without protocols computers would be unable to talk to each other.  In networking terms protocols are a lot like languages.  Imagine if this website was in german and you only spoke english.  Without access to a dictionary there would be no chance to understand this website.  When computers talk to each other they are required to speak the same language or have a translator (dictionary).   When working with networking protocols a model is used to define function and role this is known as the OSI (open systems interconnect) model.  The model was first defined in 1977, since a lot has changed since 1977 it can be hard to fit newer protocols into the OSI model.  There are two different versions of the OSI model a 7 layer and a 5 layer.  Since the 7 layer incorperates the 5 layer this article will explain the 7 layer model.  From the top down the layers are: Application, Presentation, Session, Transport, Network, Data Link, and Physical.  Each layer provides a method for communication between it’s adjacent layers.

Layer 7: Application

The application layer directly interfaces the application.  It provides the data in a method that the application expects.  It also sends requests for information to the presentation layer. Examples of Layer 7 are:

  • A web browser
  • A mail client
  • A FTP client

Layer 6: Presentation layer

The presentation layer takes the request or information from either side of it’s adjacent layers and translates them into usable form.  It will break information from the application layer into encapsulated sessions for the session layer.  It will also re-assemble the session layer data into application usage.  Common examples are:

  • SSL
  • TLS

Layer 5: Session Layer

The session layer controls dialogue and connections (sessions) between computers  It handles communication between the local and remote applications.  t provides for full-duplex, half-duplex, or simplex operation, and establishes checkpointing, adjournment, termination, and restart procedures. Common examples are:

  • NetBIOS

Layer 4: Transport Layer

The transport layer provides transparent controls for link through flow control, segmentation/desegmentation, and error control.   The transport layer also may be responsible for resending lost packets.  Common Transport layer protocols are:

  • TCP
  • UDP
  • IPSec
  • IPX

Layer 3: Network Layer

The Network layer ensures quality of service and addressing for end to end communication.  This is the layer at which most routers operate.  Common examples are:

  • IP
  • RIP
  • ARP
  • ICMP

Layer 2: Data Layer

The data layer provides a method to deal with errors that happen in the phsyical layer it also provides node to node communication unlike the network layer that provides end to end communication.  Data can also be broken up to accomidate the needs of the phsyical layer. Common examples are:

  • Ethernet (802.3)
  • Wireless (802.11 a/b/g)
  • Frame Relay
  • Token Ring

Layer 1: Phsyical Layer

The phsyical layer defines the required electrical and phsyical needs for communication this may include wave modulation, fiber optic cables, cat 5 cable and phone lines.

Intro to Linux: PHP information

It’s really common to want to review your php setup.  The easiest way is the php function phpinfo();  the following will run from the command line or from a web browser:

<?php
phpinfo();
?>

Or to just see the modules

<?php
phpinfo(INFO_MODULES);
?>

To run from the command line type

php textfile_name

Since outputting this information can be a security risk avoid running from your webserver unless the directory is secured against random visitors.

Intro to Linux: tar

Tar stands for tape archive it’s a old unix way of collecting lots of files into a single file for transport or storage on a tape drive.  Later authors have combined gzip with tar for compress on the fly.

So to open a tar.gz use the following command:

tar xvfz filename.tar.gz

To open a tar do this

tar xvf filename.tar

To create a tar file

tar cvf filename.tar files_or_location_to_add.tar

To create a tar.gz

tar czvf filename.tar.gz files_or_location_to_add.tar.gz

More information on tar can be found by

man tar