Fixing Transcodes and nuvexports

Use replex to fix them

It turns out that the raw stream isn’t quite compatible with the DVD standard. There is apparently a small glitch in some of the hardware encoding done by the chip on my PVR-150 encoder. As such, another round of processing needs to be done to detect these glitches and patch them. A command-line tool has been written which does just that: replex.

Here’s the syntax:

$replex -t DVD -k -o Out.mpg In.mpg

The above command would take an input file In.mpg, generated by nuvexport, and create a new, fully DVD compatible file Out.mpg. A slightly fancier command can be done to process all the .mpg files in a particular directory:

$find . -name '*.mpg' -print0 | xargs -0 -i replex -t DVD -k -o '{}'.replexed '{}'

This will take all input files as ending with the .mpg file extension, process them, and generate outputs that have the same base name but with ‘replexed’ appended as the new file extension.

MythTV Keybindings

Clear Key Bindings

Once and a while I really screw up my key bindings on my system. When this happends it’s nice to flush the key bindings from the database. From the command line type:

echo "delete from keybindings ;" | mysql -u mythtv -pmythtv mythconverg

Backup Key Bindings

It never hurts to backup your keybindings once and a while before your about to change them. From the command line type:

mysqldump -u mythtv -pmythtv --databases mythconverg --table keybindings > keybindings.sq

Restore from Backup of key bindings

Backups are not very good unless you can restore from the backup. Use the following command to restore a backup.

mysql -u mythtv -pmythtv mythconverg < keybindings.sql

Troubleshooting multipathd

At work I used linux  native multipathing MPIO provided via device mapper a lot.  I can be a bit hard to troubleshoot.  I have found the best way is to use the interactive console.   From there you can do a lot of things I’ll illustrate the three most common I use:

Show current config

# multipathd -k
>> show config

Reload changes to multipath.conf

#multipathd -k
>>reconfigure

Check paths

#multipathd -k
>>show paths

In addition the following command out puts a log of debug information that can really help.

multipathd -v4 

Intro to Linux: Users and groups

In another blog post I talked about how to control file permissions but I never talked about users and groups.  In linux users groups and passwords are stored in files.  You can directly modify these files but it’s a better idea to use the built in commands.

Users information is stored in /etc/passwd in the following format (fields seperated by :):

bob:x:3002:302:Bob Bobo's account:/home/bob:/bin/bash


  1. Username: It is used when user logs in. It should be between 1 and 32 characters in length.
  2. Password: An x character indicates that encrypted password is stored in /etc/shadow file.
  3. User ID (UID): Each user must be assigned a user ID (UID). UID 0 (zero) is reserved for root and UIDs 1-99 are reserved for other predefined accounts. Further UID 100-999 are reserved by system for administrative and system accounts/groups.
  4. Group ID (GID): The primary group ID (stored in /etc/group file)
  5. User ID Info: The comment field.
  6. Home directory: The absolute path to the directory the user will be in when they log in. If this directory does not exists then users directory becomes /
  7. Command/shell: The absolute path of a command or shell (/bin/bash). Typically, this is a shell. Please note that it does not have to be a shell.

Users actual password is stored in /etc/shadow in the following format:

bob:asdk1324E@#$Fsa:324:0:99999:7
 1. User name : It is your login name
 2. Password: It your encrypted password. 
 3. Last password change (lastchanged): Days since Jan 1, 1970 that password was last changed
 4. Minimum: The minimum number of days required between password changes i.e. the number of days left before the user is allowed to change his/her password
 5. Maximum: The maximum number of days the password is valid (after that user is forced to change his/her password)
 6. Warn : The number of days before password is to expire that user is warned that his/her password must be changed
 7. Inactive : The number of days after password expires that account is disabled
 8. Expire : days since Jan 1, 1970 that account is disabled i.e. an absolute date specifying when the login may no longer be used


The groups are stored in /etc/group (your Primary group will not show up here that’s in /etc/passwd)

power_users:x:3009:bob,tom,bill

 1. group_name: It is the name of group. 
 2. Password: Generally password is not used, hence it is empty/blank. 
 3. Group ID (GID): Each user must be assigned a group ID. 
 4. Group List: It is a list of user names of users who are members of the group. The user names, must be separated by commas.


So now how to we interact with these:

To add a user … useradd :

useradd -u 3002 -d /home/bill -m -c "Bill Thomas" -s /bin/bash -G 3002 bill
(-u uid)(-d home dir -m to create)(-c comment)(-s shell)(-G primary group)

To delete a user:

userdel username

This will not delete their home directory you have to do this manually.

To add a group:

groupadd -g 4021 name
(-g group id) 

To delete a group

groupdel name_or_gid

To add a user to a group as primary group (When you create a file it will be group owned by your primary group)

usermod -G gid username_or_uid

To add a user to a group as a secondary group (user has permissions but does not create files as this group)

usermod -g gid username_or_uid


To change your new users password just type the following

passwd username


	

iptables stop people who make too many connections in a time period

Well it’s a simple deal but iptables can count the number of connections in a time period and block based on it.  For example:

iptables -N SSH_CHECK
iptables -A INPUT -p tcp --dport 22 -m state NEW -j SSH_CHECK
iptables -A SSH_CHECK -m recent --set --name SSH
iptables -A SSH_CHECK -m recent --update --seconds 60 --hitcount 4 --name SSH -j DROP

So no more than 4 connections from the same ip in 60 seconds or you get blocked.




Allow NFS to ESX

In order to use NFS with vmware you have to allow it through the ESX firewall… this can be done via the GUI or in the service console with this command:

esxcfg-firewall -e nfsClient

Linux LVM

In the past I have wrote lots of different little blurbs on LVM but I have been using it a lot and needed to document all the processes.  I will consolidate all my past posts into this one:

Logical Volume Management provides a great deal of flexibility allowing you to dynamically re-allocate blocks of space to different file systems.  Traditional volume manage relies upon strict partition boundaries separating and containerizing each mount point.  Logical Volume Management in linux takes control of the whole drive (or partition) carving it out into equal chunks called physical extents (PE).  Each PE is addressed by it’s logical extent (LE) address.  Groups of LE’s are grouped together to form logical volumes (LV) that are used to mount as file systems.   Then LV’s are grouped into volume groups (VG) for management purposes.

All of the information below is the command line versions of LVM management.  There are lots of quality GUI tools to manage LVM but since I rarely run graphical linux command line is my friend.

Display LVM Information

Display Physical Volume information – pvdisplay

This command will display information on the physical volumes.  Physical volumes fall along partition and traditional linux storage boundries.   This is how you will identify what physical disks are involved in your lvm’s

[root@linuxmonkey2 ~]# pvdisplay
 --- Physical volume ---
 PV Name               /dev/sdb3
 VG Name               storagevg
 PV Size               456.83 GB / not usable 15.15 MB
 Allocatable           yes
 PE Size (KByte)       32768
 Total PE              14618
 Free PE               1
 Allocated PE          14617
 PV UUID               3bwFUg-N06S-yTr9-BkoS-nndD-XcXz-G308fy

As you can see this displays a lot of information on the physical volume and some additional information on PE’s and volume group associated with the physical volume.

Display logical volumes – lvdisplay

This command will display information on the logical volumes.  Logical volumes are mountable partitions made up of one or more PE’s.   Logical volumes have to have a traditional file system placed upon them before they are usable.  This will help you identify what things can be mounted:

lvdisplay
 --- Logical volume ---
 LV Name                /dev/storagevg/storagelv01
 VG Name                storagevg
 LV UUID                GMp2kU-kAMc-ju8o-NJRB-hQ9u-CQ1O-dBi2mX
 LV Write Access        read/write
 LV Status              available
 # open                 1
 LV Size                456.78 GB
 Current LE             14617
 Segments               1
 Allocation             inherit
 Read ahead sectors     auto
 - currently set to     256
 Block device           253:2

LV Name provides a persistent path that can be mounted via fstab you could also mount it via UUID provided by this command.

Display Volume Group – vgdisplay

This command will display information on the volume group which is a grouping of physical volumes on your system.  (not presented by the graphic very well).  By adding multiple drives to a vg you can increase the size of a lv dynamically without getting larger hard drives.

[root@linuxmonkey2 ~]# vgdisplay
 --- Volume group ---
 VG Name               storagevg
 System ID
 Format                lvm2
 Metadata Areas        1
 Metadata Sequence No  2
 VG Access             read/write
 VG Status             resizable
 MAX LV                0
 Cur LV                1
 Open LV               1
 Max PV                0
 Cur PV                1
 Act PV                1
 VG Size               456.81 GB
 PE Size               32.00 MB
 Total PE              14618
 Alloc PE / Size       14617 / 456.78 GB
 Free  PE / Size       1 / 32.00 MB
 VG UUID               0csogH-bjz3-qP4z-JM63-YLpO-YpRy-Kqci71

This displays basic volume information in order to figure out what physical volumes are part of the vg use the pvdisplay command.

Create a Linux LVM on RHEL command line

  1. Partition the disk –  fdisk /dev/sde
  2. Create the physical volume on the disk /dev/sde1 – pvcreate /dev/sde1
  3. Create a volume group on the physical volume using a 4M chunk called cheese- vgcreate -s 4M cheese /dev/sde1
  4. Create the logical volume on cheese with 11GB called mouse- lvcreate -L 11GB -n mouse cheese
  5. Now you can format and mount your LVM – mkfs.ext3 -b 4096 /dev/cheese/mouse
  6. mount /dev/cheese/mouse /mount_point