sigmoid

I open a command prompt so often that it has become a burden to have to use a stock cmd shortcut and cd to the right directory. It is easy to add an option to the context menu that comes up when you right click a directory on Windows Explorer. To do this, open the Registry Editor (regedit.exe) add a registry key under:

HKEY_CLASSES_ROOT\Directory\shell

Name it anything, I named mine CommandPrompt and give it a data value, for example VS2010 Command Prompt, this is the text that will appear in the context menu. Create a new key under this newly create CommandPrompt key and mane it command. The value of this key should use vcvarsall.bat of your Visual Studio installation in order for the environment to be set properly when you open the cmd window. The following value for that key does it for my installation:

cmd /k ""D:\dev\msvc2010\VC\vcvarsall.bat"" x86 && cd /d %1

replace D:\dev\msvc2010 with the installation path of your visual studio and you’re done. To find out the right path to the vcvarsall.bat file you can look at the shortcut created by your VS installer. You can also replace x86 (for instance with ia64, amd64, x86_amd64, x86_ia64) to request that another environment is setup if you are cross-compiling for instance.

If you have a vast number of files and you want to do something with them, then use something like the following:

grep -l "mystring" * | xargs -I{} echo {}

this will echo each filename that contains mystring in the current working directory. You could replace echo linux command with any other command, for instance with rm -f to remove those files (be careful!). The -I{} switch to xargs prevents having problems with filenames that have spaces. If you want to search in a different directory than the working directory replace * with it.

So lets say you want to find files that contain DNS in /etc. You can do this with:

grep -r -l "DNS" /etc | xargs -I{} echo {}

If you want to find files that contain any form of some string in /home/user. You can do this with:

grep -i -r -l "some string" /home/user | xargs -I{} echo {}

where -i (or –ignore-case) does a case-insensitive search (i.e. it will match any some string, or Some String, etc.) and -r stands for recursive.

If you have a complex scenario where you only want to process specific type of files or file patterns, you may want to use find instead of grep directly.

You don’t need Acrobat for this. GhostScript does an excellent job. I’ve used this under Cygwin as well as my gentoo, but should work on any platform gs runs on. Here is a stab at a simple bash script:

#!/bin/bash

usage() {
        echo "$0 {first page} {last page} {input pdf file} {output pdf file}"
}

extract() {
  gs -sDEVICE=pdfwrite -dNOPAUSE -dBATCH -dSAFER \
     -dFirstPage=$1 -dLastPage=$2 \
     -sOutputFile=$4 $3
}

EXPECTED_ARGS=4
E_BADARGS=65

if [ $# -ne $EXPECTED_ARGS ]
then
  usage
  exit $E_BADARGS
else
  extract $1 $2 $3 $4
fi

Here is a bash script I wrote to enter and exit a chrooted environment on my Gentoo.

#!/bin/bash

# define here the chroot jail path
CHROOT=/home/chroot-jail

start() {

   echo "Mounting chroot dirs"

   # This allows DNS lookups via your system's networking
   cp -L /etc/resolv.conf ${CHROOT}/etc/

   mount -o bind /proc ${CHROOT}/proc
   mount -o bind /proc/bus/usb ${CHROOT}/proc/bus/usb

   mount -o bind /dev ${CHROOT}/dev
   mount -o bind /dev/pts ${CHROOT}/dev/pts
   mount -o bind /dev/shm ${CHROOT}/dev/shm

   mount -o bind /sys ${CHROOT}/sys

   chroot ${CHROOT} /bin/bash

}

stop() {

   echo "Unmounting chroot dirs"

   umount -f ${CHROOT}/proc/bus/usb >/dev/null
   umount -f ${CHROOT}/proc >/dev/null &

   umount -f ${CHROOT}/dev/pts >/dev/null
   umount -f ${CHROOT}/dev/shm >/dev/null
   umount -f ${CHROOT}/dev >/dev/null &

   umount -f ${CHROOT}/sys >/dev/null &

}

usage() {
   echo "$0 [start|stop]"
}

if [ "$1" == "start" ]
then
   start
elif [ "$1" == "stop" ]
then
   stop
else
   usage
fi

You can find more information on setting up a chroot at the:

• Gentoo x86 Chroot Setup Guide
• 32Bit Chroot Guide for Gentoo/AMD64

I like to keep logs and since space is cheap, why not just keep 2 or 3, or maybe 10 years of logs? That’s all fine, but soon the logging directories will become cluttered. Well, there are plenty of ways to keep everything tidy and clean, but even a separate backup isn’t very suitable for me, as backups tend to cycle quicker, and I do want to keep my logs a lot longer. So, how can we go about moving anything older than a certain date into a subdirectory of our logs and let it sit there?

One way I like is a combination of find and rsync.

cd /var/logs/
find .                                                 \
     -name BACKUP -prune                               \
     -o                                                \
     -newermt 20080101 ! -newermt 20101231             \
     -type f                                           \
     -exec                                             \
     rsync -avR --remove-source-files {} ./BACKUP/ \;

It’s quite simple really, ahem… once you know it, but lets decompose the command and try to understand what each part of the command does:

1. find . — Search into the current directory
2. -name BACKUP -prune — exclude files/directories matching the pattern BACKUP
3. -o — expr1 -o expr2 means if expr1 is true do not evaluate expr2, hence whichever file/dir does not match BACKUP will be subjected to the actions of the options next
4. -newermt 20080101 ! -newermt 20101231 — the modification of the files is between 20080101 and 20101231
5. -type f — process files only
6. -exec rsync — execute the following command, in our case rsync. {} is the matched file
7. -avR --remove-source-files — These are options to rsync!!
   • -a — archive
   • -v — verbose
   • -R — recursive
   • --remove-source-files — delete files once copied
8. {} ./BACKUP/ — since {} is the filename matched, this is the file that will be copied to the destination ./BACKUP/

If you want to exclude more subdirectories/files, except BACKUP, then you could change step 2 above with:

2. \( -name BACKUP -o -name mysql \) -prune — this will exclude both BACKUP and mysql

To login to remote hosts via ssh without having to type your password all the time, one way is to use Public Key authentication. The easiest way is to copy from your client machine the public key, for instance the contents of ~/.ssh/id_rsa.pub, into the .ssh/authorized_keys of the remote host. To do this use the special utility ssh-copy-id:

ssh-copy-id stathis@remote.host.com

One of the problems I encountered is when you try to use this utility with a remote ssh server that is not listening on the default port. To use a different port use the following syntax:

ssh-copy-id '-p 2222 stathis@remote.host.com'

When moving many files across the wire, for example via ftp, sometimes files get corrupted or are transferred partially. Just because two files have the same size it does not mean that they have successfully been transferred. This problem can manifest itself when moving many thousands of files. I usually use either of two methods to verify that files have been transferred properly:

(a) Use file checksums (e.g. using md5deep, sha256deep).
(b) Use rsync’s checksum option.

Using file checksums

To create hashes for a tree of files use the md5deep utility, or the included sha256deep, which computes a message digest for each file using the SHA-256 algorithm. The md5deep project is a collection of utilities using a variety of hashing algorithms to create a checksum for each file. That list of checksums can be then used on the receiving side of a transfer to verify the integrity of the transferred files.

To create a list of file checksums, having relative paths on the transmitting side:

cd files_to_send
sha256deep -rl * > hashes.sha

Now, copy the hashes.sha file to the receiving side, along with the files. To check against this file on the receiving side:

cd files_received
sha256deep -rl -x hashes.sha * > files_did_not_match.txt

The files_did_not_match.txt file will contain any files that have problems. Resend them and verify them again.

Using rsync –checksum

You can also use rsync’s built-in checksum functionality to upload files while verifying their checksum:

rsync -av --checksum user@source.host.com:/uploads/files_to_send/* \
                     user@destination.host.com:/uploads/files_received/ \
                     > sync.log

This is a one-step operation with less hassle, but implies you have rsync. There are many things you can do with rsync, so it is recommended to take a look into it’s various options if you transfer files, mirror, or sync between hosts often. On Windows, you can download cwrsync, a packaging of cygwin and rsync.

Qt on windows does not compile with MySQL support out of the box. If you’re unsure how to compile Qt on Windows with Visual Studio take a look at my other post on how to create a Qt Shadow build with Visual Studio. Based on the setup explained there, you have to first download the MySQL server. I recommend using the zipped pre-compiled binary releases (such as mysql-5.5.10-win32.zip), otherwise you’ll have the burden to compile MySQL yourself too. Decompress this into a directory of your choice, e.g. D:\dev\mysql.

Now, when you configure your Qt build, add the appropriate options shown coloured below:

configure -ltcg -opensource -mp -qt-sql-mysql -I D:\dev\mysql\include -L D:\dev\mysql\lib -l libmysql

You should see the Qt configuration output mentioning MySQL support. Follow the subsequent steps outlined in my other post to compile Qt and if everything is setup properly, you will end up with a Qt version that supports MySQL via QSql.

Using the LGR encoding for writing greek in latex documents on gentoo presumably should be fairly straight forward, but it took me a bit of searching before I figured it out. I’m using app-text/texlive, but I have not had greek language support (duh!). The error I was getting looked like this:

kpathsea: Running mktextfm grmn1095
mktextfm: Running mf-nowin -progname=mf \mode:=ljfour; mag:=1; nonstopmode; input grmn1095
This is METAFONT, Version 2.718281 (Web2C 7.5.7)

kpathsea: Running mktexmf grmn1095

! I can’t find file `grmn1095′.
<*> …ljfour; mag:=1; nonstopmode; input grmn1095

Please type another input file name
! Emergency stop.
<*> …ljfour; mag:=1; nonstopmode; input grmn1095

Transcript written on mfput.log.
grep: grmn1095.log: No such file or directory
mktextfm: `mf-nowin -progname=mf \mode:=ljfour; mag:=1; nonstopmode; input grmn1095′ failed to make grmn1095.tfm.
kpathsea: Appending font creation commands to missfont.log.

LaTeX Font Warning: Font shape `LGR/ptm/m/n’ undefined
(Font) using `LGR/cmr/m/n’ instead on input line 28.

That’s not much help really if you don’t know what you’re looking for. The solution on Gentoo is quite simple; just recompile TeXLive with greek language support:

USE+=”linguas_el” emerge -Dva app-text/texlive

A useful page on the LGR encoding support and example tex files to test your installation can be found in the Greek with the LGR font encoding webpage. To test, download lgrxenc.def and lgrxenc-test.tex and run latex lgrxenc-test && dvipdf lgrxenc-test. If everything goes well, you should end up with a pdf file with the greek characters rendered properly. You can compare your file with lgrxenc-test.pdf. Check the example test file for the latex commands you need in order to write using the LGR encoding.

A “shadow build” is an out-of-source build, where the source code is kept in one place and the build tree in another. I use this method to compile Qt on windows with Visual Studio (as of writing this post with VS2010).

Preparation

First download the latest Qt source code package. I usually download the latest qt-everywhere-opensource-src-x.x.x from http://get.qt.nokia.com/qt/source/.

Unpack it into the directory you want the source code kept, I choose D:\dev\qt-src. Create also the destination directory where you want your build to be, I chose D:\dev\qt-build. The directories can be anything and they can be in different drives, but I recommend avoiding to use directory names with spaces, as this may cause problems.

You will also need to have Perl installed to run the configure script of Qt. If you don’t already have it, I recommend using the portable version of Strawberry Perl for Windows. You can download it here. As this is a one-off step and you won’t need to do this next time you want to build Qt, unpack it into a suitable location in your system. I chose D:\dev\perl. Notice the batch file portableshell.bat, as we will need this in a bit.

Directory of Strawberry Perl for Windows

Configuring and Compiling Qt

Now, open the Visual Studio Command Prompt, as shown here:

Visual Studio Command Prompt

You have to open the VS Command Prompt instead of a normal Command Prompt (cmd.exe), as this will load the Visual Studio variables into your environment, e.g. location of the Visual C++ compiler, linker etc. You also have to setup your environment with Perl support, so execute from within the VS Command Prompt the portableshell.bat. You can easily do this:

D:\dev\perl\portableshell.bat

Now that your environment is setup with both VS and Perl support, change into the directory you intend to build and run the configure script in the source directory:

cd D:\dev\qt-build
D:\dev\qt-src\configure -ltcg -opensource -mp

You may want to have other switches for your Qt, so investigate the options with configure -help, but these will work just fine if you don’t have any special needs. Briefly:

• -ltcg enables Link-time Code Generation, which tells the linker to call the compiler and perform whole program optimization.
• -opensource tells Qt you’re building for Open Source development, not commercial.
• -mp enables building Qt with multiple processes, which will speed up your build. Use this if you have a computer with multiple cores or cpus.

The configure script will run for a while to compile some basic tools required and configure the Qt build for your system. Once configuration is finished you are ready to compile Qt. This may take a couple of hours and around 15-20GB of free space may be required to build everything in D:\dev\qt-build. While still in D:\dev\qt-build, to start the compilation:

nmake -f D:\dev\qt-src\Makefile

Once the compilation is finished you may also want to build the documentation for Qt Assistant, which isn’t built by default, you can do this with:

nmake -f D:\dev\qt-src\Makefile docs

If everything goes according to plan Qt will be built in D:\dev\qt-build and may occupy some 15-17GB of hard disk space. If you don’t want to rebuild Qt, you can clean up the build from intermediate files:

nmake -f D:\dev\qt-src\Makefile clean

and you can also delete the sources D:\dev\qt-src to free up some space.

Software used

• Microsoft Visual Studio 2010 Ultimate Edition
• Nokia Qt v4.7.2
• Strawberry Perl for Windows (Portable) v5.12.1