This is TikiWiki v1.9.9 -Sirius- © 2002–2007 by the Tiki community Sat Oct 11, 2008 [23:34 UTC]
  add
Latest list posts
Menu
  Wiki
  Forums

LinuxTipsAndTricks

Little things that might make life with GNU/Linux (or similar platforms) easier for you.
backlinks print
history similar comment attach file

From time to time, little tips and tricks for using GNU/Linux pop up on the MLUG mailing list. I thought it might be helpful to put the up on the MLUG Tiki/Wiki so they'd be easier to find, and accessible to people who don't subscribe to the list.

Feel free to add your own tips and tricks.

? User Page" href="tiki-index.php?page=UserPagePhilG" class="wiki">pg

p.s. I don't mean to exclude you if you use another Open Source, Free Software, or Unix-like operating system - most of these tips will probably be useful for you too. And you are welcome to submit your own tips and tricks, as long as they would also be useful for GNU/Linux users.


Window Resizing & Moving Tips

Joe Baker mlug-list, Tue, 16 Dec 2003 14:06:37 -0600

A while back when we were looking at the KDE desktop I mentioned my favorite featureof KDE being that I could resize a window without moving to it's edge with the mouse.

In KDE I use Alt-RightClick & Drag to effect changes in the size of the current window.

Today I discovered that GNOME can do this also only it is the middle button on the mouse that you click & drag while holding the alt key.

This feature blows away Microsoft window managment techniques!

Also both KDE & GNOME let you move a windowby using ALT-Left Click & Drag.


Downloading distribution CD images efficiently

? User Page" href="tiki-index.php?page=UserPagePhilG" class="wiki">Phil Goembel mlug-list, Thu Nov 13 01:26:37 CST 2003

With some distros, I've found it saves a lot of time (and bandwidth) to build a new ISO (CD image file) from a local copy of an older version ISO, by using rsync. This only applies to distros that provide rsync servers, of course.

Just make a copy of the old ISO and rename it to the same name as the newer ISO, and rsync will only transfer the chunks that are different. Then rsync reassembles all the chunks (from both your local copy and the remote) in the right order to make your local copy match the remote copy.

Do a final md5sum on the new local ISO to make sure it actually does match the remote ISO's md5sum.

You'd be amazed at how little actually changes from one release to the next, except when there's a massive change like all the apps were recompiled with a new GNU compiler.


Verifying CDs:

? User Page" href="tiki-index.php?page=UserPagePhilG" class="wiki">Phil Goembel mlug-list, Thu Nov 13 01:26:37 CST 2003

When burning a data CD with cdrecord, I use the -pad option to add some zero data blocks to the end of the CD. Then, when I want to read the contents of a data CD back using dd, I specify the exact number of blocks to read back, instead of relying on dd to just read to the end of the CD.

To figure the right number of blocks to read back, take the size of the ISO image file and divide it by 2048 (and it should always be an exact multiple of 2048, or your image file is corrupted).

For example, I have an ISO image file for knoppix-std-0.1b.iso. In the session below, I burn the cd with padding, calculate how many 2048 byte blocks to read back, read that many blocks back with dd and pipe them into md5sum, then do an md5sum on the original ISO file for comparison:

>cdrecord -v speed=4 dev=2,0,0 -pad -data knoppix-std-0.1b.iso

>ls -l knoppix-std-0.1b.iso

-rw-rw-r-- 1 phil phil 625717248 Jun 26 01:42

knoppix-std-0.1b.iso

>expr 625717248 / 2048

305526

>dd if=/dev/scd0 bs=2048 count=305526 |md5sum -

305526+0 records in

305526+0 records out

b98226254b678520d1da5a93171768a1 -

>md5sum knoppix-std-0.1b.iso

b98226254b678520d1da5a93171768a1 knoppix-std-0.1b.iso

This has procedure has always worked for me. I've used it with at least 7 different drives on four different systems. Two of the drives are SCSI, the rest are IDE (using the ide-scsi module).

Before I started doing it this way, I used to get errors when dd reached the end of the CD track.

The padding was recommended by the author of cdrecord, to work around a read-ahead bug in one of the Linux cd-related kernel modules. I read about it a long time ago, so maybe the bug has been fixed. But I still notice dd tends to break (have io errors) when I don't specify the number of blocks to read or when I don't add padding blocks (sectors?).

UPDATED 2007-09-16:

I used the above procedure because dd kept terminating with errors when it got to the end of the CD. There is a design defect in the Linux CD ROM driver that causes it to read ahead past the last block on the CD.

The above process no longer works for me with two of my newest drives. The default padding apparently isn't always enough to reliably prevent the Linux read-ahead bug from happening. It seems that the best way to read back an entire CD image is to use readcd instead of dd. readcd uses raw device I/O, so it does not trigger the read-ahead logic used by the buffered I/O.

For example, I have burned CentOS-5.0-i386-LiveCD.iso to a CDR on my new burner at /dev/hdc. To read and checksum the CD from /dev/hdc using readcd instead of dd:

> # get number of 2048 byte blocks to read

> ls -l CentOS-5.0-i386-LiveCD.iso

-rw-rw-r-- 1 phil phil 711890944 Sep 9 21:34 CentOS-5.0-i386-LiveCD.iso

> expr 711890944 / 2048

347603

> #make sure CD isn't mounted (e.g. via automounter)

> umount /dev/scd0

> # read 347603 blocks from /dev/hdc and pipe them into md5sum

> readcd speed=16 dev=/dev/hdc sectors=0-347603 f=- |md5sum

Read speed: 2117 kB/s (CD 12x, DVD 1x).

Write speed: 2822 kB/s (CD 16x, DVD 2x).

Capacity: 347605 Blocks = 695210 kBytes = 678 MBytes = 711 prMB

Sectorsize: 2048 Bytes

Copy from SCSI (1,0,0) disk to file '-'

end: 347603

addr: 347603 cnt: 19

Time total: 434.400sec

Read 695206.00 kB at 1600.4 kB/sec.

67894bd2530c0063baa01ac2e93435b0 -

> # Now get the expected sum and compare.

> md5sum CentOS-5.0-i386-LiveCD.iso

67894bd2530c0063baa01ac2e93435b0 CentOS-5.0-i386-LiveCD.iso

> # Great - they match!


Finding Duplicate Files

Rick Miller mlug-list, Fri Jan 16 11:13:54 CST 2004

In response to Joseph Wagner's question:

I'm looking for utilities that can compare binary and ascii files. Basically I want to compare files that might be the same but have had their names changed.

Anyone know of some utilities that can do this? Please include any that run on winblows... Thanks in advance.

Brute force: "md5sum".

md5sum's output puts the checksum first, so you can concatenate them all in a single file and then *sort*. Any lines which start with the same checksum are most likely files which contain the exact same data.

Example: ? User Page" href="tiki-index.php?page=UserPagePhilG" class="wiki">PhilG

To find duplicate files (by content, not name) in a directory tree at /home/user:

find /home/user -type f -exec md5sum {} \; | sort >dups

dups will hold the md5 checksums and names of all the files in the directory tree, sorted by checksum. All files that have the same contents will be next to each other in dups, since they will have the same checksum.

To compare files in two or more directories, just include all the directories in the find command:

find /usr/src/linux-2.4.20-18.7 /usr/src/linux-2.4.20-19.7 -type f -exec md5sum {} \; | sort >dups

To remove the non-duplicates from a file (or output stream), you can use the uniq command. For example, to see only the lines from dups that have duplicates (matching only the checksums, which are the first 32 characters of each line):

uniq -D -w 32 dups | less

To learn more about sort and uniq, and other text utilities, type:

info textutils


Hard Drive Forensics

Joe Baker mlug-list, Fri Feb 13 13:54:49 CST 2004

Forensics (The science used by investigators when studying evidence in a crime) programs for Linux can be found by searching Freshmeat. Then type in the key word 'forensics' and search.

also...

Many people have suggested dd for copying data from a drive byte by byte, however if the source disk has errors, dd will fail at those error points...

Enter... dd_rescue

http://www.garloff.de/kurt/linux/ddrescue/index.html

A quote from the web site:

The latter three features make it suitable for rescuing data from a medium with errors, i.e. a hard disk with some bad sectors. Why?

  • Imagine, one of your partitions is crashed, and as there are some hard errors, you don't want to write to this hard disk any more. Just getting all the data off it and retiring it seems to be suitable. However, you can't access the files, as the file system is damaged.
  • Now, you want to copy the whole partition into a file. You burn it on CD-Rom, just to never lose it again. You can setup a loop device, and repair (fsck) it and hopefully are able to mount it.
  • Copying this partition with normal Un*x tools like cat or dd will fail, as those tools abort on error. dd_rescue instead will try to read and if it fails, it will go on with the next sectors. The output file naturally will have holes in it, of course. You can write a log file, to see, where all these errors are located.
  • The data rate drops very low, when errors are encountered. If you interrupt the process of copying, you don't lose anything. You can just continue at any position later. The output file will just be filled in further and not truncated as with other Un*x tools.
  • If you have one spot of bad sectors within the partition, it might be a good idea, to approach this spot from both sides. Reverse direction copy is your friend.
  • The two block sizes are a performance optimization. Large block sizes result in superior performance, but in case of errors, you want to try to salvage every single sector. So hardbs is best be set to the hardware sector size (most often 512 bytes) and softbs to a large value, such as the default 16k.

More info can be found in the README file included in the package.


DNS (Domain Name Service) Lookups

mlug-list, Sat Feb 14 16:19:54 CST 2004

Michael asks:

Is it possible to get an IP address of a domain when my current DNS servershavn't been updated? I can do a whois on the domain and get their DNSserver, but dont know how to use that information to get the IP address.

___

(Josh)

dig @nameserver.com yourdomain.com

___

(Dan)

You can do:

host hostname nameserver

which makes a dns query using a different nameserver and returns the IP or if you more info you can do:

dig ANY @nameserver hostname

You can use the --help option or read the man pages if you want more info on these commands.

___

(Andy)

Sure...

nslookup domain.com ns1.nameserver.com

dig domain.com @ns1.nameserver.com

Example...

abrezinsky@mbrez abrezinsky$ nslookup mbrez.com ns1.bytehosting.net

Server: ns1.bytehosting.net

Address: 38.119.218.30#53

Name: mbrez.com

Address: 38.119.218.102

___


Base64 Decoding

? User Page" href="tiki-index.php?page=UserPagePhilG" class="wiki">PhilG

Once in a while, I need to decode some text that is Base64 encoded, and I can never remember the name of the utility for decoding it. Here are some commands to decode a Base64 encoded file (found in a thread from the SLUG mailing list):

___

  • To: SLUG <slug@xxxxxxxxxxx>
  • Subject: Re: SLUG Base64 Decoder
  • From: John Clarke <johnc+slug@xxxxxxxxxxx>
  • Date: Fri Nov 22 14:20:03 2002

On Fri, Nov 22, 2002 at 02:03:01PM +1100, Andrew Wilson wrote:

> If anyone can point me in the direction of a working base64 decoder i would

> be most appreciative.

mimencode

~~ MIME::Decoder::Base64 ~~

Cheers,

John

___

  • To: Andrew Wilson <will@xxxxxxxxxxxxxxxxxxxxx>
  • Subject: Re: SLUG Base64 Decoder
  • From: "Broun, Bevan" <brounb@xxxxxxxxxxxxxxx>
  • Date: Fri Nov 22 14:25:03 2002

on Fri, Nov 22, 2002 at 02:03:01PM +1100, Andrew Wilson <will@netwaynetworks.com.au> wrote:

> Hay guys,

>

> If anyone can point me in the direction of a working base64 decoder i would

> be most appreciative.

> Prize : cookie

The best ive found is uudeview. search freshmeat for it.

BB

___

  • To: Andrew Wilson <will@xxxxxxxxxxxxxxxxxxxxx>
  • Subject: Re: SLUG Base64 Decoder
  • From: Matthew Hannigan <mlh@xxxxxxxxxx>
  • Date: Fri Nov 22 16:01:03 2002

On Fri, Nov 22, 2002 at 02:03:01PM +1100, Andrew Wilson wrote:

> Hay guys,

>

> If anyone can point me in the direction of a working base64 decoder i would

> be most appreciative.

> Prize : cookie

openssl will do it — no frills though:

openssl base64 -d < inputbase64.txt > output.txt

(that's -d for decode. Encoding is left as an exercise the reader :-)

Matt

___

So, to summarize, that's 3 command line tools and one Perl module:

  • mimencode
  • uudeview
  • openssl
  • ~~ MIME::Decoder::Base64 (a Perl module) ~~

My system has mimencode, and it's pretty simple to use, so that is my preferred tool. uudecode should work too, if the uuencode header is in the file. ? User Page" href="tiki-index.php?page=UserPagePhilG" class="wiki">pg

p.s. Please ignore the stupid multiple occurances of "~~" above - it's the brain damaged TikiWiki parser. It interprets the double colons as format characters and there seems to be no clean way to escape them. The it doesn't have a simple escape character, and doesn't seem to handle its published escape mechanism rationally either.

Created by: PhilG last modification: Monday September 17, 2007 [19:11:31 UTC] by PhilG


name uploaded size dls desc
No attachments for this page
Powered by Tikiwiki Powered by PHP Powered by Smarty Powered by ADOdb Made with CSS Powered by RDF powered by The PHP Layers Menu System
RSS Wiki RSS Image Galleries RSS Forums rss Directories
[ Execution time: 0.51 secs ]   [ Memory usage: 8.15MB ]   [ GZIP Disabled ]   [ Server load: 0.14 ]