Are you loosing your angle brackets? - php and libxml2

problem: php processes some xml and your angle brackets in said xml vanish!

TLDR: use the CDATA tag to wrap your character data to avoid having angle brackets vanish.

I encountered this bug for the first time when I was importing a Cacti xml graph template via the Cacti web UI. On the surface everything seems to go well with the import process but then nothing was graphing and looking deeper, it was clear the config for the graph(s) was broken, due to missing < > angle brackets.
The brackets we're correctly encoded in the xml, it seems that somewhere between php and certain version of libxml the encoded angle brackets get stripped out.

Online there are a few bug reports but no single central bug id that I could find on this. One of the more useful shares online was a bug detail report for a closed google code project which provides Cacti mysql templates. Here is the bug detail, very useful info from Elan there.

During my search for solutions, it seemed likely that a bug was regressed or introduced in libxml, but that isn't certain. It would seem that the latest stable php 4.2 on Debian squeeze and libxml2 (as of writing 2.7.8.dfsg-2+squeeze3) still has the bug.

There is also some useful info on a bug report for MediaWiki project, entitled: Import strips angle brackets on some installations (libxml2 entity bug). To summarise, the consensus seems to be that its an upstream but with libxml2. The evidence I have found would agree with this.

Currently my systems are pinned on PHP 4.2 packages, perhaps this bug is not a factor in non pinned Debian squeeze systems?

impact: wastes time - fixing things that shouldn't really be broken.

solution: use the XML CDATA tag

I can provide is a way of easily checking if your set up has the bug or not. Props to Elan for this.

$ php -r '$p = xml_parser_create(); xml_parse_into_struct($p, "<path_php_binary>", $vals, $index); print_r($vals);'

A system suffering from the bug will include the output:

[value] => path_php_binary

A system NOT suffering from the bug will output:

[value] => <path_php_binary>

Now add the CDATA tag and see if the bug goes away? You don't even need to use entities when using the CDATA tag.

$ php -r '$p = xml_parser_create(); xml_parse_into_struct($p, "<![CDATA[<path_php_binary>]]>", $vals, $index); print_r($vals);'

citation:

Props to:
Elan Ruusamae for their bug detail report on the mysql cacti templates project.

testing for interactivity in a bash script

problem: you want to know if your bash script is being run interactively or not

This can be very useful to know, for example, if you want to programmatically determine if you should output to stdout, OR only to a log file because your script is running non-interactively.

When a script is running interactively, it means a stdout is present and one can output things to stdout and the user will see them.

When a script is running non-interactively, for example when cron runs a script, stdout does not exist.

solution: check if stdout is a tty

As with so many things, there is more than one solution, but this one seems to be rock solid and very portable.

is_interactive() { if [ -t 1 ]; then return 0; else return 1; fi }

As The Advanced Bash-Scripting Guide points out, for a complete test, one needs to check if stdout could be a socket too. Here is is the improved check:

is_interactive() { if [[ -t 1 || -p /dev/stdout ]]; then return 0; else return 1; fi }

I hope you enjoy and have fun tuning to your needs.
FYI. I tested the functions with cron and ssh on Debian squeeze (6.0.4) and bash 4.1.5.

citation:

Props to:
The Advanced Bash-Scripting Guide

How do I ask screen to behave like a standard shell?

I have found myself asking this question a few times, but never really got motivated to find a solid answer until recently.

I stole the question title from a good thread over at serverfault. Where I also contributed my findings.

I hope this share helps someone!

citation:

Props to:
rg3 @ LinuxQuestions.org for this share.

Windows 7 backup and restore error 0x80070020

problem: Windows 7 backup won't run or has stopped running

impact: Your stuff is not being backed up!

solution: Try creating a backup user or simply logging out

I had a user who's backup stopped working. After consulting with Dr. Google, non of the usual solutions would resolve the issue.

One person reported solving the issue by ensuring no users were logged in when the backup schedule ran, which makes sense, seeing as the error is related to some kind of file sharing violation.

This got me thinking, perhaps the usual user account had some processing running that were locking out some files. So I had the idea to create a backup user account, to see if the back up would run under that account. It did.

At the time of writing, I don't have any better suggestions, other than trying one of the above. It would seem that for my user, the error was related to a process running under their normal user account, locking out some files. Until that process is found and killed, it would seem that backing up while logged in as that account is not going to work.

P.S. Its worth noting, I was attempting to do a full backup for the computer in question.

QNAP NAS - mount the TimeMachine share

If you've got a QNAP NAS and you're using the TimeMachine capabilities of the machine, you might be interested to know you can manually mount the TimeMachine share on your MAC at your leisure, not just when you open TimeMachine on your MAC or when a TimeMachine backup is running.

I've shared the info on the QNAP forums and repeating here for good measure.

mount_afp afp://TimeMachine:**PASSWORD**@nas-hostname/TMBackup /Volumes/somefolder

Notes

• TimeMachine is the username, specified in the NAS Time Machine cfg
• **PROTECTED** is the password , specified in the NAS Time Machine cfg
• nas-hostname is the hostname of your NAS
• TMBackup is the Display Name, specified in the NAS Time Machine cfg
• /Volumes/somefolder is a folder to mount to on your MAC

Looking for duplicate files using freeware and open source tools

Recently I have been consolidating my files and backups in an attempt to centralise my backup efforts, and to also optimise hard drive usage and reduce data duplication. For me, this spans Windows, Macs and Linux systems.
Documented here are some of my findings, useful commands, and programs I have been using to look for duplicate files.

The open source

I tried a few open source tools including Duplicate Files Finder (windows) and DUMP3 (java). With all respect to the authors  work on their projects, they didn't do it for me. I would be happy to learn of other open tools for this kind of work.

DFF seemed to work, but personally I didn't get on very well with the results window, especially with a lot of results. The file counts seemed to be a little off too :( I didn't use it in the end.
I cannot recommend DUMP3, out of the box it was sooooo slow I gave up. On the positive side the GUI has a lot of promise.
There is also DUFF (windows), which looks like it has potential but sounded buggy and I didn't try it.

Googling also revealed a duplicate file finder thread on superuser, feel free to check it out.
This su thread lead me to discover Michael Thummerer's AllDup. What a great tool and its freeware. Props to Synetech inc su user for introducing me!

Yet Another Duplicate File Remover was also mentioned on the su thread and I came across it when Googling too. I did not test YADFR but it looks like it has potential.

The freeware

Alldup is everything I can think of, that a duplicate file manager needs to be, with whistles and bells on! Alldup even stopped me overzealously nuking some MP3's by intelligently checking "are you really sure you want to do that?"

The  GUI is fantastic, and made perfect sense to me. The results window is the best I've seen, very powerful. There are many features and implementations in this program where I have thought "that's probably how I would of done it", which is why I like it so much and highly recommend you check it out.

I've checked tens of thousands of files with Alldup, of many flavours and sizes and its works FAST! The results are so easy to work with too!

Freeware also worth mentioning, but not tested is Duplicate Cleaner.

The Homebrew

Before I found AllDup, my approach was as follows, which should work with bash under Cygwin, Linux and OS X.

The first challenge was to generate a list of files to work with, I went with generating a list of file extensions for a given location, so I could see what types of files I was dealing with.

Sanity checks, to verify results later, how many and what types of objects are we dealing with?

$ find . | wc -l && find . -type f | wc -l && find . -type d | wc -l
12840 (all objects)
11354 (files)
1486 (dirs)

The last two numbers should sum equal the first number, if not, you've probably got some links or special file types in the location you're analysing. Good to know this up-front.

Now to figure out what kinds of file are in the location you're analysing. First up, check how many files don't have a regular extension:

$ find . -type f | egrep -vi '\..{1,5}$' | wc -l
32

If the result was non-zero, you'll want to inspect the results, to ensure they can be ignored, or not.

$ find . -type f | egrep -vi '\..{1,5}$' | less

The following command should give a complete list of file extensions for the given location, from the results one can choose the extensions to focus on.

$ find . -type f -and -printf "%f\n" | egrep -io '\.[^.]*$' | sort | uniq
.jpeg
.jpg
.m3u
.m4a
.m4v
.mp3
.mpeg
.ogg

Now manually inspect the list of files for the extensions that are interesting:

$ find . -regextype posix-egrep -and -type f -and -iregex '.*\.(mp3|m4a|ogg)$'  | less

Once you're happy with the list, its time to generate some checksums. There are many hashing algorithms available for doing this. Two common ones are md5 and crc. On my i7 system under Cygwin, scanning ~9000 files took 44 minutes with md5sum vs. 20 minutes with cksum.

$ find . -regextype posix-egrep -and -type f -and -iregex '.*\.(mp3|m4a|ogg)$' -and -print0 | xargs -0 -P1 -I{} -- cksum {} > my-stuff.cksum

So after generation of a list of checksums with your preferred hashing algorithm, its time to put the hashes to work. The concept is as follows:

Isolate the hashes, sort them, find duplicates, output to file, use the list of duplicate hashes to match the files in your main checksum list.

It looks something like this:

$ awk '{print $1}' my-stuff.cksum | sort | uniq -d > my-stuff-dups.txt

Now search for the dups in your cksum file:

fgrep -f my-stuff-dups.txt MP3/my-stuff.cksum | less

Free 90 day Norton Internet Security 2012 - English, German, and more!

I wrote up an post a bout this for NIS 2011, the exact same applies to 2012.

Here are the links for 2012 that i could find and/or guess. I tested each link. I'm sure you can change the link accordingly for other languages.

all products are version 2012 v19.1 with 30 days of activation
EN	DE	NL	FR
NIS 90 day 30 day	NIS	NIS	NIS
NAV	NAV	NAV	NAV