Tuesday, December 13, 2011

QNAP NAS - mount the TimeMachine share

If you've got a QNAP NAS and you're using the TimeMachine capabilities of the machine, you might be interested to know you can manually mount the TimeMachine share on your MAC at your leisure, not just when you open TimeMachine on your MAC or when a TimeMachine backup is running.

I've shared the info on the QNAP forums and repeating here for good measure.
mount_afp afp://TimeMachine:**PASSWORD**@nas-hostname/TMBackup /Volumes/somefolder
  • TimeMachine is the username, specified in the NAS Time Machine cfg
  • **PROTECTED** is the password , specified in the NAS Time Machine cfg
  • nas-hostname is the hostname of your NAS
  • TMBackup is the Display Name, specified in the NAS Time Machine cfg
  • /Volumes/somefolder is a folder to mount to on your MAC

Thursday, December 1, 2011

Looking for duplicate files using freeware and open source tools

Recently I have been consolidating my files and backups in an attempt to centralise my backup efforts, and to also optimise hard drive usage and reduce data duplication. For me, this spans Windows, Macs and Linux systems.
Documented here are some of my findings, useful commands, and programs I have been using to look for duplicate files.

The open source

I tried a few open source tools including Duplicate Files Finder (windows) and DUMP3 (java). With all respect to the authors  work on their projects, they didn't do it for me. I would be happy to learn of other open tools for this kind of work.

DFF seemed to work, but personally I didn't get on very well with the results window, especially with a lot of results. The file counts seemed to be a little off too :( I didn't use it in the end.
I cannot recommend DUMP3, out of the box it was sooooo slow I gave up. On the positive side the GUI has a lot of promise.
There is also DUFF (windows), which looks like it has potential but sounded buggy and I didn't try it.

Googling also revealed a duplicate file finder thread on superuser, feel free to check it out.
This su thread lead me to discover Michael Thummerer's AllDup. What a great tool and its freeware. Props to Synetech inc su user for introducing me!

Yet Another Duplicate File Remover was also mentioned on the su thread and I came across it when Googling too. I did not test YADFR but it looks like it has potential.

The freeware

Alldup is everything I can think of, that a duplicate file manager needs to be, with whistles and bells on! Alldup even stopped me overzealously nuking some MP3's by intelligently checking "are you really sure you want to do that?"

The  GUI is fantastic, and made perfect sense to me. The results window is the best I've seen, very powerful. There are many features and implementations in this program where I have thought "that's probably how I would of done it", which is why I like it so much and highly recommend you check it out.

I've checked tens of thousands of files with Alldup, of many flavours and sizes and its works FAST! The results are so easy to work with too!

Freeware also worth mentioning, but not tested is Duplicate Cleaner.

The Homebrew

Before I found AllDup, my approach was as follows, which should work with bash under Cygwin, Linux and OS X.

The first challenge was to generate a list of files to work with, I went with generating a list of file extensions for a given location, so I could see what types of files I was dealing with.

Sanity checks, to verify results later, how many and what types of objects are we dealing with?
$ find . | wc -l && find . -type f | wc -l && find . -type d | wc -l
12840 (all objects)
11354 (files)
1486 (dirs)
The last two numbers should sum equal the first number, if not, you've probably got some links or special file types in the location you're analysing. Good to know this up-front.

Now to figure out what kinds of file are in the location you're analysing. First up, check how many files don't have a regular extension:
$ find . -type f | egrep -vi '\..{1,5}$' | wc -l
If the result was non-zero, you'll want to inspect the results, to ensure they can be ignored, or not.
$ find . -type f | egrep -vi '\..{1,5}$' | less
The following command should give a complete list of file extensions for the given location, from the results one can choose the extensions to focus on.
$ find . -type f -and -printf "%f\n" | egrep -io '\.[^.]*$' | sort | uniq
Now manually inspect the list of files for the extensions that are interesting:
$ find . -regextype posix-egrep -and -type f -and -iregex '.*\.(mp3|m4a|ogg)$'  | less
Once you're happy with the list, its time to generate some checksums. There are many hashing algorithms available for doing this. Two common ones are md5 and crc. On my i7 system under Cygwin, scanning ~9000 files took 44 minutes with md5sum vs. 20 minutes with cksum.
$ find . -regextype posix-egrep -and -type f -and -iregex '.*\.(mp3|m4a|ogg)$' -and -print0 | xargs -0 -P1 -I{} -- cksum {} > my-stuff.cksum
So after generation of a list of checksums with your preferred hashing algorithm, its time to put the hashes to work. The concept is as follows:
Isolate the hashes, sort them, find duplicates, output to file, use the list of duplicate hashes to match the files in your main checksum list.
It looks something like this:
$ awk '{print $1}' my-stuff.cksum | sort | uniq -d > my-stuff-dups.txt
Now search for the dups in your cksum file:
fgrep -f my-stuff-dups.txt MP3/my-stuff.cksum | less

Sunday, November 27, 2011

Free 90 day Norton Internet Security 2012 - English, German, and more!

I wrote up an post a bout this for NIS 2011, the exact same applies to 2012.

Here are the links for 2012 that i could find and/or guess. I tested each link. I'm sure you can change the link accordingly for other languages.

all products are version 2012 v19.1 with 30 days of activation
NIS 90 day 30 day NIS NIS NIS

Tuesday, August 30, 2011

Counter Strike Source crashes on launch after Migrating from Vista to 7

problem: After migrating from Vista to 7, CS:S no longer launches

You have Migrated from Vista to 7. I personally chose to clean install 7, as my games and other personal files are located on different hard drives.

When your system is ready for gaming, you load Steam, and when you try to launch CS:S, you see that hl2.exe launches but quickly dies. The game does not actually load 3D mode, so its not a crash to desktop, it just bails out on launching the hl2.exe while still on the desktop.

solution: force resolution and DirectX level

In the Counter Strike Source game properties, from the Steam games list, set the launch options as follows:
-dxlevel 90 -w 1024 -h 768
This should force CS:S to sort things out, with whatever is causing it to crash on launch. You can set your preferred settings in-game, quit the game, remove the launch options and then test to ensure everything is back to normal.

It is possible that my issue was specifically with CS:S remembering the wrong display adapter reference, perhaps it was not finding the reference to the previously known adaptor and getting its pants in a twist?

Its worth noting that the issue was CS:S specific in my case. Other hl2.exe based games work working fine.

Prior to trying this solution, I tried a bunch of other generic troubleshooting steps, including trying the -32bit launch option (given that I'm running x64 Windows 7). I also tried moving my existing cstrike cfg, to see if defaults would help. None of my attempts worked.

What MIGHT of worked, which I only learned about post finding the solution, was the -autoconfig launch option.

Feel free to read more about -dxlevel and -w -h launch options at the Valve developer community.
For specific info on dxlevel (mat_dxlevel) check out the Value developer community page for that cvar.

Successfully driving 3 DVI displays from a AMD/ATI 6970 GPU (Eyefinity)

problem: You cannot get 3 displays working on your 6970

  • You have three displays with DVI inputs.
  • You have a 6970 GPU.
  • Your running windows 7 (may apply to Vista and others).
  • You cannot get all three displays operational at the same time.
In my particular case, I was using the following hardware:
  1. Dell 3007WFP-HC (LG.Philips S-IPS).
  2. Dell 2407WFP (Samsung).
  3. LG L1980Q.
  4. GigaByte GV-R697OC2-2GD (ATI 6970 GPU).
Note, the Dell 30" requires Dual-Link bandwidth to display its native 2560x1600.
Note, the Dell 24" and LG 19" only require Single-Link bandwidth.

For reference, here is a diagram of my 6970 backplate connectors. Please note, the layout and types of outputs are manufacturer dependant. So one cannot say, this will be the configuration for all 6970 cards.

With my previous card, a Gigabyte ATI 5970, which has two DVI-I Dual-Link and a mini display port outputs.
I was using the two DVI outputs for the Dell's and an Apple mini Display Port to Analogue adaptor for the LG. This was the most cost effective set up that would drive all three panels at the same time. This worked well for me, because the LG 19" has analogue D-SUB and DVI inputs.
After a decent life, one of my 5970 DVI ports burnt out and left me no choice but to RMA the card. Gigabyte were absolutly amazing in handling my case and went beyond the call of duty to satisfy my requirements and concerns with the replacement card, the GV-R697OC2-2GD.

The new card came with two adaptors, a DVI to Analogue and a DVI to HDMI and a HDMI cable. There was no Display Port adaptor.

When setting up the new card and trying to get all my displays working, the first problem I came across was a limitation with the HDMI output and adaptor. As mentioned, all my monitors have DVI inputs, I have no native HDMI or display port inputs. So this meant using adapters to get things working... or at least that is what I thought.

ATTEMPT ONE, the Dell 30" would only display it's native 2560x1600 from the DVI-I output. I then had the Dell 24" on the DVI-D output these two together were fine. I then added the LG 19" output from the HDMI with the provided HDMI cable and used the DVI to HDMI adaptor connected to the LG.

PROBLEM ONE, with this set up, the CCC said I had to disable one of the monitors to get the HDMI output to display. Jokes. I guess this is a hardware restriction of some kind.
After a Google, I established that to drive 3 panels simultaneously, I needed to output using 2x DVI and 1x Display Port. OK. So another adaptor required.

Another Google led me to the most cost effective solution, a Display Port to Single-Link DVI adaptor. Note that the Dual-Link adaptors are _VERY_ pricey! I ordered a HIS Single-Link adaptor for ~20 EUR. Here is the HIS product page.

Note that Display Port does have the bandwidth for the native 2560x1600 of my Dell 30", however the adaptor was only Single-Link. So, the Dell 30" still had to be driven from the DVI-I output.

ATTEMPT TWO, Dell 30" driven from the DVI-I output, Dell 24" driven from the HDMI to DVI and the LG 19" driven from the Display-Port to DVI. NO DICE.
PROBLEM TWO, At this stage, I wonder if I'm hitting software issues with the ATI drivers. The Dell's would work fine and the LG would get a signal but would display a blank black screen.

ATTEMPT THREE, Dell 30" driven from the DVI-I output, Dell 24" driven from the Display Port to DVI and the LG 19" driven from the HDMI. SUCCESS. After some fiddling with the software settings, I stopped using the CCC and reverted to Windows 7 native display control panel, I got all three panels working simultaneously. Thank F.

Not satisfied that I'd exhausted all connection possibilities, I tried one more.

ATTEMPT FOUR, Dell 30" driven from the DVI-I output, Dell 24" driven from the Display Port to DVI and the LG 19" driven from the DVI-D. SUCCESS. Again, it seemed like using the native windows display control panel was more reliable than the ATI CCC.


I will be totally honest and say that I have NO IDEA why my ATTEMPT TWO did not work. In theory it should. Perhaps this is a software issue.

Good news, I do have two successful connection solutions. See attempts three and four above.
I hope the details shared here help someone with their own troubleshooting.

Sunday, August 28, 2011

My experience migrating GFWM (GFWL) from Vista to 7

First a bit of background, I had the horrible task of migrating from Windows Vista to 7 on my personal PC. I say horrible, because Microsoft don't seem to of thought things through for people with hardware and drive configurations outside the norm. I wasted a number of hours recovering my old Vista install and re-building the boot cfg to get everything working, however I will leave the details for another post.

So I had a few games under Games For Windows Marketplace, formally Games For Windows Live. I was wondering how I could avoid re-downloading the games/DLC again on my windows 7 clean install.

After a little trial and error I got things working and saved a bunch of time/bandwidth. I will focus on GTA4, which I purchased via Steam and subsequently purchased the DLC via GFWM.

This process could easily be streamlined, but I wanted to include everything. Please feel free to take your own approach to speed things up.

Note, Microsoft have updated the Games for windows client, so its now called Games For Windows Marketplace (GFWM). In GTA4, the GFW system is still called Games for Windows Live (GFWL).

UPDATE 2012-MARCH-10: I did a registry backup, then used the Games For Windows Marketplace to install one of the GTA4 DLC's. Then did a registry diff using Beyond Compare. Studying the diff revealed no specific changes related the DLC downloaded from the GFWM. So I'm very confident that all of the magic that makes the GTA4 DLC work via GFWM/GFWL, is located in the file/folder structure located at:
VERY IMPORTANT: If you change the location of your installed games, or DLC via the Games for windows Marketplace UI, it COULD delete any game content in the original location and perhaps even the new location. Make sure you have a backup of your content that is NOT located in your old or new content folders.
  1. Ensured I had a tested backup of my games and DLC.
  2. Ensured I had a tested backup of the following folders from my Vista install
  3. Sorted out my Windows 7 clean install, drivers, cfg etc.
  4. Launched steam, this is on a dedicated games drive.
  5. Launched GTA4, this did some "first run" steps.
    This included installing GFWL, though a very old copy.
  6. GTA4 launched, Social Club sign in etc.
  7. Signed in via GFWL, allowed the GFWL update to download.
  8. Exited the game, GFWL update ran.
  9. Loaded GTA4 again, issues with activation, GFWL crashing.
  10. Did a windows update on GFWL and it upgraded to GFWM.
  11. Launched GTA4 again, GFWL sign in worked.
  12. Launched GTA4 TBOGT DLC but save games were missing.
  13. Placed my backed up save games in:
    \Users\$youruser\AppData\Local\Rockstar Games\GTA IV\savegames\user_{some number}\
    Check that {some number} is the most recent folder created.
  14. Loaded GFWM (direct from Start Menu) and checked my Game/DLC status, it was not correct.
  15. I updated my GFWM settings with the ones from my screenshot from Vista.
  16. Exited GFWM.
  17. Deleted and Replaced the following folders on my Windows 7 install with ones from my Vista install.
  18. Loaded GFWM (direct from Start Menu) and checked my Game/DLC status, it was correct, settings were also still correct.
  19. Exited GFWM and launched GTA4.
  20. Signed in to GFWL.
  21. Entered my GTA4 key (available via Steam game properties).
  22. Everything works, save games were also as desired.
Props to:
Rjck @ Seven Forums.com for his post on save games

Thursday, August 11, 2011

Linux HOWTO: mount a UDF iso

problem: you want to mount a UDF iso

solution: mount -t auto

It is nice and simple's, after a Google, I found grando's blog post which explained a more permenant solution to said issue. Here is how I used his tip:
mount -t auto -o loop /path/to/my/iso.iso /mnt/myiso


Props to: amazingrando @ The Penguin Trail

Sunday, August 7, 2011

Automating addition of users to a DBM DB for HTTP auth

problem: the passwords you're generating don't work

  1. You're using one or more DBM DB's for storing HTTP authentication (authn_dbm_module).
  2. You want to automate adding users to a DBM DB.
  3. You want to use SHA1 passwords.
You're at a loss, as how to generate passwords that will work with automation.

impact: manual job to add users :(

Against the pragmatic principle of "Don't Repeat Yourself! (DRY).

solution: understand sha1 encryption for DBM and authn_dbm_module

One key bit of info buried in man htpasswd is the fact SHA1 passwords for DBM and authn_dbm_module are not salted. This does make life a little easier. However I missed this when looking for solutions and before looking at the source code of dbmmanage.

With something like the following in your script, you can automate adding users to a DBM DB:
dbmmanage -s $db add $user $password $groups
However, simply supplying a SHA1 password does not work. This is because DBM stores it SHA1 passwords as a base64 binary digest. More info about this over at CPAN.

So, in PHP for example, you could do something like the following to generate a valid password for your automation:
$enc_pass = base64_encode(pack("H*", sha1($password)));
Then you would use the $enc_pass for your automation :)

Further reading

Friday, July 29, 2011

dump/output/list Linux password expiry info for all users

problem: you need password expiry info for all users

solution: one line bash script to the rescue :)

Check out my thread @ Server Fault for a number of simple, one line bash solutions.


Props to: Zoredache @ ServerFault for his quick answer

Thursday, July 28, 2011

Free 90 day Norton Internet Security 2011 - English, German, and more!

UPDATE: superseded by Free 90 day Norton Internet Security 2012 - English, German, and more!

I recently figured out this rather handy to know tip for Norton Internet Security (NIS) and possibly for Norton Anti Virus too (NAV).

To cut a long story short, it is possible to transfer licences between different localisations of NIS and perhaps also NAV.

Lets say you want 90 days free NIS, but in German?
  1. You download and install the 90 day NIS trial from Norton, Thanks to ING Direct
    (this is English but don't worry!)
  2. You make sure to register a valid email address with your Norton Account for said install
  3. You then have 90 days free NIS trial, fully operational etc
  4. You then sign in to the created Norton account
  5. You copy, paste and save the licence from the trial, per screenshot
  6. Now, you need to procure the German version of NIS! Get it direct from Norton!
  7. You now uninstall the EN version of NIS, I think its needs a reboot.
  8. Then you install the DE version of NIS
  9. When NIS launches, hit the Subscribe link/button for the given language
  10. When presented with the subscription screen, just enter the product key you saved earlier and progress the form as normal. JOB DONE.
This might work for other Norton products too, like Antivirus 2011? If you try it, please do share your results!
Here are some handy direct links to some Norton products in various locales, courtesy of Smokey's Security Weblog.
all products are version 2011 v18.5
NIS 90 day 30 day NIS NIS NIS
(all links in this table are direct from Norton)

Props go out to:

Smokey @ Smokey's Security Weblog
MAYUR @ webtrickz.com

Sunday, February 6, 2011

Get your iPhone officially unlocked - for FREE - via iTunes

problem: Your iPhone is SIM locked

Your provider had your phone locked when you took on your contract... now you want to use a different SIM but it doesn't work.

solution: Ask your provider to unlock the phone!

It really should be very simple and FREE. For example o2 in the UK provide a online form for their customers and they'll unlock the phone within 14 days. This probably works for more than just iPhones too. Read more about o2's process on their site.

What is the catch? As you can imagine, there is likely a minimum contract term that needs to be fulfilled with most providers or at least they might claim that there is. Most countries have laws related to SIM locking and it depends on the providers and countries stance on such laws. Read more on SIM locking on wikipedia.

I went through the o2 process with a family members iPhone and it was totally painless. They quoted 14 days but it took less than 48 hours. A picture of the official unlocking in iTunes!

The process went like this:

  1. Googled iPhone unlocking, found Olly's post linked below
  2. Went to o2's site and search for unlocking etc
  3. Found the unlocking FAQ article on o2's site
  4. Filled out the unlocking form (dead easy)
  5. Received the SMS from o2 that the phone had been unlocked
  6. IMPORTANT: Inserted non o2 sim in to the phone
  7. Powered the phone on
  8. With an computer connected to the net, sync'ed the iPhone with iTunes
  9. After a brief wait, the iPhone received updated carrier programming and the new SIM was operational


A big thanks and props to Olly Farshi @ giga om for sharing his knowledge and experience on this topic.