Run shell script in a KiTTY cygterm (cygwin) from Notepad++

Approach 1 with helper script

"R:\REPOSITORY\FILES\Dropbox\linux\shared\home\XXXX\scripts\cygterm-launcher.bat" "$(FULL_CURRENT_PATH)"

helper script

for /f %%i in ('C:\cygwin\bin\cygpath %1') do set UnixPath=%%i

start "" "S:\Program Files\KiTTY\kitty_portable.exe" -cygterm -cmd "bash %UnixPath%" -

Approach 2 without helper script

c:\cygwin\bin\bash -c "_script=$(/usr/bin/cygpath '$(FULL_CURRENT_PATH)');  /cygdrive/s/Program\ Files/KiTTY/kitty_portable.exe -cygterm -cmd \"bash '$_script'\" -"

Note: sleuthkit Debian package

It contains istat and blkcat and friends.

Tools for forensics analysis (per Wikipedia):

The Sleuth Kit (TSK) is a library and collection of Unix- and Windows-based utilities for extracting data from disk drives and other storage so as to facilitate the forensic analysis of computer systems. It forms the foundation for Autopsy, a better known tool that is essentially a graphical user interface to the command line utilities bundled with The Sleuth Kit.

Mingle MQL API by example

I was playing around with the Mingle API's today, some notes to self for future development...

An example of a raw GET request to Mingle's MQL API, notes:

• against Mingle 12.1
• URL encoding on the query string values

GET /api/v2/projects/YOUR_PROJECT/cards/execute_mql.xml?mql=SELECT%20number%2C%20name%20WHERE%20'Release'%20%3D%20(Next%20Release)%20AND%20Type%20%3D%20Feature HTTP/1.1
Host: YOUR_HOST:YOUR_PORT
User-Agent: curl/7.24.0 (i686-pc-cygwin) libcurl/7.24.0 OpenSSL/1.0.1c zlib/1.2.7 libidn/1.22 libssh2/1.4.0
Accept: */*
Cookie: mingle_12_1_2_session_id=SOME_SESSION; login=SOME_LOGIN

Note above how I'm hi-jacking an existing Mingle session to perform this request.

curl exmple

Here is a curl command and its resultant raw request, note I'm running this through a local debugging proxy, using https and basic authentication. I use Charles web debugging proxy, I am a huge advocate of this tool, and the use of any debug proxy in general :)

curl -k --proxy localhost:8888 'https://YOUR_USER:YOUR_PASS@YOUR_HOST:8080/api/v2/projects/YOUR_PROJECT/cards/execute_mql.xml?mql=SELECT%20number%2C%20name%20WHERE%20%27Release%27%20%3D%20%28Next%20Release%29%20AND%20Type%20%3D%20Feature'

In order for the basic authentication to work, you need to turn on Mingle's basic authentication, a simple change and its documented here. I restarted Mingle after this change.

The raw request from the curl

GET /api/v2/projects/YOUR_PROJECT/cards/execute_mql.xml?mql=SELECT%20number%2C%20name%20WHERE%20%27Release%27%20%3D%20%28Next%20Release%29%20AND%20Type%20%3D%20Feature HTTP/1.1
Authorization: Basic --REMOVED--
User-Agent: curl/7.24.0 (i686-pc-cygwin) libcurl/7.24.0 OpenSSL/1.0.1c zlib/1.2.7 libidn/1.22 libssh2/1.4.0
Host: YOUR_HOST:8080
Accept: */*

HTTPS HOWTO

If you're interested in setting up https for your Mingle instance, you can read my HOWTO. I highly recommend this, why send your credentials in clear text when you don't have to? :)

mingle backup, restore, reset, migrate notes

Having spent some time on debugging a java/mingle issue recently, I document here some useful steps for dealing with backing up, restoring, resetting to default and migrating mingle.

My $MINGLE_DATA_DIR = /var/lib/mingle
My $MINGLE_INSTALL_DIR = /home/mingle/current-mingle

Backup mingle

1. Exported projects of interest manually via web UI
   For a lot of projects, alternatively use $MINGLE_INSTALL_DIR/tools/export_import/export_all_projects.rb
   Exports end up in $MINGLE_INSTALL_DIR/exported_projects
2. stop mingle
   $ invoke-rc.d mingled stop
3. backup /var/lib/mingle
   $ cp -a /var/lib/mingle{,~backup}
4. postgres dumpall
   $ sudo -u postgres -i -- pg_dumpall > ~/postgre-dumpall-pre-mingle-drop.dump

Reset mingle to near out of the box status

1. backup per above (mingle will be stopped)
2. $ rm -r /var/lib/mingle/*
3. $ sudo -u postgres -i -- psql -d mingle -c "DROP SCHEMA public CASCADE;"

This approach keeps the mingle db and credentials but drops everything else. You can now start mingle and go to the website to start the install process.

Mingle full purge and restore (out of the box)

1. backup per above (mingle will be stopped)
2. $ rm -r /var/lib/mingle/* $MINGLE_INSTALL_DIR
3. $ invoke-rc.d postgresql stop
4. $ mv -v /var/lib/postgresql/$VERSION{,~} && mv -v /etc/postgresql/$VERSION {,~}
5. $ pg_createcluster --start $VERSION main
6. check and diffs on the new cluster, including port, make a note of the db port
   $ diff -r /etc/postgresql/ $VERSION ~ /etc/postgresql/$VERSION
7. restore the db dump
   $ sudo -u postgres -i -- psql -f postgre-dumpall-pre-mingle-drop.dump postgres|less
   $ sudo -u postgres -i -- vacuumdb -a -z
8. extract mingle tgz
9. launch mingle

Server migration

Observe the following advice from Mingle team:

• complete the Mingle installation process using the new database settings completely, including creation of an initial user
• license the new instance
• create any users who are not associated with a project manually using Mingle as they will not be imported with the projects
• review the templates after the import has completed to remove any duplicates of the standard templates that may have been installed

This approach is aimed at a server that doesn't previously have an instance of postgres, or at least an instance that its data can be nuked by another servers full db dump.

1. backup per above (mingle will be stopped)
2. tgz and move your $MINGLE_DATA_DIR backup and psql dump to the new server
3. install postgres on the target server, note the port
4. restore the db dump, this will create users and the db
   $ sudo -u postgres -i -- psql -f postgre-dumpall-pre-mingle-drop.dump postgres|less
   $ sudo -u postgres -i -- vacuumdb -a -z
5. extract your $MINGLE_DATA_DIR , set permissions, set db port as required
6. extract mingle from its tgz to $MINGLE_INSTALL_DIR
7. launch mingle

Mingle shouldn't know any different, if the mingle doesn't start up normally, you've missed a step or something has gone wrong, check the logs!

Importing projects from the cli/shell

1. $ mkdir $MINGLE_INSTALL_DIR/exported_projects
2. place your .mingle files in aforementioned dir
3. run the importer script
   $ tools/run tools/export_import/import_projects.rb --mingle.dataDir=$MINGLE_DATA_DIR

bash fc - fix command cheatsheet

bash's BUILTIN fc command (fix command) is very useful and generally under-utilised. Here is a quick cheatsheet of some common fc commands which can be very useful indeed.

Personally it was hard to recall and decrypt the man bash fc section, so I wrote up a little cheatsheet for future reference, as follows.

$ fc -l 5680 5690

This will echo the commands in the shell history from 5680 5690. No execution.

$ fc -l -5 (minus the letter L and minus the number one)

This will echo the last 5 commands in the shell history, the same as history 5. No execution.

CAREFUL! The following commands require care, so that you don't inadvertently execute commands on your system! Please read how they work and how to handle them before trying them for yourself!

$ fc 5699 (careful!)

This opens the editor stored in FCEDIT, falling back on EDITOR. If neither env var is set, vi is used.
The editor will be populated with history line 5699 in this example.
When editing is complete, the commands are echoed and executed!
If you don't want to execute anything, ensure the file is empty and save and exit.

$ fc 5680 5690 (careful!)

Same as above but the editor will contain the commands within the shell history range.
If you don't want to execute anything, ensure the file is empty and save and exit.

$ fc -10 -1 (careful!)

Same as above but the editor will contain the last 10 commands in the shell history.
If you don't want to execute anything, ensure the file is empty and save and exit.

$ fc -s number|command (careful!)

Where number matches a shell history number OR command will match the the most recent shell history command that begins with command, then execute the command.

$ fc -s match=replacement number|command (careful!)

Same as above and each instance of match is replaced by replacement, then execute the command.

Feel free to checkout man bash to learn more in depth about this command including some options I didn't cover here.

ffmpeg and x264 under cygwin

x264

$ ./configure --enable-shared
$ make && make install && make install-lib-shared && make install-lib-dev

ffmpeg

$ ./configure --enable-gpl --enable-libx264 --extra-cflags="-I/usr/local/include" --extra-ldflags="-L/usr/local/lib" --enable-static --disable-shared
$ make && make install

A bash prompt with a visual indicator of the previous command exit code

Inspired by Lars Strand's blog post on the subject, I have taken the time to craft a solution for my .bashrc.

Feel free to check out the details which I shared on an existing thread on unix.stackexchange.

HOWTO KiTTY portable - cygwin support and import settings from PuTTY

SuperUser: Any simple way to add PuTTY settings to kitty?

PuTTYcyg FAQ: https://web.archive.org/web/20120619050045/http://code.google.com/p/puttycyg/wiki/FAQ

GTAIV command line arguments

One can use the commandline.txt with ths single line -help to get a pop up with the list of available options for launching GTAIV. I used SnagIt to capture the pop ups output.

This output is from GTAIV steam version 1.0.7.0 (the latest version as of writing).

LaunchGTAIV.exe [options]
Where [options] are one or more of:

==== [ Global ] ====
-adapter: Use the specified screen adapter number ?zero-based?
-disableimposters: Turn of the imposter rendering for vehicles
-noBlockOnLostFocus: Don't block the window update when it loses focus.

==== [AUDIO] ====
-fullspecaudio: Force high-end CPU audio footprint
-minspecaudio: Force low-end CPU audio footprint

==== [GLOBAL] ====
-benchmark: Determines if we run the benchmark immediately
-safemode: Sets graphics setting to lowest setting

==== [GRAPHICS] ====
-forcehighqualitymirrors: Enable 64 bit mirrors
-forcer2vb: force r2vb
-frameLimit: number of frames to limit game to
-framelockinwindow: Force framelock to work even in a window ?works best with
 60Hz monitor refresh?
-fullscreen: Force fullscreen mode
-gpucount: Lets you manually set the GPU count if query fails
-height:  Set height of main render window ?default is 480?
-managed: Use D3D runtime managed resources
-nominimize: Disables the ability to restore the game from minimize and
 changing resolutions - Reduces System Memory Footprint
-norestrictions: Do not limit graphics settings
-noswapdelay: Disable sleep delay before Present ?disable fix for hard Present
 stalls?
-novblank: Disable wait for vblank
-refreshrate: Set refresh rate of main render window
-stereo:  Set game to support stereo rendering mode
-unmanaged: Use application managed resources
-width:   Set width of main render window ?default is 640?
-windowed: Force windowed mode

==== [INPUT] ====
-usedirectinput: Allow DirectInput alongside XInput support.

==== [MEMORY] ====
-availablevidmem: Percentage of available video memory
-memrestrict: Set the restriction the amount of available memory for managed
 resources
-no_3GB:  Disable 32bit OS with /3GB
-nomemrestrict: Do not restrict the amount of available memory for managed
 resources
-noprecache: Do not precache resources
-percentvidmem: Percentage of video memory to make available to GTA
-reserve: Amount of memory to set aside for other applications
-reservedApp: Amount of memory to leave available within application space

==== [QUALITY SETTINGS] ====
-autoconfig: Automatically adjust quality setting to maintain desired frame
 rate ?15-120?
-detailquality: Set detail distance ?0-99?
-renderquality: Set anisotropic filtering ?0-4?
-shadowdensity: Set the number of lights that cast shadows
-shadowquality: Set the shadow quality ?0-4?
-texturequality: Set texture quality ?0-2?
-viewdistance: Set LOD view distance ?0-99?

==== [TIME] ====
-notimefix: Disable Time Fix

Props to:
mkey82 @ GTA Forums for publishing some great related info

Debian-Linux: start / stop / restart / status mingle init script

I have created and shared the following init script for Mingle 3.x

http://community.thoughtworks.com/posts/4c5c087474

Hope it helps :)

HOWTO: SSL your Mingle 3.5.x instance (including free SSL)

Following on from my self.documentation post on Java and its keytool, I shared a HOWTO on the mingle community for SSL'ing a Mingle instance.

HOWTO: SSL your Mingle 3.5.x instance (including free SSL)

Hope it helps someone and saves a little time!

Java Key and Certificate Management - Using an existing private key

problem: You want to use your existing private key and cert

It is not immediately obvious how to do this with Java's keytool. It took me a good while to figure out, so I'm documenting here.

solution: Java 6 can treat a PKCS12 file as a keystore

With Java 6 keytool, one can import a PKCS12 file as a keystore, here are the steps I used.

First, convert PEM format key/cert to a PKCS12 format cert

$ openssl pkcs12 -export -in thedomain.com.crt -inkey thedomain.com.key -out thedomain.com.p12

Then use keytool to import the PKCS12 cert

$ keytool -importkeystore -destkeystore thedomain.com.keystore -srckeystore thedomain.com.p12 -srcstoretype PKCS12 -alias 1

My understanding is, that if you have access to a version 6 keytool, then one can use the generated file keystore from 6 on older Java versions, but don't quote me on that.

Related: Generating a 2048 bit RSA private key and CSR

It worth mentioning that, if your just looking to generate a 2048 bit key/cert with keytool, to get a CSR for an SSL certificate authority, then you'd want to use something like this:

$ keytool -genkey -keyalg RSA -keysize 2048 -keystore thedomain.com.keystore -alias thealias

Then to get the CSR:

$ keytool -keyalg RSA -keystore thedomain.com.keystore -alias thealias -certreq -file thedomain.com.csr

Then when you get the cert back from the CA, import with keytool:

$ keytool -importcert -trustcacerts -keystore thedomain.com.keystore -alias thealias -file thedomain.com.crt

Free class 1 SSL

I have used these methods to get free SSL class 1 protection from http://www.startssl.com.

citation:

Props to:
NCSA CyberSecurity @ University of Illinois
Graham Leggett @ Cunning blog
Knowledge Base @ Comdo

Windows 7 taskbar pinned items stop working (after windows update?)

problem: After a few windows updates this week and the subsequent reboot, none of my taskbar pinned shortcuts or items worked!

The message was "Can't open this item" for each one of my pinned taskbar items.

The icons displayed just fine and the right click function worked as expected but launching the pinned items with left click gave the aforementioned message :(

Did I just get hit by a Microsoft's Patch Tuesday drive-by?

Feel free to skip right to the solution, the rest of the info is background/context.

Further reading

As a disclaimer, I have been getting weird issues with missing dll's of late, and have been forced to use the sfc /scannow command a few times in recent weeks. At first I thought this was after a few blue screens, due to playing with my RAM voltages, to ensure they were as low as possible... however now I'm wondering if there is a bigger issue somewhere.

Its possible that the so far unknown issue that has been causing these dll's to go missing or get corrupt, could of also hosed my taskbar items? I'm not sure, I would of imagined if I had an issue like that, things would be blowing up all over the place? It might not be related.

Worthy of note, a item in my Library also vanished at the same time as my pinned items... which was easily enough restored, but definitely odd.

In recent weeks, I've started to use Windows 7 Sleep function, e.g. at night when I have a lot open or if I know I'm walking away for a while. Since forever I've avoided sleep/suspend mode on windows because its always caused issues that need a reboot to fix. Perhaps sleeping is still a major no-no?

My main OS drive is also an SSD, so I'm wondering if there is something up with that, or perhaps SSD's don't like sleep mode, because up until now, the SSD has been working exactly as expected.

Oh the joys! I will post an update if anything worthwhile should arise on any of that.

solution: make shortcuts in \User Pinned\TaskBar

What I ended up doing to fix things, after not finding much help via the search engines:

1. Locating where the taskbar stuff is kept
   %USERPROFILE%\AppData\Roaming\Microsoft\Internet Explorer\Quick Launch\User Pinned\TaskBar
2. This location was empty for me :(
3. So I manually created program shortcuts with the same names as my pinned items.
4. Things started to magically work again.

To get the exact name of the pinned item, I SHIFT+RIGHT CLICKED a pinned item and chose properties, to see the name of the item. Then copy & pasted that as the name of the related item in the aforementioned path.

Hope that helps someone out!

sudo everything as anyone without a password prompt

problem: you can sudo as root without a password but not other users

You want to be able to sudo from any host, as any user (and/or group), any command, without being prompted for your password. Right now you don't get the pw prompt for root, but you do get prompted for other users (and/or groups).

This is in part, due to the most commonly known and distributed NOPASSWD sudoers cfg. Its easy to tweak it to give you NOPASSWD for _EVERYTHING_. I personally find the sudoers config and man very hard to follow. What about you? It was only through looking at other cfg's online and trial and error, that I was able to figure out the right syntax.

solution: update sudoers cfg

Its easy when you know how! Right?

#user    host    (user:group)    tag         cmd
kyle     ALL=    (ALL:ALL)       NOPASSWD:   ALL

In English "On any host, as any user or group, allow kyle to run any command without a password". I hope this proves useful again one day!

Are you loosing your angle brackets? - php and libxml2

problem: php processes some xml and your angle brackets in said xml vanish!

TLDR: use the CDATA tag to wrap your character data to avoid having angle brackets vanish.

I encountered this bug for the first time when I was importing a Cacti xml graph template via the Cacti web UI. On the surface everything seems to go well with the import process but then nothing was graphing and looking deeper, it was clear the config for the graph(s) was broken, due to missing < > angle brackets.
The brackets we're correctly encoded in the xml, it seems that somewhere between php and certain version of libxml the encoded angle brackets get stripped out.

Online there are a few bug reports but no single central bug id that I could find on this. One of the more useful shares online was a bug detail report for a closed google code project which provides Cacti mysql templates. Here is the bug detail, very useful info from Elan there.

During my search for solutions, it seemed likely that a bug was regressed or introduced in libxml, but that isn't certain. It would seem that the latest stable php 4.2 on Debian squeeze and libxml2 (as of writing 2.7.8.dfsg-2+squeeze3) still has the bug.

There is also some useful info on a bug report for MediaWiki project, entitled: Import strips angle brackets on some installations (libxml2 entity bug). To summarise, the consensus seems to be that its an upstream but with libxml2. The evidence I have found would agree with this.

Currently my systems are pinned on PHP 4.2 packages, perhaps this bug is not a factor in non pinned Debian squeeze systems?

impact: wastes time - fixing things that shouldn't really be broken.

solution: use the XML CDATA tag

I can provide is a way of easily checking if your set up has the bug or not. Props to Elan for this.

$ php -r '$p = xml_parser_create(); xml_parse_into_struct($p, "<path_php_binary>", $vals, $index); print_r($vals);'

A system suffering from the bug will include the output:

[value] => path_php_binary

A system NOT suffering from the bug will output:

[value] => <path_php_binary>

Now add the CDATA tag and see if the bug goes away? You don't even need to use entities when using the CDATA tag.

$ php -r '$p = xml_parser_create(); xml_parse_into_struct($p, "<![CDATA[<path_php_binary>]]>", $vals, $index); print_r($vals);'

citation:

Props to:
Elan Ruusamae for their bug detail report on the mysql cacti templates project.

testing for interactivity in a bash script

problem: you want to know if your bash script is being run interactively or not

This can be very useful to know, for example, if you want to programmatically determine if you should output to stdout, OR only to a log file because your script is running non-interactively.

When a script is running interactively, it means a stdout is present and one can output things to stdout and the user will see them.

When a script is running non-interactively, for example when cron runs a script, stdout does not exist.

solution: check if stdout is a tty

As with so many things, there is more than one solution, but this one seems to be rock solid and very portable.

is_interactive() { if [ -t 1 ]; then return 0; else return 1; fi }

As The Advanced Bash-Scripting Guide points out, for a complete test, one needs to check if stdout could be a socket too. Here is is the improved check:

is_interactive() { if [[ -t 1 || -p /dev/stdout ]]; then return 0; else return 1; fi }

I hope you enjoy and have fun tuning to your needs.
FYI. I tested the functions with cron and ssh on Debian squeeze (6.0.4) and bash 4.1.5.

citation:

Props to:
The Advanced Bash-Scripting Guide

How do I ask screen to behave like a standard shell?

I have found myself asking this question a few times, but never really got motivated to find a solid answer until recently.

I stole the question title from a good thread over at serverfault. Where I also contributed my findings.

I hope this share helps someone!

citation:

Props to:
rg3 @ LinuxQuestions.org for this share.

Windows 7 backup and restore error 0x80070020

problem: Windows 7 backup won't run or has stopped running

impact: Your stuff is not being backed up!

solution: Try creating a backup user or simply logging out

I had a user who's backup stopped working. After consulting with Dr. Google, non of the usual solutions would resolve the issue.

One person reported solving the issue by ensuring no users were logged in when the backup schedule ran, which makes sense, seeing as the error is related to some kind of file sharing violation.

This got me thinking, perhaps the usual user account had some processing running that were locking out some files. So I had the idea to create a backup user account, to see if the back up would run under that account. It did.

At the time of writing, I don't have any better suggestions, other than trying one of the above. It would seem that for my user, the error was related to a process running under their normal user account, locking out some files. Until that process is found and killed, it would seem that backing up while logged in as that account is not going to work.

P.S. Its worth noting, I was attempting to do a full backup for the computer in question.