blocked/denied permissions on an NTFS folder which you don't own

problem: blocked/denied permissions on an NTFS folder which you don't own

impact: prevents you working with the folder and its contents

solution: use the SYSTEM account

some copy

psexec -s -i cmd

links: http://technet.microsoft.com/en-us/sysinternals/bb795533

sqlite3 + skype chat history + group chat history

Raw post, but very useful info for extracting messages from Skype chat SQLite databases. sqlite3.exe "$(cygpath 'C:\Users\kmilnes\Downloads\main-ml.db')"

Open a SQLite DB:

sqlite3 /path/to/your/main.db

Cygwin: sqlite3.exe "$(cygpath 'c:\Downloads\main.db')"

SHOW TABLES:

SELECT name FROM sqlite_master WHERE type='table' ORDER BY name;

DESCRIBE a table:

PRAGMA table_info(Conversations);

Schema (CREATE SYNTAX) for a table:

.schema TABLENAME

A list of unique usernames you've chatted with:

SELECT DISTINCT(dialog_partner) FROM Messages;

Show a list of messages for a given chat partner, ordered by timestamp:

SELECT author, from_dispname, datetime(timestamp, 'unixepoch') as date, body_xml FROM Messages where dialog_partner = 'your_dialog_partner' ORDER BY timestamp;

Show a unique list of participants from group chats you've had:

SELECT DISTINCT(participants) FROM Chats;

Get the chat id for a given group conversation:

SELECT id FROM Chats WHERE participants = 'participant1 participant2';

Get the conversation id for a given group conversation:

SELECT id FROM Conversations WHERE chat_dbid = 17627;

Get the messages for a given conversation id, ordered by timestamp:

SELECT author, from_dispname, datetime(timestamp, 'unixepoch') as date, body_xml FROM Messages where convo_id = 17631 ORDER BY timestamp;

Run shell script in a KiTTY cygterm (cygwin) from Notepad++

Approach 1 with helper script

"R:\REPOSITORY\FILES\Dropbox\linux\shared\home\XXXX\scripts\cygterm-launcher.bat" "$(FULL_CURRENT_PATH)"

helper script

for /f %%i in ('C:\cygwin\bin\cygpath %1') do set UnixPath=%%i

start "" "S:\Program Files\KiTTY\kitty_portable.exe" -cygterm -cmd "bash %UnixPath%" -

Approach 2 without helper script

c:\cygwin\bin\bash -c "_script=$(/usr/bin/cygpath '$(FULL_CURRENT_PATH)');  /cygdrive/s/Program\ Files/KiTTY/kitty_portable.exe -cygterm -cmd \"bash '$_script'\" -"

Note: sleuthkit Debian package

It contains istat and blkcat and friends.

Tools for forensics analysis (per Wikipedia):

The Sleuth Kit (TSK) is a library and collection of Unix- and Windows-based utilities for extracting data from disk drives and other storage so as to facilitate the forensic analysis of computer systems. It forms the foundation for Autopsy, a better known tool that is essentially a graphical user interface to the command line utilities bundled with The Sleuth Kit.

Mingle MQL API by example

I was playing around with the Mingle API's today, some notes to self for future development...

An example of a raw GET request to Mingle's MQL API, notes:

• against Mingle 12.1
• URL encoding on the query string values

GET /api/v2/projects/YOUR_PROJECT/cards/execute_mql.xml?mql=SELECT%20number%2C%20name%20WHERE%20'Release'%20%3D%20(Next%20Release)%20AND%20Type%20%3D%20Feature HTTP/1.1
Host: YOUR_HOST:YOUR_PORT
User-Agent: curl/7.24.0 (i686-pc-cygwin) libcurl/7.24.0 OpenSSL/1.0.1c zlib/1.2.7 libidn/1.22 libssh2/1.4.0
Accept: */*
Cookie: mingle_12_1_2_session_id=SOME_SESSION; login=SOME_LOGIN

Note above how I'm hi-jacking an existing Mingle session to perform this request.

curl exmple

Here is a curl command and its resultant raw request, note I'm running this through a local debugging proxy, using https and basic authentication. I use Charles web debugging proxy, I am a huge advocate of this tool, and the use of any debug proxy in general :)

curl -k --proxy localhost:8888 'https://YOUR_USER:YOUR_PASS@YOUR_HOST:8080/api/v2/projects/YOUR_PROJECT/cards/execute_mql.xml?mql=SELECT%20number%2C%20name%20WHERE%20%27Release%27%20%3D%20%28Next%20Release%29%20AND%20Type%20%3D%20Feature'

In order for the basic authentication to work, you need to turn on Mingle's basic authentication, a simple change and its documented here. I restarted Mingle after this change.

The raw request from the curl

GET /api/v2/projects/YOUR_PROJECT/cards/execute_mql.xml?mql=SELECT%20number%2C%20name%20WHERE%20%27Release%27%20%3D%20%28Next%20Release%29%20AND%20Type%20%3D%20Feature HTTP/1.1
Authorization: Basic --REMOVED--
User-Agent: curl/7.24.0 (i686-pc-cygwin) libcurl/7.24.0 OpenSSL/1.0.1c zlib/1.2.7 libidn/1.22 libssh2/1.4.0
Host: YOUR_HOST:8080
Accept: */*

HTTPS HOWTO

If you're interested in setting up https for your Mingle instance, you can read my HOWTO. I highly recommend this, why send your credentials in clear text when you don't have to? :)

mingle backup, restore, reset, migrate notes

Having spent some time on debugging a java/mingle issue recently, I document here some useful steps for dealing with backing up, restoring, resetting to default and migrating mingle.

My $MINGLE_DATA_DIR = /var/lib/mingle
My $MINGLE_INSTALL_DIR = /home/mingle/current-mingle

Backup mingle

1. Exported projects of interest manually via web UI
   For a lot of projects, alternatively use $MINGLE_INSTALL_DIR/tools/export_import/export_all_projects.rb
   Exports end up in $MINGLE_INSTALL_DIR/exported_projects
2. stop mingle
   $ invoke-rc.d mingled stop
3. backup /var/lib/mingle
   $ cp -a /var/lib/mingle{,~backup}
4. postgres dumpall
   $ sudo -u postgres -i -- pg_dumpall > ~/postgre-dumpall-pre-mingle-drop.dump

Reset mingle to near out of the box status

1. backup per above (mingle will be stopped)
2. $ rm -r /var/lib/mingle/*
3. $ sudo -u postgres -i -- psql -d mingle -c "DROP SCHEMA public CASCADE;"

This approach keeps the mingle db and credentials but drops everything else. You can now start mingle and go to the website to start the install process.

Mingle full purge and restore (out of the box)

1. backup per above (mingle will be stopped)
2. $ rm -r /var/lib/mingle/* $MINGLE_INSTALL_DIR
3. $ invoke-rc.d postgresql stop
4. $ mv -v /var/lib/postgresql/$VERSION{,~} && mv -v /etc/postgresql/$VERSION {,~}
5. $ pg_createcluster --start $VERSION main
6. check and diffs on the new cluster, including port, make a note of the db port
   $ diff -r /etc/postgresql/ $VERSION ~ /etc/postgresql/$VERSION
7. restore the db dump
   $ sudo -u postgres -i -- psql -f postgre-dumpall-pre-mingle-drop.dump postgres|less
   $ sudo -u postgres -i -- vacuumdb -a -z
8. extract mingle tgz
9. launch mingle

Server migration

Observe the following advice from Mingle team:

• complete the Mingle installation process using the new database settings completely, including creation of an initial user
• license the new instance
• create any users who are not associated with a project manually using Mingle as they will not be imported with the projects
• review the templates after the import has completed to remove any duplicates of the standard templates that may have been installed

This approach is aimed at a server that doesn't previously have an instance of postgres, or at least an instance that its data can be nuked by another servers full db dump.

1. backup per above (mingle will be stopped)
2. tgz and move your $MINGLE_DATA_DIR backup and psql dump to the new server
3. install postgres on the target server, note the port
4. restore the db dump, this will create users and the db
   $ sudo -u postgres -i -- psql -f postgre-dumpall-pre-mingle-drop.dump postgres|less
   $ sudo -u postgres -i -- vacuumdb -a -z
5. extract your $MINGLE_DATA_DIR , set permissions, set db port as required
6. extract mingle from its tgz to $MINGLE_INSTALL_DIR
7. launch mingle

Mingle shouldn't know any different, if the mingle doesn't start up normally, you've missed a step or something has gone wrong, check the logs!

Importing projects from the cli/shell

1. $ mkdir $MINGLE_INSTALL_DIR/exported_projects
2. place your .mingle files in aforementioned dir
3. run the importer script
   $ tools/run tools/export_import/import_projects.rb --mingle.dataDir=$MINGLE_DATA_DIR

bash fc - fix command cheatsheet

bash's BUILTIN fc command (fix command) is very useful and generally under-utilised. Here is a quick cheatsheet of some common fc commands which can be very useful indeed.

Personally it was hard to recall and decrypt the man bash fc section, so I wrote up a little cheatsheet for future reference, as follows.

$ fc -l 5680 5690

This will echo the commands in the shell history from 5680 5690. No execution.

$ fc -l -5 (minus the letter L and minus the number one)

This will echo the last 5 commands in the shell history, the same as history 5. No execution.

CAREFUL! The following commands require care, so that you don't inadvertently execute commands on your system! Please read how they work and how to handle them before trying them for yourself!

$ fc 5699 (careful!)

This opens the editor stored in FCEDIT, falling back on EDITOR. If neither env var is set, vi is used.
The editor will be populated with history line 5699 in this example.
When editing is complete, the commands are echoed and executed!
If you don't want to execute anything, ensure the file is empty and save and exit.

$ fc 5680 5690 (careful!)

Same as above but the editor will contain the commands within the shell history range.
If you don't want to execute anything, ensure the file is empty and save and exit.

$ fc -10 -1 (careful!)

Same as above but the editor will contain the last 10 commands in the shell history.
If you don't want to execute anything, ensure the file is empty and save and exit.

$ fc -s number|command (careful!)

Where number matches a shell history number OR command will match the the most recent shell history command that begins with command, then execute the command.

$ fc -s match=replacement number|command (careful!)

Same as above and each instance of match is replaced by replacement, then execute the command.

Feel free to checkout man bash to learn more in depth about this command including some options I didn't cover here.