Mingle MQL API by example

I was playing around with the Mingle API's today, some notes to self for future development...

An example of a raw GET request to Mingle's MQL API, notes:

• against Mingle 12.1
• URL encoding on the query string values

GET /api/v2/projects/YOUR_PROJECT/cards/execute_mql.xml?mql=SELECT%20number%2C%20name%20WHERE%20'Release'%20%3D%20(Next%20Release)%20AND%20Type%20%3D%20Feature HTTP/1.1
Host: YOUR_HOST:YOUR_PORT
User-Agent: curl/7.24.0 (i686-pc-cygwin) libcurl/7.24.0 OpenSSL/1.0.1c zlib/1.2.7 libidn/1.22 libssh2/1.4.0
Accept: */*
Cookie: mingle_12_1_2_session_id=SOME_SESSION; login=SOME_LOGIN

Note above how I'm hi-jacking an existing Mingle session to perform this request.

curl exmple

Here is a curl command and its resultant raw request, note I'm running this through a local debugging proxy, using https and basic authentication. I use Charles web debugging proxy, I am a huge advocate of this tool, and the use of any debug proxy in general :)

curl -k --proxy localhost:8888 'https://YOUR_USER:YOUR_PASS@YOUR_HOST:8080/api/v2/projects/YOUR_PROJECT/cards/execute_mql.xml?mql=SELECT%20number%2C%20name%20WHERE%20%27Release%27%20%3D%20%28Next%20Release%29%20AND%20Type%20%3D%20Feature'

In order for the basic authentication to work, you need to turn on Mingle's basic authentication, a simple change and its documented here. I restarted Mingle after this change.

The raw request from the curl

GET /api/v2/projects/YOUR_PROJECT/cards/execute_mql.xml?mql=SELECT%20number%2C%20name%20WHERE%20%27Release%27%20%3D%20%28Next%20Release%29%20AND%20Type%20%3D%20Feature HTTP/1.1
Authorization: Basic --REMOVED--
User-Agent: curl/7.24.0 (i686-pc-cygwin) libcurl/7.24.0 OpenSSL/1.0.1c zlib/1.2.7 libidn/1.22 libssh2/1.4.0
Host: YOUR_HOST:8080
Accept: */*

HTTPS HOWTO

If you're interested in setting up https for your Mingle instance, you can read my HOWTO. I highly recommend this, why send your credentials in clear text when you don't have to? :)

mingle backup, restore, reset, migrate notes

Having spent some time on debugging a java/mingle issue recently, I document here some useful steps for dealing with backing up, restoring, resetting to default and migrating mingle.

My $MINGLE_DATA_DIR = /var/lib/mingle
My $MINGLE_INSTALL_DIR = /home/mingle/current-mingle

Backup mingle

1. Exported projects of interest manually via web UI
   For a lot of projects, alternatively use $MINGLE_INSTALL_DIR/tools/export_import/export_all_projects.rb
   Exports end up in $MINGLE_INSTALL_DIR/exported_projects
2. stop mingle
   $ invoke-rc.d mingled stop
3. backup /var/lib/mingle
   $ cp -a /var/lib/mingle{,~backup}
4. postgres dumpall
   $ sudo -u postgres -i -- pg_dumpall > ~/postgre-dumpall-pre-mingle-drop.dump

Reset mingle to near out of the box status

1. backup per above (mingle will be stopped)
2. $ rm -r /var/lib/mingle/*
3. $ sudo -u postgres -i -- psql -d mingle -c "DROP SCHEMA public CASCADE;"

This approach keeps the mingle db and credentials but drops everything else. You can now start mingle and go to the website to start the install process.

Mingle full purge and restore (out of the box)

1. backup per above (mingle will be stopped)
2. $ rm -r /var/lib/mingle/* $MINGLE_INSTALL_DIR
3. $ invoke-rc.d postgresql stop
4. $ mv -v /var/lib/postgresql/$VERSION{,~} && mv -v /etc/postgresql/$VERSION {,~}
5. $ pg_createcluster --start $VERSION main
6. check and diffs on the new cluster, including port, make a note of the db port
   $ diff -r /etc/postgresql/ $VERSION ~ /etc/postgresql/$VERSION
7. restore the db dump
   $ sudo -u postgres -i -- psql -f postgre-dumpall-pre-mingle-drop.dump postgres|less
   $ sudo -u postgres -i -- vacuumdb -a -z
8. extract mingle tgz
9. launch mingle

Server migration

Observe the following advice from Mingle team:

• complete the Mingle installation process using the new database settings completely, including creation of an initial user
• license the new instance
• create any users who are not associated with a project manually using Mingle as they will not be imported with the projects
• review the templates after the import has completed to remove any duplicates of the standard templates that may have been installed

This approach is aimed at a server that doesn't previously have an instance of postgres, or at least an instance that its data can be nuked by another servers full db dump.

1. backup per above (mingle will be stopped)
2. tgz and move your $MINGLE_DATA_DIR backup and psql dump to the new server
3. install postgres on the target server, note the port
4. restore the db dump, this will create users and the db
   $ sudo -u postgres -i -- psql -f postgre-dumpall-pre-mingle-drop.dump postgres|less
   $ sudo -u postgres -i -- vacuumdb -a -z
5. extract your $MINGLE_DATA_DIR , set permissions, set db port as required
6. extract mingle from its tgz to $MINGLE_INSTALL_DIR
7. launch mingle

Mingle shouldn't know any different, if the mingle doesn't start up normally, you've missed a step or something has gone wrong, check the logs!

Importing projects from the cli/shell

1. $ mkdir $MINGLE_INSTALL_DIR/exported_projects
2. place your .mingle files in aforementioned dir
3. run the importer script
   $ tools/run tools/export_import/import_projects.rb --mingle.dataDir=$MINGLE_DATA_DIR

bash fc - fix command cheatsheet

bash's BUILTIN fc command (fix command) is very useful and generally under-utilised. Here is a quick cheatsheet of some common fc commands which can be very useful indeed.

Personally it was hard to recall and decrypt the man bash fc section, so I wrote up a little cheatsheet for future reference, as follows.

$ fc -l 5680 5690

This will echo the commands in the shell history from 5680 5690. No execution.

$ fc -l -5 (minus the letter L and minus the number one)

This will echo the last 5 commands in the shell history, the same as history 5. No execution.

CAREFUL! The following commands require care, so that you don't inadvertently execute commands on your system! Please read how they work and how to handle them before trying them for yourself!

$ fc 5699 (careful!)

This opens the editor stored in FCEDIT, falling back on EDITOR. If neither env var is set, vi is used.
The editor will be populated with history line 5699 in this example.
When editing is complete, the commands are echoed and executed!
If you don't want to execute anything, ensure the file is empty and save and exit.

$ fc 5680 5690 (careful!)

Same as above but the editor will contain the commands within the shell history range.
If you don't want to execute anything, ensure the file is empty and save and exit.

$ fc -10 -1 (careful!)

Same as above but the editor will contain the last 10 commands in the shell history.
If you don't want to execute anything, ensure the file is empty and save and exit.

$ fc -s number|command (careful!)

Where number matches a shell history number OR command will match the the most recent shell history command that begins with command, then execute the command.

$ fc -s match=replacement number|command (careful!)

Same as above and each instance of match is replaced by replacement, then execute the command.

Feel free to checkout man bash to learn more in depth about this command including some options I didn't cover here.

ffmpeg and x264 under cygwin

x264

$ ./configure --enable-shared
$ make && make install && make install-lib-shared && make install-lib-dev

ffmpeg

$ ./configure --enable-gpl --enable-libx264 --extra-cflags="-I/usr/local/include" --extra-ldflags="-L/usr/local/lib" --enable-static --disable-shared
$ make && make install

A bash prompt with a visual indicator of the previous command exit code

Inspired by Lars Strand's blog post on the subject, I have taken the time to craft a solution for my .bashrc.

Feel free to check out the details which I shared on an existing thread on unix.stackexchange.

HOWTO KiTTY portable - cygwin support and import settings from PuTTY

SuperUser: Any simple way to add PuTTY settings to kitty?

PuTTYcyg FAQ: https://web.archive.org/web/20120619050045/http://code.google.com/p/puttycyg/wiki/FAQ

GTAIV command line arguments

One can use the commandline.txt with ths single line -help to get a pop up with the list of available options for launching GTAIV. I used SnagIt to capture the pop ups output.

This output is from GTAIV steam version 1.0.7.0 (the latest version as of writing).

LaunchGTAIV.exe [options]
Where [options] are one or more of:

==== [ Global ] ====
-adapter: Use the specified screen adapter number ?zero-based?
-disableimposters: Turn of the imposter rendering for vehicles
-noBlockOnLostFocus: Don't block the window update when it loses focus.

==== [AUDIO] ====
-fullspecaudio: Force high-end CPU audio footprint
-minspecaudio: Force low-end CPU audio footprint

==== [GLOBAL] ====
-benchmark: Determines if we run the benchmark immediately
-safemode: Sets graphics setting to lowest setting

==== [GRAPHICS] ====
-forcehighqualitymirrors: Enable 64 bit mirrors
-forcer2vb: force r2vb
-frameLimit: number of frames to limit game to
-framelockinwindow: Force framelock to work even in a window ?works best with
 60Hz monitor refresh?
-fullscreen: Force fullscreen mode
-gpucount: Lets you manually set the GPU count if query fails
-height:  Set height of main render window ?default is 480?
-managed: Use D3D runtime managed resources
-nominimize: Disables the ability to restore the game from minimize and
 changing resolutions - Reduces System Memory Footprint
-norestrictions: Do not limit graphics settings
-noswapdelay: Disable sleep delay before Present ?disable fix for hard Present
 stalls?
-novblank: Disable wait for vblank
-refreshrate: Set refresh rate of main render window
-stereo:  Set game to support stereo rendering mode
-unmanaged: Use application managed resources
-width:   Set width of main render window ?default is 640?
-windowed: Force windowed mode

==== [INPUT] ====
-usedirectinput: Allow DirectInput alongside XInput support.

==== [MEMORY] ====
-availablevidmem: Percentage of available video memory
-memrestrict: Set the restriction the amount of available memory for managed
 resources
-no_3GB:  Disable 32bit OS with /3GB
-nomemrestrict: Do not restrict the amount of available memory for managed
 resources
-noprecache: Do not precache resources
-percentvidmem: Percentage of video memory to make available to GTA
-reserve: Amount of memory to set aside for other applications
-reservedApp: Amount of memory to leave available within application space

==== [QUALITY SETTINGS] ====
-autoconfig: Automatically adjust quality setting to maintain desired frame
 rate ?15-120?
-detailquality: Set detail distance ?0-99?
-renderquality: Set anisotropic filtering ?0-4?
-shadowdensity: Set the number of lights that cast shadows
-shadowquality: Set the shadow quality ?0-4?
-texturequality: Set texture quality ?0-2?
-viewdistance: Set LOD view distance ?0-99?

==== [TIME] ====
-notimefix: Disable Time Fix

Props to:
mkey82 @ GTA Forums for publishing some great related info