Mingle MQL API by example

I was playing around with the Mingle API's today, some notes to self for future development...

An example of a raw GET request to Mingle's MQL API, notes:

• against Mingle 12.1
• URL encoding on the query string values

GET /api/v2/projects/YOUR_PROJECT/cards/execute_mql.xml?mql=SELECT%20number%2C%20name%20WHERE%20'Release'%20%3D%20(Next%20Release)%20AND%20Type%20%3D%20Feature HTTP/1.1
Host: YOUR_HOST:YOUR_PORT
User-Agent: curl/7.24.0 (i686-pc-cygwin) libcurl/7.24.0 OpenSSL/1.0.1c zlib/1.2.7 libidn/1.22 libssh2/1.4.0
Accept: */*
Cookie: mingle_12_1_2_session_id=SOME_SESSION; login=SOME_LOGIN

Note above how I'm hi-jacking an existing Mingle session to perform this request.

curl exmple

Here is a curl command and its resultant raw request, note I'm running this through a local debugging proxy, using https and basic authentication. I use Charles web debugging proxy, I am a huge advocate of this tool, and the use of any debug proxy in general :)

curl -k --proxy localhost:8888 'https://YOUR_USER:YOUR_PASS@YOUR_HOST:8080/api/v2/projects/YOUR_PROJECT/cards/execute_mql.xml?mql=SELECT%20number%2C%20name%20WHERE%20%27Release%27%20%3D%20%28Next%20Release%29%20AND%20Type%20%3D%20Feature'

In order for the basic authentication to work, you need to turn on Mingle's basic authentication, a simple change and its documented here. I restarted Mingle after this change.

The raw request from the curl

GET /api/v2/projects/YOUR_PROJECT/cards/execute_mql.xml?mql=SELECT%20number%2C%20name%20WHERE%20%27Release%27%20%3D%20%28Next%20Release%29%20AND%20Type%20%3D%20Feature HTTP/1.1
Authorization: Basic --REMOVED--
User-Agent: curl/7.24.0 (i686-pc-cygwin) libcurl/7.24.0 OpenSSL/1.0.1c zlib/1.2.7 libidn/1.22 libssh2/1.4.0
Host: YOUR_HOST:8080
Accept: */*

HTTPS HOWTO

If you're interested in setting up https for your Mingle instance, you can read my HOWTO. I highly recommend this, why send your credentials in clear text when you don't have to? :)

mingle backup, restore, reset, migrate notes

Having spent some time on debugging a java/mingle issue recently, I document here some useful steps for dealing with backing up, restoring, resetting to default and migrating mingle.

My $MINGLE_DATA_DIR = /var/lib/mingle
My $MINGLE_INSTALL_DIR = /home/mingle/current-mingle

Backup mingle

1. Exported projects of interest manually via web UI
   For a lot of projects, alternatively use $MINGLE_INSTALL_DIR/tools/export_import/export_all_projects.rb
   Exports end up in $MINGLE_INSTALL_DIR/exported_projects
2. stop mingle
   $ invoke-rc.d mingled stop
3. backup /var/lib/mingle
   $ cp -a /var/lib/mingle{,~backup}
4. postgres dumpall
   $ sudo -u postgres -i -- pg_dumpall > ~/postgre-dumpall-pre-mingle-drop.dump

Reset mingle to near out of the box status

1. backup per above (mingle will be stopped)
2. $ rm -r /var/lib/mingle/*
3. $ sudo -u postgres -i -- psql -d mingle -c "DROP SCHEMA public CASCADE;"

This approach keeps the mingle db and credentials but drops everything else. You can now start mingle and go to the website to start the install process.

Mingle full purge and restore (out of the box)

1. backup per above (mingle will be stopped)
2. $ rm -r /var/lib/mingle/* $MINGLE_INSTALL_DIR
3. $ invoke-rc.d postgresql stop
4. $ mv -v /var/lib/postgresql/$VERSION{,~} && mv -v /etc/postgresql/$VERSION {,~}
5. $ pg_createcluster --start $VERSION main
6. check and diffs on the new cluster, including port, make a note of the db port
   $ diff -r /etc/postgresql/ $VERSION ~ /etc/postgresql/$VERSION
7. restore the db dump
   $ sudo -u postgres -i -- psql -f postgre-dumpall-pre-mingle-drop.dump postgres|less
   $ sudo -u postgres -i -- vacuumdb -a -z
8. extract mingle tgz
9. launch mingle

Server migration

Observe the following advice from Mingle team:

• complete the Mingle installation process using the new database settings completely, including creation of an initial user
• license the new instance
• create any users who are not associated with a project manually using Mingle as they will not be imported with the projects
• review the templates after the import has completed to remove any duplicates of the standard templates that may have been installed

This approach is aimed at a server that doesn't previously have an instance of postgres, or at least an instance that its data can be nuked by another servers full db dump.

1. backup per above (mingle will be stopped)
2. tgz and move your $MINGLE_DATA_DIR backup and psql dump to the new server
3. install postgres on the target server, note the port
4. restore the db dump, this will create users and the db
   $ sudo -u postgres -i -- psql -f postgre-dumpall-pre-mingle-drop.dump postgres|less
   $ sudo -u postgres -i -- vacuumdb -a -z
5. extract your $MINGLE_DATA_DIR , set permissions, set db port as required
6. extract mingle from its tgz to $MINGLE_INSTALL_DIR
7. launch mingle

Mingle shouldn't know any different, if the mingle doesn't start up normally, you've missed a step or something has gone wrong, check the logs!

Importing projects from the cli/shell

1. $ mkdir $MINGLE_INSTALL_DIR/exported_projects
2. place your .mingle files in aforementioned dir
3. run the importer script
   $ tools/run tools/export_import/import_projects.rb --mingle.dataDir=$MINGLE_DATA_DIR

bash fc - fix command cheatsheet

bash's BUILTIN fc command (fix command) is very useful and generally under-utilised. Here is a quick cheatsheet of some common fc commands which can be very useful indeed.

Personally it was hard to recall and decrypt the man bash fc section, so I wrote up a little cheatsheet for future reference, as follows.

$ fc -l 5680 5690

This will echo the commands in the shell history from 5680 5690. No execution.

$ fc -l -5 (minus the letter L and minus the number one)

This will echo the last 5 commands in the shell history, the same as history 5. No execution.

CAREFUL! The following commands require care, so that you don't inadvertently execute commands on your system! Please read how they work and how to handle them before trying them for yourself!

$ fc 5699 (careful!)

This opens the editor stored in FCEDIT, falling back on EDITOR. If neither env var is set, vi is used.
The editor will be populated with history line 5699 in this example.
When editing is complete, the commands are echoed and executed!
If you don't want to execute anything, ensure the file is empty and save and exit.

$ fc 5680 5690 (careful!)

Same as above but the editor will contain the commands within the shell history range.
If you don't want to execute anything, ensure the file is empty and save and exit.

$ fc -10 -1 (careful!)

Same as above but the editor will contain the last 10 commands in the shell history.
If you don't want to execute anything, ensure the file is empty and save and exit.

$ fc -s number|command (careful!)

Where number matches a shell history number OR command will match the the most recent shell history command that begins with command, then execute the command.

$ fc -s match=replacement number|command (careful!)

Same as above and each instance of match is replaced by replacement, then execute the command.

Feel free to checkout man bash to learn more in depth about this command including some options I didn't cover here.

ffmpeg and x264 under cygwin

x264

$ ./configure --enable-shared
$ make && make install && make install-lib-shared && make install-lib-dev

ffmpeg

$ ./configure --enable-gpl --enable-libx264 --extra-cflags="-I/usr/local/include" --extra-ldflags="-L/usr/local/lib" --enable-static --disable-shared
$ make && make install