Release notes¶

(version numbers can be interpreted as year.month.bug-fix-release-number)

Next release¶

Fixed broken tab-completion for names and ids: mq ls -i <tab> (MR: !132).

Version 23.1.0¶

Fixed a problem with dependencies inside subfolders (issue: #51).

Version 22.12.0¶

Added --extra-scheduler-args option to submit and resubmit commands.
Added special flag to node description.
Make sure old daemons from older versions stop running (MR: !130).

Version 22.11.3¶

Fixed dependency bug (MR: !128).

Version 22.11.2¶

Fix issue: #48 and other regressions after move to sqlite3.

Version 22.11.1¶

Add missing weight argument to myqueue.workflow.run().

Version 22.11.0¶

Important

No more <task-name>.state files. MyQueue will only know the state af a task if it is listed in your queue. There are two exceptions to this rule:

If a task is set to create some files like here:
```
def workflow():
    run(..., creates=['file1.abc', 'file2.xyz'], ...)
```
then MyQueue will consider the task done if those files exist. See myqueue.workflow.run().
If a task is a Python function like here:
```
def workflow():
    run(function=func, args=[...], name='abc', ...)
```
then MyQueue will consider the task done if the result file exists (in this case abc.result). See myqueue.caching.json_cached_function.

Your queue is no longer stored in a .myqueue/queue.json file. Instead, it is now in a sqlite3 file in .myqueue/queue.sqlite3. Your old JSON file will automatically be migrated to the new format.
Removed the mq run command (it may return later: issue: #44).
Calling a Python function from a workflow (run(function=...)) will now write the return value to a file called <task-name>.result in the JSON format. Previously the return value was written to the .state file.
Removing tasks part of a workflow now needs a --force (as MyQueue will no longer know the states of such tasks).
Most commands have been sped up by delaying import of rich and networkx.
The resubmit command will no longer remove the old task. Use --remove to get the old behavior.
The Resources of a task now includes a task-weight. This can be used to limit the number of running tasks. See more here: Task weight.

Version 22.9.0¶

Hitting CTRL-C in the middle of submitting jobs is now safe.

Version 22.7.1¶

Fixed: mq list does not work with ID specifier (-i).

Version 22.7.0¶

Tasks will no longer activate a virtual environment if a venv/ folder is found in one of the parent folders.
Tasks submitted from an activated virtual environment will now activate that environment when the job starts running.
Better error message when sbatch/qsub/bsub fails.
Improved parsing of stderr from failed jobs.
Depth first submit ordering. A workflow with an A task and a B task where B depends on A would previously run all the A tasks and then all the B tasks. The order will now be A, B in the first folder, then A, B in the next folder and so on.

Version 22.6.0¶

Fixed bug related to several users having write access to the same .myqueue/ folder.

Version 22.3.0¶

There is now one background daemon per user. This will allow several users to share a .myqueue/ folder.

Version 22.1.0¶

The List (ls): List tasks in queue command can now list several folders instead of, as previously, only one. They must all belong to the same .myqueue/ folder though.

Version 21.8.0¶

The simple “local” scheduler is now feature complete. See Name of scheduler.
The mpi_implementations configuration option is no longer needed and has been deprecated.
MyQueue no longer tries to keep track of all your .myqueue/ folders. Consequently, the --all option has been removed from the list, kick and sync commands.
There is a new mq info --all [folder] command that will searsch for your .myqueue/ folders and print a status line for each.
There is now one background daemon per .myqueue/ folder. See The daemon background process.

Version 21.7.0¶

Email notifications: mq modify ... -N dA. See Modify: Modify task(s) and Notifications.
You can now use mq info to get information about your MyQueue installation:
- version
- location of the source code
- location of .myqueue/ folder
- configuration

Version 21.4.2¶

Make things work with Python 3.7.

Version 21.4.1¶

Backwards compatibility fix.

Version 21.4.0¶

For workflow tasks, name.done and name.FAILED files have now been replaced by a name.state file. MyQueue will still read the old files, but no longer write them.

Version 21.2.0¶

PRELIMINARY: New way to specify workflows using myqueue.workflow.run(), myqueue.workflow.wrap() and myqueue.workflow.resources(). See Workflow script.

Version 21.1.0¶

New config command for guessing your configuration. See Guessing your configuration.
LSF-backend fixes.

Version 20.11.3¶

Bugfix: LSF-backend fixes.

Version 20.11.2¶

Bugfix: Don’t remove FAILED-files in dry-run mode.

Version 20.11.1¶

Fix “workflow target” bug and MQ: comments bug.

Version 20.11.0¶

New mq workflow ... --arguments "key=val,..." option. See Workflow: Submit tasks from Python script or several scripts matching pattern.
Two new columns in list output: arguments and info. Can be hidden with: mq ls -c aI-.
Deprecated venv/activate script. Use venv/bin/activate instead. See Using a Python virtual environment.
Resources can now be specified in the scripts as special comments:
```
# MQ: resources=24:2h
```

Version 20.9.1¶

Fix workflow+openmpi issue.

Version 20.9.0¶

Red error messages.
Progress-bar.

Version 20.5.0¶

Using pytest for testing.
Simple local queue for use without a real scheduler.
New extra_args configuration parameter (Extra arguments for submit command). Replaces, now deprecated, features and reservation parameters.
Use python3 -m myqueue.config to auto-configure your system.
Memory usage is now logged.

Version 20.1.2¶

Bug-fix release with fix for single-process tasks (see Resources).

Version 20.1.1¶

This is the version submitted to JOSS.

Version 20.1.0¶

New shortcuts introduced for specifying States: a is qhrd and A is FCMT.

Version 19.11.1¶

New command: Daemon: Interact with the background process.

Version 19.11.0¶

Small bugfixes and improvements.

Version 19.10.1¶

Added support for LSF scheduler.
Added --max-tasks option for submit and workflow commands.

Version 19.10.0¶

Shell-style wildcard matching of task names and error messages is now possible:
```
$ mq ls -n "*abc-??.py"
$ mq resubmit -s F -e "*ZeroDivision*"
```
Three new Command-line interface options: mq -V/--version, mq ls --not-recursive and mq submit/workflow -f/--force.
All task-events (queued, running, stopped) are now logged to ~/.myqueue/log.csv. List tasks from log-file with:
```
$ mq ls -L ...
```

Version 19.9.0¶

New -C option for the mq ls command for showing only the count of tasks in the queue:
```
$ mq ls -C
running: 12, queued: 3, FAILED: 1, total: 16
```
A background process will now automatically kick your queues every ten minutes.
Project moved to a new myqueue group: https://gitlab.com/myqueue/myqueue/

Version 19.8.0¶

The module:function syntax has been changed to module@function.

Arguments to tasks are now specified like this:

$ mq submit [options] "<task> arg1 arg2 ..." [folder1 [folder2 ...]]

New run command:

$ mq run [options] "<task> arg1 arg2 ..." [folder1 [folder2 ...]]

Version 19.6.0¶

Tasks will now activate a virtual environment if a venv/ folder is found in one of the parent folders. The activation script will be venv/activate or venv/bin/activate if venv/activate does not exist.

Version 19.5.0¶

New --target option for workflows.
New API’s for submitting jobs: myqueue.task.Task.submit() and myqueue.submit().
New --name option for the submit command.

No more --arguments option. Use:

$ mq submit [options] <task> [folder1 [folder2 ...]] -- arg1 arg2 ...

Version 19.2.0¶

Fix test-suite.

Version 19.1.0¶

Recognizes mpiexex variant automatically.
New “detailed information” subcommand.

Version 18.12.0¶

The restart parameter is now an integer (number of restarts) that counts down to zero. Avoids infinite loop.

Version 0.1.0¶

Initial release.