In this News and Views episode, Phil Ewels talks to Ben Sherman about upcoming features in core Nextflow development: a new CLI v2 interface; submission of job arrays and intermediate file cleanup during runs.

CLI v2

• The new command-line interface will be invoked with the nf command
• It will co-exist with the current nextflow CLI, at least for a while
• Expect argument / option parsing more in line with linux norms
  • No longer a single-dash for core flags and double-dash for pipeline parameters
  • Single dash for short flags, double for long-form flags
  • Core and pipeline options separated with --

See the pull-request here: Add CLI v2 #3600

Job Arrays

• Soon you’ll be able to submit job arrays to HPC and cloud
• Group the submission of any number (10s-1000s) of jobs
• Still independent jobs, but submitted in one go

See the pull-request here: Implement job array using TaskRun as carrier #3905

Intermediate file cleanup

• New ability to track which intermediate files are no longer needed
• Allows files to be deleted from the work directory during a run
• Reduces the peak storage required for large pipeline runs

See the pull-request here: Add support for temporary output paths #3818