Abstract

Scientific analyses often consist of a number of tools that run one after the other, in order to go from the raw data to scientific insight. Between these specialized tools, simple data manipulation steps are often needed as a kind of "glue" between tools. For example, the output of tool A may produce a file that contains all the information needed as input for tool B, but tool B expects the columns in a different order. Or in genomic data analysis, some tools expect chromosome X to be listed as chrX, while others simply expect X. In these situations, extra data manipulation steps are needed to prepare files for input to analysis tools.

About This Material

This is a Hands-on Tutorial from the GTN which is usable either for individual self-study, or as a teaching material in a classroom.

Questions this will address

• How can I do basic data manipulation with JQ?
• Which functions are available to convert, reformat, filter, sort etc my JSON-based data?

Learning Objectives

• Familiarize yourself with data manipulation filters in JQ
• Perform basic data manipulation tasks in JQ
• Become comfortable converting JSON-based files in a variety of ways.
• Reason about the expected outcome of tools