It's inevitable that if you deal with enough data, especially exporting data from source systems, that you will eventually be required to deal with a .zip, .gz, or tar.gz file.
To get this into a database, you will likely need to unzip the file and use the decompressed files inside. The following steps will guide you through the process of unzipping a compressed file in MITTO, saving the decompressed files in MITTO, and preparing them to be loaded into a database.
- Make sure that the unzip utility is installed on the Mitto box
sudo apt-get install unzip. If you are unsure how to SSH to your MITTO instance, or your instance is being hosted by Zuar, please file a ticket here! - Create a command line job
unzipwith the following code:
{
"cmd": "unzip /var/mitto/data/_filename_.zip -d /var/mitto/data",
"cmd_env": {},
"exec": false,
"shell": true
}
The command is unzip follow by the location of the zip file in MITTO /var/mitto/data/filename.zip , ending with the destination to place the extracted files -d /var/mitto/data (MITTO's data folder).
These commands can be adjusted to target .gz files and tar.gz files, or gunzip or tar -xf filename.tar.gz respectively.
What if you want to unzip multiple files or create a wildcard match? Here is how to search for multiple files that are named something like data_2018.zip, data_2019.zip, data_2020.zip, data_2021.zip:
{
"cmd": "unzip /var/mitto/data/'data_*.zip' -d /var/mitto/data",
"cmd_env": {},
"exec": false,
"shell": true
}
Lastly, what if you need to automate this as the data continues to change and you must overwrite old files that are being decompressed? Add a simple overwrite command -o after the unzip command. The cmd line will then look like:
"cmd": "unzip -o /var/mitto/data/'data_*.zip' -d /var/mitto/data",
All together, the final solution will look like:
{
"cmd": "unzip -o /var/mitto/data/'data_*.zip' -d /var/mitto/data",
"cmd_env": {},
"exec": false,
"shell": true
}
This command will unzip all files that start with data_ and end with .zip , place them in MITTO's data folder and overwrite existing files with the same name.
