Automate Bash, R, and Python Scripts With Mitto

Automate Bash, R, and Python Scripts With Mitto

In this blog post we will discuss the Mitto command line job, and then walk through examples of automating scripts from various languages:

  • Bash
  • R
  • Python

With Zuar's Mitto platform, you can automate your ELT/ETL processes and have data flowing from hundreds of potential sources into a single destination. Transport, warehouse, transform, model, report, and monitor: it's all managed by Mitto.

Related how-to video:

Mitto Command Line Jobs

A powerful feature in Mitto is the ability to automate commands and scripts with command line jobs.

These "cmd" jobs allow Mitto users to run any command on the shell of their Mitto instance.

An example might be a job that looks like the following:

    "cmd": "echo 'hello world' > /var/mitto/data/hello_world.txt",
    "cmd_env": {},
    "exec": false,
    "shell": true

Instructions for creating a command line job in Mitto can be found here.

In our example case, title the job [cmd] Hello World!, add the JSON configuration above, and click "Done". On the newly created job's page, click the little pencil to edit the job type. Select cmd from the select menu.

Running this job will create a text file at: /var/mitto/data/hello_world.txt.

You can verify it's there by looking on the Files page in Mitto (clicking on Files in the menu on the left).

Here are the contents of the hello_world.txt:

Bash Scripts With Mitto

Alternatively, you can upload and run a bash script in your Mitto command jobs.

Bash, the most commonly available unix shell, is

the shell, or command language interpreter, for the GNU operating system.

Bash commands can be saved in a file and executed as a script.

The following Bash script ( creates a CSV file: written_by_bash.csv


echo "id, name" > /var/mitto/data/written_by_bash.csv
echo "1, Allen" >> /var/mitto/data/written_by_bash.csv
echo "2, Steve" >> /var/mitto/data/written_by_bash.csv

Upload the Bash script to Mitto via the file manager.

Then create and run the following command job. Once again, the job type needs to be set to cmd.

    "cmd": "bash /var/mitto/data/",
    "cmd_env": {},
    "exec": false,
    "shell": true

The resulting file should look like this:

R Scripts With Mitto

You can also automate R scripts with Mitto.

R is a language and environment for statistical computing and graphics.

The following script (test_r_script) will create a tab separated file (written_by_r.tsv) with the numbers 1-5 on the first line and the numbers 6-10 on the second line.

x <- matrix(1:10, ncol = 5)

fil <- tempfile("data")
write(t(x), fil)

write(x, file = "/var/mitto/data/written_by_r.tsv", sep = "\t")

Upload the R script to Mitto via the file manager.

Create another cmd job (HINT: you can duplicate an existing job) and edit the job's config:

    "cmd": "Rscript /var/mitto/data/test_r_script.R",
    "cmd_env": {},
    "exec": false,
    "shell": true

Finally, on the main job page, change the job type from io to cmd.

Running the job will create a file at: /var/mitto/data/written_by_r.tsv. You can verify this file exists in the file editor. The resulting file should look like this:

Python Scripts With Mitto

Automating a Python script is a similar process in Mitto with a command job.

"Python is an interpreted, high-level, general-purpose programming language."

The following script ( will create a CSV: written_by_python.csv.

#!/usr/bin/env python3
"""NOTE: You must use python3"""
f = open("/var/mitto/data/written_by_python.csv", "w")
f.write("id, name\n1, Allen\n2, Steve\n")

One thing to note is that #!/usr/bin/env python won't work, as Mitto has python3 installed.

Upload the Python script to Mitto via the file manager.

Again, we'll create a simple job with the following JSON config, and we'll be sure to change the job type from io to cmd.

    "cmd": "python3 /var/mitto/data/",
    "cmd_env": {},
    "exec": false,
    "shell": true

The resulting file should look like this:

Learn more about automating Python scripts with Mitto.


The script examples covered above were extremely simple, however they show the power of automating scripts in Mitto with the command line job.

While the examples above cover a lot of use cases, they only touch the surface of the Mitto command line job.

Some other command line interface utilities Mitto users use include:

While Mitto has an extensive (and growing!) plugin library to facilitate many automation tasks out of the box, command line jobs enable many more advanced automation use cases.

To close, here are a few helpful Mitto-related links:

Anatomy of a Mitto IO job | Zuar
This blog post is part 1 of a multi part blog series on understanding andcustomizing Mitto IO jobs. Mitto [] comes with many different plugins[…
Getting Started with Mitto | Zuar
The video tutorials below are for those looking to get started using Mitto orlearn more about the available features. Start with the following topics: * Mitto UI Tour * Creating IO (Input/Output) Jobs * Creating SQL Jobs * Creating Sequences * Creating Schedules * Adding Tags * Creating Not…
Python vs. Java: Uses, Performance, Learning
In the worlds of computer science and data science, there are many programminglanguages, and no single language is superior to another. In other words, eachlanguage is best suited to solve certain problems, and in fact there is often noone best language to choose for a given programming project. …