Release Notes: Mitto Version 3.6.3¶
Breaking Changes¶
None.
Issues Resolved in this Release¶
Epic¶
Issue |
Summary |
Description |
|---|---|---|
Improvements to credentials handling in SQL job and Query job wizards |
The Runner user interface as well as associated Runner documentation has been improved to use only the term “Database URL” where “Connection URI”, “Database URI”, or DBO” may have been previously used. |
New Feature¶
Issue |
Summary |
Description |
|---|---|---|
Add timezones to Run Info times on Job Details Page |
The timezone associated with job start/stop times is now displayed in a job’s “Run Info”. |
|
Unified JSON logging format for Docker and Elastic ingestion |
Runner now has the ability to transmit logs from Docker services as well as individual Runner job logs to Zuar’s central ELK logging facility. |
|
[Improvements to credentials] Standardize Database URL Terminology |
The Runner user interface as well as associated Runner documentation has been improved to use only the term “Database URL” where “Connection URI”, “Database URI”, or DBO” may have been previously used. |
|
[Improvements to credentials] Implement Database_url Credential Type |
“Basic Database” credentials have been deprecated and replaced with “Database URL” credentials. When an instance is upgraded, any “Basic Database” credentials will be automatically converted to “Database URL” credentials. |
|
[Improvements to credentials] Query Job Wizard Database URL and Credentials separation |
The Query Job wizard has been redesigned to more clearly lead the user through the process of successfully creating a job. This includes support for the newly released Database URL credentials. |
|
[Improvements to credentials] SQL Job Wizard |
The SQL Job wizard has been redesigned to more clearly lead the user through the process of successfully creating a job. This includes support for the newly released Database URL credentials. |
Improvement¶
Issue |
Summary |
Description |
|---|---|---|
Webapp container - add |
The |
|
Job parameters to limit / disable sampling |
It is now possible to disable or limit data sampling in order to decrease runtime for jobs processing stable data. This is configurable via new new job parameter. |
|
allow compression of temporary JSON store to be configurable via job config |
It is now possible to compress intermediate JSON files used by jobs to reduce the disk space consumed by these files. This is configurable by a job parameter whose value defaults to “do not compress”. |
|
reword information displayed when new software is available |
The message displayed to a customer when checking if new software is available has been improved for clarity. |
|
Add ability to sort jobs in a squence on the sequence’s page |
It is now possible to sort jobs in a Sequence by “job”, “status”, or “last_completed”. |
|
some jobs in the “running” state aren’t running |
After restarting an instance, it was possible for a job not currently running to randomly display a status of “running”. Fixes for this ticket ensure that job status is always accurate after a restart. |
|
reduce load on backend caused by frequent UI updates |
On very old systems with millions of records in their job run history, the Job History page could become sluggish under certain circumstances. The new history implementation in this release ensure good response times regardless of the amount of job history. |
|
Prune tables that constantly grow and add indexes |
To improve performance, some database tables are now pruned according to schedule to remove unneeded data. Indexes have also been added to the tables to further improve performance. |
|
Add ability to limit length of sampling log messages |
To avoid extremely long log records, the data sampling implementation has been updated to provide control over the maximum length of values that are logged by sampling. This is controllable by a job configuration parameter. |
|
add ability to provide ‘params’ to |
It is now possible to POST small amounts of data to webhooks. The data is passed to the job being started by the webhook allowing control over the job’s behavior at runtime. |
|
Make it easier to page through job history |
It is now easier to page through job history. New functionality includes the ability to navigate directly to a specific page of the history. |
|
allow resizing of columns on Jobs page |
Column widths are now resizable, making it easier to view long job names and other values. |
|
improve UI paging behavior |
Paging has been improved to make it easier to page through job history. The ability to navigate directly to a specific page was added as well. |
|
Remove |
Because Debian package artifacts for Runner are no longer built, the |
|
Upgrade Postgres version |
Runner instances currently use Postgres version 13. Customer instances can now be upgraded to any newer Postgres version, including 18. This is a manual upgrade process that must be scheduled with Zuar’s DevOps group; it does not automatically take place during a regular software upgrade. |
|
Disable snowflake.sqlalchemy warning |
Disable unnecessary warning about requirements incompatibility. |
|
Ensure SetUpdatedAt Performs Upsert on Every RegexInput JOB Run |
Additional tests were added to ensure |
|
Add the ‘yarl’ package to Runner |
The ‘yarl’ package is now available for use in Runner service virtualenvs. |
|
Finish buildout of logging to ELK |
Runner now logs Docker services and Job logs to ELK. |
|
Improve integration of mitto-docs and mitto/docs; convert mitto/docs to reference |
Documentation in the |
|
JSON schema documentation for DatabaseURL credentials |
JSON Schema documentation for the new “Database URL” credentials was added to documentation. |
|
Change copyright in docs from “Zuar, Inc.” to “Zuar” |
Copyright updated to “Zuar”. |
Task¶
Issue |
Summary |
Description |
|---|---|---|
Updates to ‘docs/requirements.txt’ |
Updated dependencies to work properly and harmonize with the |
Bug¶
Issue |
Summary |
Description |
|---|---|---|
On Snowflake output, Postgres numeric column with decimal values as source, destination column rounds values to Integer |
The correct Snowflake type is now used when copying from Postrgres. |
|
CSV IO (regex) turning zero into a nonzero number |
Scientific notation for very small numbers is now handled correctly. |
|
Wrong link in email |
Email now contains the correct link. |
|
Error after Download folder |
Folders and their contents now download correctly. |
|
remove use of –extra-index-url |
Python packages now installed without the |
|
Error in the logs after running Job |
Remove error message incorrectly printed in job logs. |
|
Running jobs count is incorrect |
Running job count is now accurate. |
|
“Sequence Jobs” section of Sequence page does not show accurate status |
Run status of jobs in a sequence is now accurately represented. |
|
incorrect job “Save” and “Save and Start” behavior |
Behavior of “Save” and “Save and Start” has been substantially improved. Remaining improvements will be delivered via |
|
inconsistent scrolling in job log |
Erratic scrolling behavior has been corrected. |
|
Times at bottom of Swimlanes chart for 7 days, This Week, Last 30 days, and This Month are all zeroed out. |
Times displayed at the bottom of swimlanes have been corrected. |
|
Licensing “Last Sync” and “Next Sync” are empty |
Not reproducible. |
|
Job incorrectly appears in swimlane after instance upgrade while job is running |
Contents of swimlanes are now accurate. |
|
RegexInput no longer working with upserts |
Corrected a bug in |
|
Query wizard uses incorrect case for Snowflake table |
||
Available disk space incorrect when Runner installed on non-root device |
Runner now shows disk space for the volume on which it is installed, which may differ from the volume on which the OS is installed. |
|
Scheduled Jobs Occasionally Rerunning Exactly 1 Hour After Scheduled Run |
Corrected erroneous timing for some job runs. |
|
incorrect Runner upgrade behavior |
The upgrade behavior, as described, is working as intended. |
|
[Improvements to credentials] Fix credential file naming mismatch for database_url type |
Credentials file names are now updated as part of the migration to the Database URL credentials type. |
|
Convert Snowflake tests to get credentials using ‘zcreds’ instead of from ‘.mitto_test’ |
Credential used in CI now come from Vault. |
|
Convert Mariadb tests to get credentials using ‘zcreds’ instead of from ‘.mitto_test’ |
Credential used in CI now come from Vault. |
|
Convert BigQuery tests to get credentials using ‘zcreds’ instead of from ‘.mitto_test’ |
Credential used in CI now come from Vault. |
|
Convert SQL Server tests to get credentials using ‘zcreds’ instead of from ‘.mitto_test’ |
Credential used in CI now come from Vault. |
|
Convert redshift tests to get credentials using ‘zcreds’ instead of from ‘.mitto_test’ |
Credential used in CI now come from Vault. |
|
Configure git user.name and user.email for commits |
Added ability to provision a git username and email when automatically checking in job configuration changes. |
|
FE// “Confirm Mitto Restart” modal uses incorrect terminology (“Mitto” instead of “Runner”) |
||
Missing environment variables for zcreds – tests skipped or failed |