Hydra← All projects

QA metrics

Quality

Production Releases
N/A
Production Defect Leakage
N/A
Average Defect Fix Time
N/A

Test Coverage

1 week2 weeks1 month2 months

Automation Coverage

MusesAcademy Product - Total test case automation coverage by platform · trend over 30 days

All tests

Automated share of all test cases
All platforms
59.76%
59.8%55.6%
07/1023/10
245 of 410 automated
mobile
50.79%
52.4%42.9%
07/1023/10
97 of 191 automated
web
67.58%
68.1%65.5%
07/1023/10
148 of 219 automated

Automatable tests

Coverage counting only tests tagged “automatable”
All platforms
62.76%
63.2%61.1%
07/1023/10
214 of 341 automated
mobile
51.23%
53.2%46.1%
07/1023/10
83 of 162 automated
web
73.18%
76.2%70.4%
07/1023/10
131 of 179 automated

Split by Testomat tag — cases without a platform tag aren't counted. “mobile” covers both iOS and Android.

Test Cases Traction

Draft backlog

Drafts: tagged “draft” now, however old · trend over 30 days
Tests in draft
60
6024
07/1023/10
24 older than 7 days · 0 older than 30 days · oldest 20 days

Updated existing test cases

Share of tests created over 7 days ago, not tagged “draft”, that were edited since · trend over 30 days
All tests
22.91%
53.2%0.4%
07/1023/10
104 of 454 updated, last 7 days

Updated = Testomat's last-edit time falls in the window, so a test edited twice counts once. Changes from an import from code count too.

Out of sync

Automated tests edited in Testomat since their last import from code · trend over 30 days
Automated tests
6
63
07/1023/10
of 262 automated, out of sync with the code

Testomat's own “Out of sync”: the code still has the old version until the test is reimported from code or the edit is rejected in Testomat.

Test Execution

1 week2 weeks1 month2 months

Test runs

MusesAcademy Product - Finished runs labelled web or mobilePlatform (iOS / Android) · last 7 days, trend per 24 h

Runs and success rate

Success rate = passed of the tests that ran in finished runs; skipped tests didn't run and are left out
Runs
All platforms
26
101
17/1023/10
last 7 days
Android
0
00
17/1023/10
last 7 days
iOS
2
10
17/1023/10
last 7 days
web
24
91
17/1023/10
last 7 days
Success rate
All platforms
92.0%
100.0%49.3%
17/1023/10
2,869 of 3,120 tests passed, 2 skipped, last 7 days
Android
—
No tests ran in the last 7 days
iOS
90.1%
93.5%86.3%
20/10 10:2621/10 10:26
155 of 172 tests passed, 2 skipped, last 7 days
web
92.1%
100.0%49.3%
17/1023/10
2,714 of 2,948 tests passed, last 7 days

Flaky rate

Flaky = failed, then passed on a retry, of the tests that ran in finished runs
All platforms
0.35%
0.6%0.0%
17/1023/10
11 of 3,120 tests passed only on a retry, last 7 days
Android
—
No tests ran in the last 7 days
iOS
0.00%
0.0%0.0%
20/10 10:2621/10 10:26
0 of 172 tests passed only on a retry, last 7 days
web
0.37%
0.7%0.0%
17/1023/10
11 of 2,948 tests passed only on a retry, last 7 days

Testomat keeps one result per test even when it was retried: one that failed and then passed is marked “Success after retry” and counts as passed in the success rate too. A suite that doesn't retry failed tests never shows any.

Average run duration

Passed and failed runs labelled Regression / Smoke, launch to finish
regression
Android
—
No runs in the last 7 days
iOS
54.9 min
55.554.2
20/10 10:2621/10 10:26
2 runs, last 7 days
web
10.7 min
12.29.6
17/1023/10
16 runs, last 7 days
smoke
Android
—
No runs in the last 7 days
iOS
—
No runs in the last 7 days
web
12.3 min
216.1
20/10 10:2623/10 10:26
7 runs, last 7 days