Hydra← All projects

QA metrics

Quality

Production Releases
N/A
Production Defect Leakage
N/A
Average Defect Fix Time
N/A

Test Coverage

1 week2 weeks1 month2 months

Automation Coverage

MusesAcademy Product - Total test case automation coverage by platform · trend over 60 days

All tests

Automated share of all test cases
All platforms
59.76%
59.8%55.6%
07/1023/10
245 of 410 automated
mobile
50.79%
52.4%42.9%
07/1023/10
97 of 191 automated
web
67.58%
67.9%65.5%
07/1023/10
148 of 219 automated

Automatable tests

Coverage counting only tests tagged “automatable”
All platforms
62.76%
62.8%61.1%
07/1023/10
214 of 341 automated
mobile
51.23%
53.2%46.1%
07/1023/10
83 of 162 automated
web
73.18%
75.3%70.4%
07/1023/10
131 of 179 automated

Split by Testomat tag — cases without a platform tag aren't counted. “mobile” covers both iOS and Android.

Test Cases Traction

Draft backlog

Drafts: tagged “draft” now, however old · trend over 60 days
Tests in draft
60
6024
07/1023/10
42 older than 7 days · 0 older than 30 days · oldest 20 days

Updated existing test cases

Share of tests created over 7 days ago, not tagged “draft”, that were edited since · trend over 60 days
All tests
22.91%
53.2%0.4%
07/1023/10
104 of 454 updated, last 7 days

Updated = Testomat's last-edit time falls in the window, so a test edited twice counts once. Changes from an import from code count too.

Out of sync

Automated tests edited in Testomat since their last import from code · trend over 60 days
Automated tests
6
103
07/1023/10
of 262 automated, out of sync with the code

Testomat's own “Out of sync”: the code still has the old version until the test is reimported from code or the edit is rejected in Testomat.

Test Execution

1 week2 weeks1 month2 months

Test runs

MusesAcademy Product - Finished runs labelled web or mobilePlatform (iOS / Android) · last 14 days, trend per 24 h

Runs and success rate

Success rate = passed of the tests that ran in finished runs; skipped tests didn't run and are left out
Runs
All platforms
47
111
10/1023/10
last 14 days
Android
0
00
10/1023/10
last 14 days
iOS
2
10
10/1023/10
last 14 days
web
45
101
10/1023/10
last 14 days
Success rate
All platforms
94.6%
100.0%66.1%
10/1023/10
5,549 of 5,865 tests passed, 6 skipped, last 14 days
Android
—
No tests ran in the last 14 days
iOS
90.1%
93.5%86.3%
20/10 11:2621/10 11:26
155 of 172 tests passed, 2 skipped, last 14 days
web
94.7%
100.0%66.1%
10/1023/10
5,394 of 5,693 tests passed, 4 skipped, last 14 days

Flaky rate

Flaky = failed, then passed on a retry, of the tests that ran in finished runs
All platforms
0.29%
0.7%0.0%
10/1023/10
17 of 5,865 tests passed only on a retry, last 14 days
Android
—
No tests ran in the last 14 days
iOS
0.00%
0.0%0.0%
20/10 11:2621/10 11:26
0 of 172 tests passed only on a retry, last 14 days
web
0.30%
0.7%0.0%
10/1023/10
17 of 5,693 tests passed only on a retry, last 14 days

Testomat keeps one result per test even when it was retried: one that failed and then passed is marked “Success after retry” and counts as passed in the success rate too. A suite that doesn't retry failed tests never shows any.

Average run duration

Passed and failed runs labelled Regression / Smoke, launch to finish
regression
Android
—
No runs in the last 14 days
iOS
54.9 min
55.554.2
20/10 11:2621/10 11:26
2 runs, last 14 days
web
10.8 min
13.58
10/1023/10
31 runs, last 14 days
smoke
Android
—
No runs in the last 14 days
iOS
—
No runs in the last 14 days
web
9.7 min
214.3
13/1023/10
12 runs, last 14 days