Hydra← All projects

QA metrics

Quality

Production Releases
N/A
Production Defect Leakage
N/A
Average Defect Fix Time
N/A

Test Coverage

1 week2 weeks1 month2 months

Automation Coverage

MusesAcademy Product - Total test case automation coverage by platform · trend over 14 days

All tests

Automated share of all test cases
All platforms
59.76%
59.8%55.6%
09/1023/10
245 of 410 automated
mobile
50.79%
52.4%43.3%
09/1023/10
97 of 191 automated
web
67.58%
67.9%65.5%
09/1023/10
148 of 219 automated

Automatable tests

Coverage counting only tests tagged “automatable”
All platforms
62.76%
63.2%61.1%
09/1023/10
214 of 341 automated
mobile
51.23%
53.2%46.6%
09/1023/10
83 of 162 automated
web
73.18%
75.3%70.4%
09/1023/10
131 of 179 automated

Split by Testomat tag — cases without a platform tag aren't counted. “mobile” covers both iOS and Android.

Test Cases Traction

Draft backlog

Drafts: tagged “draft” now, however old · trend over 14 days
Tests in draft
60
6024
09/1023/10
42 older than 7 days · 0 older than 30 days · oldest 20 days

Updated existing test cases

Share of tests created over 7 days ago, not tagged “draft”, that were edited since · trend over 14 days
All tests
22.91%
53.2%0.4%
09/1023/10
104 of 454 updated, last 7 days

Updated = Testomat's last-edit time falls in the window, so a test edited twice counts once. Changes from an import from code count too.

Out of sync

Automated tests edited in Testomat since their last import from code · trend over 14 days
Automated tests
6
63
09/1023/10
of 262 automated, out of sync with the code

Testomat's own “Out of sync”: the code still has the old version until the test is reimported from code or the edit is rejected in Testomat.

Test Execution

1 week2 weeks1 month2 months

Test runs

MusesAcademy Product - Finished runs labelled web or mobilePlatform (iOS / Android) · last 30 days, trend per 24 h

Runs and success rate

Success rate = passed of the tests that ran in finished runs; skipped tests didn't run and are left out
Runs
All platforms
91
110
24/0923/10
last 30 days
Android
0
00
24/0923/10
last 30 days
iOS
6
10
24/0923/10
last 30 days
web
85
110
24/0923/10
last 30 days
Success rate
All platforms
89.6%
100.0%35.9%
24/0923/10
9,527 of 10,627 tests passed, 204 skipped, last 30 days
Android
—
No tests ran in the last 30 days
iOS
93.8%
100.0%86.3%
25/0921/10
410 of 437 tests passed, 167 skipped, last 30 days
web
89.5%
100.0%35.9%
24/0923/10
9,117 of 10,190 tests passed, 37 skipped, last 30 days

Flaky rate

Flaky = failed, then passed on a retry, of the tests that ran in finished runs
All platforms
0.33%
0.9%0.0%
24/0923/10
35 of 10,627 tests passed only on a retry, last 30 days
Android
—
No tests ran in the last 30 days
iOS
0.00%
0.0%0.0%
25/0921/10
0 of 437 tests passed only on a retry, last 30 days
web
0.34%
0.9%0.0%
24/0923/10
35 of 10,190 tests passed only on a retry, last 30 days

Testomat keeps one result per test even when it was retried: one that failed and then passed is marked “Success after retry” and counts as passed in the success rate too. A suite that doesn't retry failed tests never shows any.

Average run duration

Passed and failed runs labelled Regression / Smoke, launch to finish
regression
Android
—
No runs in the last 30 days
iOS
45.6 min
55.533.6
25/0921/10
5 runs, last 30 days
web
12.9 min
30.54.5
24/0923/10
58 runs, last 30 days
smoke
Android
—
No runs in the last 30 days
iOS
—
No runs in the last 30 days
web
14.7 min
65.43.6
26/0923/10
18 runs, last 30 days